IR-SETNET: SPARSITY AWARE ENSEMBLE NETWORK FOR INFRARED IMAGING BASED DRONE LOCALIZATION AND TRACKING IN DISTORTED SURVEILLANCE VIDEOS

Although Unmanned Aerial Vehicles (drones) have been extensively deployed for various applications, unauthorized operations can cause serious security threats and infringement of privacy. Target (drone) localization plays a crucial role in military operations that is extremely challenging under low visibility conditions, uneven illumination, weak contrast, background distortions and close resemblances to other flying entities such as birds. In such cases, infrared imaging outperforms RGB imaging since the images in the former case are based on temperature differences and they help identify and track objects that are otherwise not visible in images from RGB cameras. In this paper, a sparse ensemble network is proposed for infrared drone localization combined with a CLIP-based zero shot tracking network for tracing the trajectory of the target. Static pruning and model quantization are adopted for introducing sparsity in the ensemble network. This is further combined with a vision transformer for multi-class distortion classification in real time surveillance videos. The end-to-end pipeline is termed as IR-SETNET which achieves an improvement of 25% in target localization with a mean average precision (mAP) score of 99.90%, drone tracking accuracy of 95.60%, mean distortion classification accuracy of 89.75% and a five-fold improvement in inference speed on introduction of sparsity.