Night Vision Surveillance: Object Detection using Thermal and Visible Images

Abstract

Surveillance has become an important task in recent time mainly due to the increasing of crime rates. The existing research on surveillance for day time has achieved better performance by detecting and tracking objects using deep learning algorithms. However, it is difficult to achieve the same performance for night vision mainly due to low illumination and/or bad weather situation. One of the important tasks in surveillance is object detection which results in both class and location of the detected object with clear boundary of the objects from the image. We propose an efficient object detection module using fusion of thermal and visible images. Fusion module consists of encoder-decoder network in which encoder uses depthwise convolution to extracts efficient features from the given thermal and visible images. Then after, fused image is reconstructed using convolutional layers and final map is utilized in object detection algorithm (i.e., mask RCNN). The proposed method shows the effectiveness of utilization of pre-processing module i.e., fusion in object detection algorithm. Here, it is observed that the proposed method performs better for night vision when images are trained carefully with various features. Moreover, proposed method performs better on real time night vision images having no illumination condition.