A novel infrared video surveillance system using deep learning based techniques

Huaizhong Zhang,Chunbo Luo,Qi Wang,Pablo Casaseca-De-La-Higuera,Andrew Parmley,Jesus Monge-Alvarez,Matthew Kitchin

doi:10.1007/s11042-018-5883-y

Abstract

This paper presents a new, practical infrared video based surveillance system, consisting of a resolution-enhanced, automatic target detection/recognition (ATD/R) system that is widely applicable in civilian and military applications. To deal with the issue of small numbers of pixel on target in the developed ATD/R system, as are encountered in long range imagery, a super-resolution method is employed to increase target signature resolution and optimise the baseline quality of inputs for object recognition. To tackle the challenge of detecting extremely low-resolution targets, we train a sophisticated and powerful convolutional neural network (CNN) based faster-RCNN using long wave infrared imagery datasets that were prepared and marked in-house. The system was tested under different weather conditions, using two datasets featuring target types comprising pedestrians and 6 different types of ground vehicles. The developed ATD/R system can detect extremely low-resolution targets with superior performance by effectively addressing the low small number of pixels on target, encountered in long range applications. A comparison with traditional methods confirms this superiority both qualitatively and quantitatively.

Highlights

Infrared thermography (IRT, or thermal video) has been widely used in civilian and military applications such as surveillance, night vision and tracking, weather forecasting, firefighting, facility inspections, etc. for collecting high quality image data that is beyond the human visual perception range
This paper presents an infrared video based surveillance system consisting of a resolution-enhanced automatic object detection/recognition (ATD/R) system that can be widely used in various civilian and military applications
The developed ATD/R system is evaluated with the collected Trial 1 (T1) and Trial 2 (T2) datasets

Summary

Introduction

Infrared thermography (IRT, or thermal video) has been widely used in civilian and military applications such as surveillance, night vision and tracking, weather forecasting, firefighting, facility inspections, etc. for collecting high quality image data that is beyond the human visual perception range. Recent advances in IRT cameras have significantly improved the resolution and bit-depth of thermal images, which had previously often been considered inferior to visual images, thereby making IRT images suitable and widely used in scenarios containing high value targets, including remote surveillance applications where distant vehicles, pedestrians or buildings are monitored For this reason, automatic detection and recognition of these targets has raised increasing interest in both academia and industry [17]. Despite the advances in acquisition technology, long range object detection and recognition in IRT images collected under real-world settings is still a challenging research topic Such images are usually acquired at a very long distance, leading to extremely low numbers of pixels on target.

Results

Discussion

Conclusion