Foreground Detection for Infrared Videos With Multiscale 3-D Fully Convolutional Network

Abstract

Foreground detection for infrared (IR) videos is an important and fundamental problem in many applications, e.g., IR surveillance, IR object tracking, and so on. Conventional foreground detection algorithms developed for visible videos do not focus on the problems for IR videos, e.g., low contrast, coarse texture, lack of color information, and so on. Recent foreground detection methods based on deep neural network (DNN) demonstrated significant improvement, but most of them still use only spatial features, which is less obvious in IR images. In this letter, we add deeply learned multiscale temporal features to improve the performance of background subtraction for IR videos. We propose a novel multiscale 3-D fully convolutional network (MFC3-D) to establish a mapping from image sequences to pixelwise classification results and to learn deep and hierarchical multiscale spatial–temporal features of the input images sequence. The experimental results show that the MFC3-D can learn spatial–temporal features effectively and achieved state-of-the-art results on the test data set, comparing to other DNN-based methods and traditional background subtraction methods.