Pedestrian detection based on light perception fusion of visible and thermal images

Abstract

Pedestrian detection is a fundamental function of many intelligent systems, which can be addressed by multimodal fusion technologies. This paper proposed a novel visible and thermal image fusion approach based on light perception. In order to solve the unexplainable problem of neural network fusion algorithms, the fusion technology used in this study was based on the outputs from single sensor detectors. A light perception module was developed to detect the lighting situation to guide the fusion process. A detector selection strategy was designed to improve the robustness of the model. Different improvement components including gated unit, parameter optimization, soft treatment, and parameter adaption, were taken to enhance the sensor fusion performance. The publicly available KAIST dataset was used to examine the detection performance of our proposed approach. The experimental results show that our proposed method can outperform the compared state-of-the-art methods with lower log-average miss rate and good generalizability.