Attention-Based Cross-Modality Feature Complementation for Multispectral Pedestrian Detection

Abstract

Multispectral pedestrian detection based on deep learning can provide a robust and accurate detection under different illumination conditions, which has important research significance in safety. In order to reduce the log-average miss rate of the object under different illumination conditions, a new one-stage detector suitable for multispectral pedestrian detection is proposed. First, in order to realize the complementarity between the information flows of the two modalities in the feature extraction stage to reduce the object loss, a low-cost cross-modality feature complementary module (CFCM) is proposed. Second, in order to suppress the background noise in different environments and enhance the semantic information and location information of the object, so as to reduce the error detection of the object, an attention-based feature enhancement fusion module (AFEFM) is proposed. Thirdly, through the feature complementarity of color-thermal image pair and the multi-scale fusion of depth feature layer, the horizontal and vertical multi-dimensional data mining of parallel deep neural network is realized, which provides effective data support for object detection algorithm. Finally, through the reasonable arrangement of proposed modules, a robust multispectral detection framework is proposed. The experimental results on the Korea Advanced Institute of Science and Technology (KAIST) pedestrian benchmark show that the proposed method has the lowest log-average miss rate compared with other state-of-the-art multispectral pedestrian detectors, and has a good balance in speed and accuracy.