Multispectral Pedestrian Detection in Autonomous Driving: A Review

Abstract

Multispectral pedestrian detection that utilizes information fusion of visible-light and thermal cameras has attracted much attention in the autonomous driving field. Pedestrians detected are highlighted in thermal images due to radiated temperatures captured by thermal cameras, building a more effective pedestrian detector in various illumination environments. This paper introduces several recent multispectral pedestrian detection techniques, and details their various problems. Four benchmark datasets are discussed, and performance evaluation metrics are presented. Furthermore, this paper discusses interesting observations on the experiments and the potential research directions for multispectral pedestrian detection problems. Finally, this review concludes that, although various multispectral pedestrian detection methods have been proposed, there still exist several future challenges in the multispectral pedestrian detection task, such as detecting small pedestrians, combining infrared and visible-light images at different resolutions, detecting occluded pedestrians, and dealing with the trade-off between detection accuracy and speed. Therefore, research in the multispectral pedestrian–detection area is expected to expand significantly in the coming years. Insightful discussions and prospective future work are delivered by this review, which can serve as a reference for researchers in multispectral pedestrian detection and related fields.