Multimodality pedestrian detection based on YOLOv5

Abstract

A pedestrian detection method based on YoloV5 RGB and thermal image data fusion is proposed in this paper. This method uses two yolov5 branch structures to learn the features of RGB and thermal image data respectively, and finally uses multimodal features for fusion detection. It includes the following stages. Firstly, we use yolov5 network as a branch network to learn features from paired RGB and thermal image data. The two yoloV5-based backbone networks extract the features of the two modes for preliminary fusion, and then obtain the importance parameters of the two modes through feature compression and extraction. The effective information is enhanced, and redundant information is eliminated by multiplying the initial features. Finally, the fusion feature is used for target detection. Through this method, the detection effect is improved. We have done a lot of experiments on the public KAIST and VOT2019 pedestrian data set, and the experiments show that our method is better than the advanced method.