Multi-spectral fusion network for full-time robust pedestrian detection

Abstract

An enhanced YOLOv4 multi-spectral fusion pedestrian detection approach is proposed to address the problem of fusion network robustness in pedestrian detection. This method can effectively and accurately complete pedestrian detection. First, the feature extraction backbone is upgraded, and channel attention Module CAM and spatial attention Module SAM are added to the feature extraction backbone to allow for adaptive feature layer adjustment. Fusion processing based on channel and pixel direction is performed on the altered feature layer, and then the fusion layer is anticipated. The experiment is performed on the KAIST Dataset, and the capacity to generalize was assessed using the OTCBVS Benchmark Dataset. The proposed multi-spectral fusion detection approach is effective, according to the experimental results. The log-average miss rate (MR) reaches 11.03 and 8.79 throughout the full day and night when the false positive per image (FPPI) is 10-2~100 , and it also achieves good detection performance during the day. The proposed multi-spectral fusion detection approach is universal in various data sets, according to the generalization ability analysis experiment. Pedestrian detection accuracy may be accomplished adaptively regardless of whether it is daytime or nighttime detection, and speed is substantially enhanced.