Dilated high-resolution network driven RGB-T multi-modal crowd counting

Abstract

Crowd counting aims to estimate the number of pedestrians in a scene. However, the problems of insufficient illumination and large-scale variation affect the accuracy of crowd counting. In this paper, a dilated high-resolution network (DHRNet) driven RGB-T multi-modal crowd counting model is proposed to address the above problems. In terms of the importance of RGB and thermal modalities, a thermal-main and RGB-auxiliary strategy is chosen instead of treating them equally as in previous works. In terms of the fusion of RGB and thermal modalities, a cross-modal fusion module is designed and embedded in the input feature level of DHRNet. In terms of DHRNet output feature utilization, a multilayer perceptron regression head is proposed to predict high-quality density maps. The experimental results on public datasets show that our proposed network significantly outperforms state-of-the-art methods and reaches a new level of performance. The ablation studies verify the effectiveness of the thermal-main and RGB-auxiliary strategy and the proposed modules.