CFFNet: Coordinated feature fusion network for crowd counting

Abstract

Long skip connection or encoder-decoder networks for crowd counting have proven to be effective methods to generate high-resolution density maps. However, the simple and coarse feature fusion ignores the disharmony between features, that is, spatial misalignment and semantic inconsistency, which will weaken feature representation and degrade network performance. In this paper, we propose an end-to-end trainable architecture called Coordinated Feature Fusion Network (CFFNet) to tackle the aforementioned problems. The proposed model contains a powerful baseline network and embeds two primary modules: Spatial Alignment Module (SAM) and Semantic Consistency Module (SCM). Specifically, the SAM can learn the transformation offset of pixels to alleviate the spatial misalignment caused by the feature resolution difference; the SCM based on the multi-scale attention mechanism can capture pixel-wise weight to alleviate the semantic inconsistency due to the feature level gap. Extensive experiments on four benchmark crowd datasets (the ShanghaiTech, the UCF-QNRF, the JHU-CRWORD++ and the NWPU-Crowd), indicate that CFFNet can achieve state-of-the-art counting performance and high robustness.