Network For Crowd Counting Research Articles

Multi-Scale and spatial position-based channel attention network for crowd counting

Crowd counting algorithms have recently incorporated attention mechanisms into convolutional neural networks (CNNs) to achieve significant progress. The channel attention model (CAM), as a popular attention mechanism, calculates a set of probability weights to select important channel-wise feature responses. However, most CAMs roughly assign a weight to the entire channel-wise map, which makes useful and useless information being treat indiscriminately, thereby limiting the representational capacity of networks. In this paper, we propose a multi-scale and spatial position-based channel attention network (MS-SPCANet), which integrates spatial position-based channel attention models (SPCAMs) with multiple scales into a CNN. SPCAM assigns different channel attention weights to different positions of channel-wise maps to capture more informative features. Furthermore, an adaptive loss, which uses adaptive coefficients to combine density map loss and headcount loss, is constructed to improve network performance in sparse crowd scenes. Experimental results on four public datasets verify the superiority of the scheme.

A point and density map hybrid network for crowd counting and localization based on unmanned aerial vehicles

ABSTRACT Crowd counting and localisation are essential tasks in crowd analysis and are vital to ensure public safety. However, these tasks via UAV bring new obstacles compared with video surveillance (e.g. viewpoint and scale variations, background clutter, and small scales). To overcome the difficulties, this research presents a novel network named PDNet. It employs the multi-task learning approach to combine the point regression and density map regression. PDNet includes a backbone to extract multi-scale features, a Dilated Feature Fusion module (DFF), a Density Map Attention module (DMA), a density map branch and a point branch. Aims of DFF is to address the difficulties of small targets and scale variations by establishing relationships between targets and their surroundings. DMA is created to address the challenges of complicated backgrounds, allowing the PDNet to focus on the target's location. In addition, the density map branch and point branch are designed for density maps regression and point regression, respectively. Experiments on the DroneCrowd dataset demonstrate that our proposed network outperforms state-of-the-art approaches in terms of localisation, L-mAP (53.85%), L-AP@10 (59.14%), L-AP@15 (63.64%), and L-AP@20 (66.21%), and we improved counting performance and significantly reduced inference time. In addition, ablation experiments are conducted to prove the modules' effectiveness.

Lw-Count: An Effective Lightweight Encoding-Decoding Crowd Counting Network

Crowd counting is a task of intelligent applications, and its operation efficiency is very important. However, in order to obtain a better counting performance, most of the existing works often design larger and more complex network structures, which will result in them occupying considerable memory, time and other resources at runtime, seriously limiting their deployment scope and making it difficult to be widely used in practical scenarios. In this paper, in order to overcome the above problems, we propose an effective lightweight encoding-decoding crowd counting network, named Lw-Count. Specifically, in the encoding process, we design an efficient and lightweight convolution module (ELCM), which extracts the crowd features in the network through a refined ghost block to reduce the network parameters and computing cost, and solves the problem of inaccurate counting caused by uneven crowd distribution through spatial group normalization (SGN). In the decoding process, we design a scale regression module (SRM) to reduce the error details and chessboard effect caused by linear interpolation and deconvolution. In addition, we design a new loss function, which enhances the spatial correlation of the density map and the sensitivity of the crowd through a regional normalized cross-correlation loss and counting loss, to ensure the counting accuracy. Extensive experiments on five mainstream datasets demonstrate that Lw-Count achieves an optimal trade-off between counting performance and running speed compared with other methods.

MACC Net: Multi-task attention crowd counting network

MACC Net: Multi-task attention crowd counting network

Semantic-refined spatial pyramid network for crowd counting

• We design a novel semantic-refined spatial pyramid network (SSPNet). • The SSPNet can benefit the fusing of multi-scale information. • The SSPNet can enhance the representation ability of deep semantic features. • The multi-scale feature will be adaptively integrated by our method. • The texture features are obtained to enrich the multi-scale features. In this paper, we propose a novel encoder-decoder model called Semantic-refined Spatial Pyramid Network (SSPNet) for generating high-quality density maps, which aims to build a scale-aware counting network to estimate the number of crowds accurately. The SSPNet consists of the front-end based on VGG-16, spatial pyramid multi-scale module (SPMM), and semantic enhancement module (SEM). First, a series of convolutional neural layers are utilized as the front-end to get deeper features without the extra computational cost. Moreover, the SPMM, which has a spatial pyramid structure with multiple receptive fields, is employed to capture multi-scale features. Furthermore, the SEM is designed to refine the features captured by SPMM, which uses deep semantic information to better integrate multi-scale features. Finally, the shallow texture information is adopted to compensate for the detail of the feature map to enhance the quality of the density map. Extensive experiments and comparisons on three challenge datasets, including ShanghaiTech Part_A & Part_B, UCF_CC_50, and UCF-QNRF, illustrate the superiority of our method.

Spatial-Frequency Attention Network for Crowd Counting.

Counting the number of people in crowded scenarios is a crucial task in video surveillance and urban security system. Widely deployed surveillance cameras provide big data for training, a compelling deep learning-based counting network. However, large-scale variations in dense crowds are still not entirely solved. To address this problem, we propose a spatial-frequency attention network (SFANet) for crowd counting in this article. A bottleneck spatial attention module is built to emphasize features in various spatial locations and select a region containing individuals adaptively in the spatial domain. As a complementary, in the frequency domain, a multispectral channel attention module is adopted to obtain a more complete set of frequency components for representing each channel. The two attention modules are combined to focus on the discriminative region and suppress the misleading information by their mutual promotion. Experimental results on five benchmark crowd data sets demonstrate that the SFANet can achieve the state-of-the-art performance in terms of accuracy and robustness.

Group-split attention network for crowd counting

Crowd counting is a considerable yet challenging task in intelligent video surveillance and urban security systems. The performance has been significantly boosted along with the springing up of the convolutional neural networks (CNNs). However, accurate and efficient crowd counting in congested scenes remains under-explored due to scale variation and cluttered background. To address these problems, we propose a biologically inspired crowd counting method named group-split attention network (GSANet). The GSANet consists of three principal modules, namely GS module, dual-aware attention module, and aggregation module. The GS module processes the subfeatures of each group in parallel, and groups the input feature map to reduce the computational cost. The dual-aware attention module synergies the spatial and channel dimensional information to alleviate the estimation error in background regions. The aggregation module adopts a learning-based cross-group strategy to aggregate and facilitate the fusion of feature maps along different channel dimensions. Extensive experimental results on five benchmark crowd datasets demonstrate that the GSANet achieves superior performances in terms of accuracy and efficiency.

Cascaded parallel crowd counting network with multi-resolution collaborative representation.

Accurately estimating the size and density distribution of a crowd from images is of great importance to public safety and crowd management during the COVID-19 pandemic, but it is very challenging as it is affected by many complex factors, including perspective distortion and background noise information. In this paper, we propose a novel multi-resolution collaborative representation framework called the cascaded parallel network (CP-Net), consisting of three parallel scale-specific branches connected in a cascading mode. In the framework, the three cascaded multi-resolution branches efficiently capture multi-scale features through their specific receptive fields. Additionally, multi-level feature fusion and information filtering are performed continuously on each branch to resist noise interference and perspective distortion. Moreover, we design an information exchange module across independent branches to refine the features extracted by each specific branch and deal with perspective distortion by using complementary information of multiple resolutions. To further improve the robustness of the network to scale variance and generate high-quality density maps, we construct a multi-receptive field fusion module to aggregate multi-scale features more comprehensively. The performance of our proposed CP-Net is verified on the challenging counting datasets (UCF_CC_50, UCF-QNRF, Shanghai Tech A&B, and WorldExpo’10), and the experimental results demonstrate the superiority of the proposed method.

Multi-scale dilated convolution of feature Fusion Network for Crowd counting

Multi-scale dilated convolution of feature Fusion Network for Crowd counting

Information Entropy Augmented High Density Crowd Counting Network

The research proposes an innovated structure of the density map-based crowd counting network augmented by information entropy. The network comprises of a front-end network to extract features and a back-end network to generate density maps. In order to validate the assumption that the entropy can boost the accuracy of density map generation, a multi-scale entropy map extraction process is imported into the front-end network along with a fine-tuned convolutional feature extraction process, In the back-end network, extracted features are decoded into the density map with a multi-column dilated convolution network. Finally, the decoded density map can be mapped as the estimated counting number. Experimental results indicate that the devised network is capable of accurately estimating the count in extremely high crowd density. Compared to similar structured networks which don’t adapt entropy feature, the proposed network exhibits higher performance. This result proves the feature of information entropy is capable of enhancing the efficiency of density map-based crowd counting approaches.

MGSNet: A multi-scale and gated spatial attention network for crowd counting

MGSNet: A multi-scale and gated spatial attention network for crowd counting

COMAL: compositional multi-scale feature enhanced learning for crowd counting

Accurately modeling the crowd’s head scale variations is an effective way to improve the counting accuracy of the crowd counting methods. Most counting networks apply a multi-branch network structure to obtain different scales of head features. Although they have achieved promising results, they do not perform very well on the extreme scale variation scene due to the limited scale representability. Meanwhile, these methods are prone to recognize background objects as foreground crowds in complex scenes due to the limited context and high-level semantic information. We propose a compositional multi-scale feature enhanced learning approach (COMAL) for crowd counting to handle the above limitations. COMAL enhances the multi-scale feature representations from three aspects: (1) The semantic enhanced module (SEM) is developed for embedding the high-level semantic information to the multi-scale features; (2) The diversity enhanced module (DEM) is proposed to enrich the variety of crowd features’ different scales; (3) The context enhanced module (CEM) is designed for strengthening the multi-scale features with more context information. Based on the proposed COMAL, we develop a crowd counting network under the encoder-decoder framework and perform extensive experiments on ShanghaiTech, UCF_CC_50, and UCF-QNRF datasets. Qualitative and quantitive results demonstrate the effectiveness of the proposed COMAL.

Jointly attention network for crowd counting

Nowadays, crowd counting has shown great practical value in public safety and related fields. Most leading algorithms exploit CNN to generate density maps and have improved the estimation accuracy. However, the counting models still suffer from the challenge of huge scale variations. In order to mitigate this issue, we propose a novel approach named Jointly Attention Network (JANet) for Crowd Counting. It is composed of two major schemes: the Multi-order Scale Attention (MSA) module and the Multi-pooling Relational Channel Attention (MRCA) module. The MSA module explores meaningful high-order statistics and helps the backbone network obtain more discriminative features with rich scale information in an explicit manner. The MRCA module compactly represents the global scope relations and accounts the interdependence among all channel-wise nodes, which is complementary to MSA module. Meanwhile, the Distributed Combinatorial Loss (DCL) is designed to achieve the distributed supervision on intermediate layers at each level. Finally, we conduct extensive studies on multiple crowd counting datasets, the ShanghaiTech, the UCF-QNRF, the JHU-CROWD++, the NWPU-Crowd. The experimental results indicate that our proposed method has achieved superior performance.

HRANet: Hierarchical region-aware network for crowd counting.

Aiming to tackle the most intractable problems of scale variation and complex backgrounds in crowd counting, we present an innovative framework called Hierarchical Region-Aware Network (HRANet) for crowd counting in this paper, which can better focus on crowd regions to accurately predict crowd density. In our implementation, first, we design a Region-Aware Module (RAM) to capture the internal differences within different regions of the feature map, thus adaptively extracting contextual features within different regions. Furthermore, we propose a Region Recalibration Module (RRM) which adopts a novel region-aware attention mechanism (RAAM) to further recalibrate the feature weights of different regions. By the integration of the above two modules, the influence of background regions can be effectively suppressed. Besides, considering the local correlations within different regions of the crowd density map, a Region Awareness Loss (RAL) is designed to reduce false identification while producing the locally consistent density map. Extensive experiments on five challenging datasets demonstrate that the proposed method significantly outperforms existing methods in terms of counting accuracy and quality of the generated density map. In addition, a series of specific experiments in crowd gathering scenes indicate that our method can be practically applied to crowd localization.

SGCNet: Scale-aware and global contextual network for crowd counting

SGCNet: Scale-aware and global contextual network for crowd counting

ECCNAS: Efficient Crowd Counting Neural Architecture Search

Recent solutions to crowd counting problems have already achieved promising performance across various benchmarks. However, applying these approaches to real-world applications is still challenging, because they are computation intensive and lack the flexibility to meet various resource budgets. In this article, we propose an efficient crowd counting neural architecture search (ECCNAS) framework to search efficient crowd counting network structures, which can fill this research gap. A novel search from pre-trained strategy enables our cross-task NAS to explore the significantly large and flexible search space with less search time and get more proper network structures. Moreover, our well-designed search space can intrinsically provide candidate neural network structures with high performance and efficiency. In order to search network structures according to hardwares with different computational performance, we develop a novel latency cost estimation algorithm in our ECCNAS. Experiments show our searched models get an excellent trade-off between computational complexity and accuracy and have the potential to deploy in practical scenarios with various resource budgets. We reduce the computational cost, in terms of multiply-and-accumulate (MACs), by up to 96% with comparable accuracy. And we further designed experiments to validate the efficiency and the stability improvement of our proposed search from pre-trained strategy.

Multi-scale Attention Recalibration Network for crowd counting

Crowd counting using deep convolutional neural networks (CNN) has achieved encouraging progress in recent years. Nevertheless, how to efficiently address the problems of scale variation and complex backgrounds remain a major challenge. For this, we present an innovative Multi-scale Attention Recalibration Network termed MARNet for obtaining more accurate crowd counting. This is achieved mainly by introducing and integrating two significant modules into the proposed model. More concretely, a Feature Pyramid Module (FPM) is first designed to achieve multi-scale feature enhancement by utilizing multiple dilated convolutions with different rates, thus providing rich contextual information for subsequent operations. Besides, to adequately take advantage of these contextual information, a Feature Recalibration Module (FRM) is devised by integrating a Dimension Attention (DA) block with a Region Recalibration (RR) block. The DA block is mainly used for modeling the semantic dependencies between different dimensions of contextual information, while the RR block is responsible for reassigning attention weights for different regions based on the semantic dependencies. By the integration of the above two blocks, the proposed method can be targeted to capture the crowd features for accurately estimating crowd density. Extensive experiments on multiple publicly crowd counting datasets well demonstrate that our method significantly outperforms most existing methods in terms of the counting accuracy and the quality of the generated density map.

MSFFA: a multi-scale feature fusion and attention mechanism network for crowd counting

MSFFA: a multi-scale feature fusion and attention mechanism network for crowd counting

AdaCrowd: Unlabeled Scene Adaptation for Crowd Counting

We address the problem of image-based crowd counting. In particular, we propose a new problem called <i>unlabeled scene-adaptive crowd counting</i>. Given a new target scene, we would like to have a crowd counting model specifically adapted to this particular scene based on the target data that capture some information about the new scene. In this paper, we propose to use one or more unlabeled images from the target scene to perform the adaptation. In comparison with the existing problem setups (e.g. fully supervised), our proposed problem setup is closer to the real-world applications of crowd counting systems. We introduce a novel <monospace>AdaCrowd</monospace> framework to solve this problem. Our framework consists of a crowd counting network and a guiding network. The guiding network predicts some parameters in the crowd counting network based on the unlabeled images from a particular scene. This allows our model to adapt to different target scenes. The experimental results on several challenging benchmark datasets demonstrate the effectiveness of our proposed approach compared with other alternative methods. Code is available at <uri>https://github.com/maheshkkumar/adacrowd</uri>

Towards more effective PRM-based crowd counting via a multi-resolution fusion and attention network

The paper focuses on improving the recent plug-and-play patch rescaling module (PRM) based approaches for crowd counting. In order to make full use of the PRM potential and obtain more reliable and accurate results for challenging images with crowd-variation, large perspective, extreme occlusions, and cluttered background regions, we propose a new PRM based multi-resolution and multi-task crowd counting network by exploiting the PRM module with more effectiveness and potency. The proposed model consists of three deep-layered branches with each branch generating feature maps of different resolutions. These branches perform a feature-level fusion across each other to build the vital collective knowledge to be used for the final crowd estimate. Additionally, early-stage feature maps undergo visual attention to strengthen the later-stage channel’s understanding of the foreground regions. The integration of these deep branches with the PRM module and the early-attended blocks proves to be more effective than the original PRM based schemes through extensive numerical and visual evaluations on four benchmark datasets. The proposed approach yields a significant improvement by a margin of 12.6% in terms of the RMSE evaluation criterion. It also outperforms state-of-the-art methods in cross-dataset evaluations.