Network For Crowd Counting Research Articles

Density-aware and background-aware network for crowd counting via multi-task learning

• We propose a density-aware and background-aware multi-task learning network for crowd counting. • A multi-task joint loss is proposed for our multi-task learning network. • Extensive experiments are conducted on three crowd counting benchmark datasets and promising results are achieved. In this paper, we propose a density-aware and background-aware network via multi-task learning (MTL-DB) for crowd counting. It aims to enable the model to capture the high-level semantic information of density and background via multi-task joint training, which may jointly optimize the generation of density maps. Initially, MTL-DB utilizes the first ten layers of VGG-16 with Batch Normalization as the front-end to extract primary features which will be shared by all tasks. Then, a multi-task back-end is constructed by integrating the main task of density map estimation with two auxiliary tasks, i.e., density classification and background segmentation. The density classification auxiliary task captures the density-related information with a fully connected classifier, while the background segmentation auxiliary task applies dilated convolutional network to distinguish the head area of pedestrians and background. With high-level semantic awareness, the main task generates estimated density maps utilizing normal convolutional layers. Furthermore, a multi-task joint loss is proposed to improve the quality of estimated density maps. Extensive experiments on three challenging crowd datasets (ShanghaiTech Part A & B, UCF_CC_50, and UCF_QNRF) verified the effectiveness of this multi-task learning model. MTL-DB outperformed other multi-task learning methods on the ShanghaiTech dataset, both Part A and Part B.

Attention guided feature pyramid network for crowd counting

Crowd counting has become a hot topic because of its wide applications in video surveillance and public security. However, one main problem of the deep learning methods for crowd counting is that the location information about the crowd is degraded irreversibly due to the spatial down-sampling of convolutional neural networks, which degrades the quality of generated density maps. To remedy the above problem, we propose an attention guided feature pyramid network (AG-FPN) for crowd counting, which can adaptively generate a high-quality density map with accurate spatial locations by combining the high- and low-level features. An attention block is added to each encoder layer to further emphasize the crowd regions and suppress the background clutters in feature extraction. Experimental results on the ShanghaiTech, UCF_CC_50, WorldExpo’10 and UCF-QNRF datasets demonstrate the superiority of the proposed method over state-of-the-art approaches.

Context-aware pyramid attention network for crowd counting

Context-aware pyramid attention network for crowd counting

CDADNet: Context-guided dense attentional dilated network for crowd counting

Crowd counting is a conspicuous task in computer vision owing to scale variations, perspective distortions, and complex backgrounds. Existing research usually adopts the dilated convolution network to enlarge the receptive fields to solve the problem of scale variations. However, these methods easily bring background information into the large receptive fields to generate poor quality density maps. To address this problem, we propose a novel backbone called Context-guided Dense Attentional Dilated Network (CDADNet). CDADNet contains three components: an attentional module, a context-guided module and a dense attentional dilated module. The attentional module is used to provide attention maps which can remove background information, while the context-guided module is proposed to extract multi-scale contextual information. Moreover, the dense attentional dilated module aims to generate high-granularity density maps and the cascaded strategy is used to preserve information from changing scales. To verify the feasibility of our method, we compare it to the existing approaches on five crowd counting datasets (ShanghaiTech (Part_A and Part_B), WorldEXPO’10, UCSD, UCF_CC_50). The comparison results demonstrate that CDADNet is effective and robust for various scenes.

EDENet: Elaborate density estimation network for crowd counting

For the CNN-based density estimation approaches in the field of crowd counting, how to generate a high-quality density map with accurate counting performance and detailed spatial description is still an open question. In this paper, to tackle the aforementioned contradiction, we propose an end-to-end trainable architecture called Elaborate Density Estimation Network for Crowd Counting (EDENet), which can gradually generate high-quality density estimation maps based on distributed supervision. Specifically, EDENet is composed of Feature Extraction Network (FEN), Feature Fusion Network (FFN), Double-Head Network (DHN) and Adaptive Density Fusion Network (ADFN). The FEN adopts VGG as the backbone network and employs Spatial Adaptive Pooling (SAP) to extract coarse-grained features. The FFN can effectively fuse contextual information and localization information for enhancing the spatial description ability of fine-grained features. In the DHN, the Density Attention Module (DAM) can provide attention masks of foreground-background, thereby urging the Density Regression Module (DRM) to focus on the pixels around the head annotations to regress density maps with different resolutions. The ADFN constructed on the basis of the adaptive weighting mechanism can directly introduce coarse-grained density representation into high-resolution density maps to strengthen the commonality and dependency among density maps. Extensive experiments on four benchmark crowd datasets (the ShanghaiTech, the UCF-QNRF, the JHU-CRWORD++ and the NWPU-Crowd) indicate that EDENet can achieve state-of-the-art recognition performance and high robustness. Not only that, the density map with the highest Peak Signal to Noise Ratio (PSNR) can be considered to be of high quality.

CFFNet: Coordinated feature fusion network for crowd counting

Long skip connection or encoder-decoder networks for crowd counting have proven to be effective methods to generate high-resolution density maps. However, the simple and coarse feature fusion ignores the disharmony between features, that is, spatial misalignment and semantic inconsistency, which will weaken feature representation and degrade network performance. In this paper, we propose an end-to-end trainable architecture called Coordinated Feature Fusion Network (CFFNet) to tackle the aforementioned problems. The proposed model contains a powerful baseline network and embeds two primary modules: Spatial Alignment Module (SAM) and Semantic Consistency Module (SCM). Specifically, the SAM can learn the transformation offset of pixels to alleviate the spatial misalignment caused by the feature resolution difference; the SCM based on the multi-scale attention mechanism can capture pixel-wise weight to alleviate the semantic inconsistency due to the feature level gap. Extensive experiments on four benchmark crowd datasets (the ShanghaiTech, the UCF-QNRF, the JHU-CRWORD++ and the NWPU-Crowd), indicate that CFFNet can achieve state-of-the-art counting performance and high robustness.

MFP‐Net: Multi‐scale feature pyramid network for crowd counting

Although deep learning has been widely used for dense crowd counting, it still faces two challenges. Firstly, the popular network models are sensitive to scale variance of human head, human occlusions, and complex background due to repeated utilization of vanilla convolution kernels. Secondly, the vanilla feature fusion often depends on summation or concatenation, which ignores the correlation of different features leading to information redundancy and low robustness to background noise. To address these issues, a multi-scale feature pyramid network (MFP-Net) for dense crowd counting is proposed in this paper. The proposed MFP-Net makes two contributions. Firstly, the feature pyramid fusion module is designed that adopts rich convolutions with different depths and scales, not only to expand the receptive field, but also to improve the inference speed of models by using parallel group convolution. Secondly, a feature attention-aware module is added in the feature fusion stage. The module can achieve local and global information fusion by capturing the importance of the spatial and channel domains to improve model robustness. The proposed MFP-Net is evaluated on five publicly available datasets, and experiments show that the MFP-Net not only provides better crowd counting results than comparative models, but also requires fewer parameters.

An improved deep multiscale crowd counting network with perspective awareness

An improved deep multiscale crowd counting network with perspective awareness

Pyramid-dilated deep convolutional neural network for crowd counting

Pyramid-dilated deep convolutional neural network for crowd counting

Congested Crowd Counting via Adaptive Multi-Scale Context Learning.

In this paper, we propose a novel congested crowd counting network for crowd density estimation, i.e., the Adaptive Multi-scale Context Aggregation Network (MSCANet). MSCANet efficiently leverages the spatial context information to accomplish crowd density estimation in a complicated crowd scene. To achieve this, a multi-scale context learning block, called the Multi-scale Context Aggregation module (MSCA), is proposed to first extract different scale information and then adaptively aggregate it to capture the full scale of the crowd. Employing multiple MSCAs in a cascaded manner, the MSCANet can deeply utilize the spatial context information and modulate preliminary features into more distinguishing and scale-sensitive features, which are finally applied to a 1 × 1 convolution operation to obtain the crowd density results. Extensive experiments on three challenging crowd counting benchmarks showed that our model yielded compelling performance against the other state-of-the-art methods. To thoroughly prove the generality of MSCANet, we extend our method to two relevant tasks: crowd localization and remote sensing object counting. The extension experiment results also confirmed the effectiveness of MSCANet.

Average up-sample network for crowd counting

The task of crowd counting is receiving increased attention recently, but it still faces many challenges, such as extremely dense scene, scale variation and background clutter. The quality of generated density map plays an important role in counting performance. In this paper, we propose an encoder-decoder structure network called Average Up-sample Convolution Neural Network (AU-CNN), for high-quality density map and accurate counting estimation. The encoder extracts the features of input image while the decoder gradually recovers the size of feature map to the original size of input image by developing a simple but effective average up-sample module. The average up-sample module takes the average of interpolation results from three different up-sample methods, without adding any other redundant parameters. Moreover, compared with most existing counting algorithm using only Euclidean loss, we use a combined loss function of Euclidean loss and count loss to optimize the network, which is demonstrated effective in performance improving. Experiments on the ShanghaiTech, UCF_CC_50, and UCF_QNRF demonstrate the great counting performance and robustness of our proposed method.

Cascade-guided multi-scale attention network for crowd counting

The performance of crowd counting based on density estimation has been greatly improved with the development of deep learning. However, it is still a major issue to obtain high-quality density map due to the clutter of background, as well as the interference of perspective changes within and between scenes. In this paper, we propose a cascade-guided crowd counting network, which is mainly embedded with scale aware model (SAM) and attention aware model (AAM). First, SAM considers share-net design and multi-directional perspective transform in convolution to deal with multi-scale varying and smooth transition, while reducing the background noise in shallow features. Second, AAM further encodes the semantic inter dependencies by using the two-dimensional features of location and channel in order to let the network learn to pay attention to the key information. Finally, the global and local features are concatenated and taken into decoder to generate the estimated density map for crowd counting. Comprehensive experiments based on three established datasets show that the proposed method not only has higher accuracy, but also has stronger robustness to scale variation and background noise.

Adversarial scale-adaptive neural network for crowd counting

Crowd counting aims to count the number of pedestrians in an image or a video. Currently, scale variations in crowd counting are inevitable and challenging in practice. Besides, for the crowd counting task, there is only a small amount of annotated data available. It can be seen from previous methods of crowd counting and object detection that the two tasks have similar attention areas. However, existing methods in crowd counting generally ignore the similarities and specialties between the crowd counting task and the object detection task. In this paper, in order to solve the above challenges, we propose an adversarial scale-adaptive neural network (ASANet), consisting of three branches. First, a private branch for the crowd counting task concentrates on generating high-quality density maps. Second, another private branch for the object detection task aims to correctly detect and recognize objects. Third, we design a common branch to learn the similar attention area of the two tasks and assist crowd counting. Experimental results demonstrate an outstanding performance of the ASANet over state-of-the-art methods on three public datasets (ShanghaiTech, UCF_CC_50, and UCF_QNRF).

MSR‐FAN: Multi‐scale residual feature‐aware network for crowd counting

MSR‐FAN: Multi‐scale residual feature‐aware network for crowd counting

Multi‐level feature fusion network for crowd counting

Crowd counting has become a noteworthy vision task due to the needs of numerous practical applications, but it remains challenging. State-of-the-art methods generally estimate the density map of the crowd image with the high-level semantic features of various deep convolutional networks. However, the absence of low-level spatial information may result in counting errors in the local details of the density map. To this end, a novel framework named Multi-level Feature Fusion Network (MFFN) for single image crowd counting is proposed. The proposed MFFN, which is constructed in an encoder–decoder fashion, incorporates semantic and spatial information for generating high-resolution density maps of input crowd images. Skip connections are developed between the encoder and the decoder so that low-level spatial information and high-level semantic features can be combined by element-wise addition. In addition, a dense dilated convolution block is placed behind the encoder, extracting multi-scale context features to guide feature fusion by a channel attention mechanism. The model is trained by multi-task learning; semantic segmentation supervision is introduced to enhance feature representation. Extensive experiments are conducted on three crowd counting datasets (ShanghaiTech, UCF_CC_50, UCF-QNRF), and the results show that MFFN outperforms state-of-the-art methods. In addition, sufficient ablation studies are performed to verify the effectiveness of each component in our proposed method.

Crowd Density Estimation based on Global Reasoning

The problem of crowd counting in single images and videos has attracted more and more attention in recent years. The crowd counting task has made massive progress by now due to the Convolutional Neural Network (CNN). However, filters in the shallow convolutional layer of the CNN only model the local region rather than the global region, which cannot capture context information from the crowd scene efficiently. In this paper, we propose a Graph-based Global Reasoning (GGR) network for crowd counting to solve this problem. Each input image is processed by the VGG-16 network for feature extracting, and then the GGR Unit reasons the context information from the extracted feature. Especially, the extracted feature firstly is transformed from the feature space to the interaction space for global context reasoning with the Graph Convolutional Network (GCN). Then, the output of the GCN projects the context information from the interaction space to the feature space. The experiments on the UCF-QNRF dataset demonstrate the effectiveness of the proposed method.

Multi‐scale supervised network for crowd counting

Crowd counting is getting more and more attention in our daily life, because it can effectively prevent some safety problems. However, due to scale variations and background noise in the image, such as buildings and trees, getting the accurate number from image is a hard work. In order to address these problems, this work introduces a new multi-scale supervised network. The proposed model uses part of vgg16 model as the backbone to extract feature. In the training process, a multi-scale dilated convolution module is added at the end of each stage of the backbone network to generate attention map with different resolutions to help the model focus on the head area in feature map. In addition, the dilated convolution adopts three dilation ratios to fit different sizes of head in the image. Finally, in order to get the high-quality density map with high-resolution, the authors employ the upsampling operation to restore the density map size to the quarter size of original image. A large number of experiments on these four datasets show that the proposed network has greatly improved the counting accuracy of many existing methods.

Adaptive weighted crowd receptive field network for crowd counting

Crowd counting plays an important role in crowd analysis and monitoring. To this end, we propose a novel method called Adaptive Weighted Crowd Receptive Field Network (AWRFN) for crowd counting to estimate the number of people and the spatial distribution of input crowd images. The proposed AWRFN is composed of four modules: backbone, crowd receptive field block (CRFB), recurrent block (RB), and channel attention block (CAB). Backbone utilizes the first ten layers of VGG16 to extract base features of input images. CRFB is a multi-branch architecture simulating a real human visual system for further obtaining refined and discriminative crowd features. RB generates strong semantic and global information by recurrently stacking convolutional layers with the same parameters. CAB outputs appropriate weights to supervise each channel of the feature maps output from CRFB, which uses the outputs of RB as guidance. Different from previous works using Euclidean Loss, we employ L1_Smooth Loss to train our network in an end-to-end fashion. To demonstrate the effectiveness of our proposed method, we implement AWRFN on two representative datasets including the ShanghaiTech dataset and the UCF_CC_50 dataset. The experimental results prove that our method is both effective and robust compared with the state-of-the-art approaches.

A multi-scale and multi-level feature aggregation network for crowd counting

Recently, crowd counting has drawn widespread attention in computer vision, but it is extremely challenging because of the varying scales and densities. Many existing methods focus on improving the multi-scale representation by utilizing multi-column or multi-branch architectures with different kernel sizes. However, such networks cannot extract the feature maps with large receptive fields due to limitation of depth. In addition, the importance of utilizing the multi-level feature information in a deep network is ignored. In this paper, we propose a multi-scale and multi-level features aggregation network (MFANet) for accurate and efficient crowd counting, and it can be trained by end-to-end. A vital component of the network is the scale and level aggregation module (SLAM), which can extract multi-scale features and make full use of multi-level feature information for more accurate estimation. When six SLAMs are stacked together and applied to our network, our method can achieve the best performance. Furthermore, we introduce a new loss function called normalized Euclidean loss (NEL) to balance the contribution of all samples to network training. To demonstrate the performance of the proposed method, extensive experiments are conducted on four benchmark crowd counting datasets, including ShanghaiTec Part A/B, UCF-CC-50, Mall, and UCF-QNRF. Experimental results show that our MFANet achieves state-of-the-art performance in crowd counting and crowd localization.

The Multi-channel and Multi-scale Network for Crowd Counting

In this paper, we propose a novel multi-channel and multi-scale network for processing crowds in crowded scenarios and improving counting accuracy. In the estimated crowd count study, different distribution groups have different contributions to the total number of crowd, and the more crowded people have stricter requirements on details. Therefore, we designed two branches in the crowd counting network: the backbone network performs feature extraction operations on the original image, which mainly obtains effective information from the global, and our branch network focuses on the crowd gathering area, which better focuses on the details of the crowd distribution. Finally, the global information is complemented with local details to obtain high-quality feature expressions. To deal with scale changes, Inspired by atrous spatial pyramid pooling structures, we introduce dilated convolution with different sampling rates in the network to expand the receptive field. We carried out a large number of experimental verifications on popular data sets, and the proposed method is superior to existing methods.