Dense Crowd Counting Research Articles

CrowdUNet: Segmentation assisted U-shaped crowd counting network

With the end of the COVID-19 pandemic, the number of pedestrians in various public places has increased dramatically. Estimating the size and density distribution of crowds accurately from images is essential for public safety. At present, there are still many factors that limit the accuracy of dense crowd counting, such as perspective distortion, background clutter and heavy occlusion. To be capable of accurately estimating the crowd size in RGB images, we propose a crowd counting network called CrowdUNet, which is assisted by a segmentation task. It applies the segmentation results to the crowd counts, making the network more focused on the prediction of foreground regions. We combine Swin Transformer Block and CNN to build a Swin Transformer Convolution(STC) module to extract deep semantic features. We analyze the characteristics of crowd images and propose a novel decoder structure called Coordinate Decoder(CD), which better aggregates low and high level features and improve the robustness of the network. In order to obtain accurate regression results, we also propose a regression head with multi-scale receptive fields, which is called Spatial Pyramid Convolution (SPC). Extensive experiments on four challenging crowd counting datasets namely ShanghaiTech A, ShanghaiTech B, UCF p=CC 50, and UCF-QNRF have validated the proposed method.

Mask focal loss: a unifying framework for dense crowd counting with canonical object detection networks

Mask focal loss: a unifying framework for dense crowd counting with canonical object detection networks

A Fusion-Based Dense Crowd Counting Method for Multi-Imaging Systems

Dense crowd counting has become an essential technology for urban security management. The traditional crowd counting methods mainly apply to the scene with a single view and obvious features but cannot solve the problem with a large area and fuzzy crowd features. Therefore, this paper proposes a crowd counting method based on high and low view information fusion (HLIF) for large and complex scenes. First, a neural network based on an attention mechanism (AMNet) is established to obtain a global density map from a high view and crowd counts from a low view. Then, the temporal correlation and spatial complementarity between cameras are used to calibrate the overlap areas of the two images. Finally, the total number of people is calculated by combining the low-view crowd counts and the high-view density map. Compared to single-view crowd counting methods, HLIF is experimentally more accurate and has been successfully applied in practice.

Dual-branch counting method for dense crowd based on self-attention mechanism

A dense crowd counting method based on self-attention mechanism with dual-branch fusion network is proposed in this paper. Our method aims to address the problems of large variations in head scales and complex backgrounds in dense crowd images. This method combines the CNN and Transformer network frameworks and consists of shallow feature extraction network, dual-branch fusion network, and deep feature extraction network. The VGG16 network is employed by the shallow feature extraction network to extract low-level features. A multi-scale CNN branch and a Transformer branch built on an improved self-attention module make up the dual-branch fusion network, which collects local and global information on crowd areas, respectively. The Transformer network, which is based on a mixed attention module, is employed by the deep feature extraction network to further separate complicated backgrounds and concentrate on crowd areas. Both counting-level weakly supervised and location-level fully supervised methods are employed in the experiments. On four widely used datasets, the results demonstrate that the proposed method outperforms the most recent research. Our method has a higher counting accuracy with low parameter volumes and a counting accuracy of 89.1% under full supervision when compared to existing weakly supervised methods. The results of the experiments demonstrate that the method has excellent crowd counting performance and can accurately count in high-density and high-occlusion scenes.

LCNNet: Light-weight convolutional neural networks for understanding the highly congested scenes

With the development of convolutional neural networks, many improved algorithms have been successively proposed to promote the accuracy of dense crowd counting. However, these algorithms are deployed with expensive computing resources, which is unbearable for small devices such as embedded systems with limited computing resources. To realize the real-time counting on the small devices, it is of great significance how to trade off the computation cost and processing accuracy of the dense crowd-counting algorithm. Thus, we propose a lightweight dense crowd-counting algorithm (LCNNet) to improve this issue. Specifically, the proposed LCNNet consists of two subnetworks, a feature extraction subnetwork, and a regression subnetwork, with a bottleneck depth-separable convolution with a residuals module as the basic module. The LCNNet effectively improves computational efficiency and reduces the computational cost, which can be performed on small devices. Extensive evaluations on four benchmark datasets well demonstrate the effectiveness of the proposed LCNNet for dense crowd-counting models. Meanwhile, the proposed LCNNet can maintain a comparable level of computational accuracy and computational cost on Vehicle counting datasets.

Research on 24-Hour Dense Crowd Counting and Object Detection System Based on Multimodal Image Optimization Feature Fusion

Motivation. In the environment of day and night video surveillance, in order to improve the accuracy of machine vision dense crowd counting and target detection, this paper designs a day and night dual-purpose crowd counting and crowd detection network based on multimode image fusion. Methods. Two sub-models, RGBD-Net and RGBT-Net, are designed in this paper. The depth image features and thermal imaging features are effectively fused with the features of visible light images, so that the model has stronger anti-interference characteristics and robustness to the light noise interference caused by the sudden fall of light at night. The above models use density map regression-guided detection method to complete population counting and detection. Results. The model completed daytime training and testing on MICC dataset. Through verification, the average absolute error of the model was 1.025, the mean square error was 1.521, and the recall rate of target detection was 97.11%. Night vision training and testing were completed on the RGBT-CC dataset. After verification, the average absolute error of the network was 18.16, the mean square error was 32.14, and the recall rate of target detection was 97.65%. By verifying the effectiveness of the multimode medium-term fusion network, it is found to exceed the current most advanced bimodal fusion method. Conclusion. The experimental results show that the proposed multimodal fusion network can solve the counting and detection problem in the video surveillance environment during day and night. The ablation experiment further proves the effectiveness of the parameters of the two models.

Context-Aware Multi-Scale Aggregation Network for Congested Crowd Counting.

In this paper, we propose a context-aware multi-scale aggregation network named CMSNet for dense crowd counting, which effectively uses contextual information and multi-scale information to conduct crowd density estimation. To achieve this, a context-aware multi-scale aggregation module (CMSM) is designed. Specifically, CMSM consists of a multi-scale aggregation module (MSAM) and a context-aware module (CAM). The MSAM is used to obtain multi-scale crowd features. The CAM is used to enhance the extracted multi-scale crowd feature with more context information to efficiently recognize crowds. We conduct extensive experiments on three challenging datasets, i.e., ShanghaiTech, UCF_CC_50, and UCF-QNRF, and the results showed that our model yielded compelling performance against the other state-of-the-art methods, which demonstrate the effectiveness of our method for congested crowd counting.

LigMSANet: Lightweight multi-scale adaptive convolutional neural network for dense crowd counting

Scale variation and real-time counting are challenging problems for crowd counting in highly congested scenes. To remedy these issues, we proposed a Lightweight Multi-Scale Adaptive Network (LigMSANet). There are two strong points in our method. First, the scale limitation is broken and the proportion of neurons with different receptive field sizes are adjusted spontaneously according to input images through a novel multi-scale adaptation module (MSAM). Second, the model performance is significantly improved at a little cost of parameter by replacing the standard convolution with the depthwise separable convolution and a tailored MobileNetV2 with 5 bottleneck blocks (here, the step size of the fourth bottleneck block is 1). To demonstrate the effectiveness of the proposed method, we conduct extensive experiments on three major crowd counting datasets (ShanghaiTech, UCF_CC_50 and UCSD) and our method achieves superior performance to state-of-the-art methods while with much less parameters and runtimes.

Night vision dense crowd counting based on mid-term fusion of thermal imaging features

为了提高人群计数模型对尺度和光噪声的鲁棒性，设计了一种多模态图像融合网络。提出了一种针对夜间人群统计模型，并设计了一个子网络Rgb-T-net，网络融合了热成像特征和可见光图像的特征，增强了网络对热成像和夜间人群特征的判断能力。模型采用自适应高斯核对密度图进行回归，在Rgb-T-CC数据集上完成了夜视训练和测试。经验证网络平均绝对误差为18.16，均方误差为32.14，目标检测召回率为97.65%，计数性能和检测表现优于当前最先进的双峰融合方法。实验结果表明，所提出的多模态特征融合网络能够解决夜视环境下的计数与检测问题，消融实验进一步证明了融合模型各部分参数的有效性。

ACSL: Adaptive correlation-driven sparsity learning for deep neural network compression

Deep convolutional neural network compression has attracted lots of attention due to the need to deploy accurate models on resource-constrained edge devices. Existing techniques mostly focus on compressing networks for image-level classification, and it is not clear if they generalize well on network architectures for more challenging pixel-level tasks, e.g., dense crowd counting or semantic segmentation. In this paper, we propose an adaptive correlation-driven sparsity learning (ACSL) framework for channel pruning that outperforms state-of-the-art methods on both image-level and pixel-level tasks. In our ACSL framework, we first quantify the data-dependent channel correlation information with a channel affinity matrix. Next, we leverage these inter-dependencies to induce sparsity into the channels with the introduced adaptive penalty strength. After removing the redundant channels, we obtain compact and efficient models, which have significantly less number of parameters while maintaining comparable performance with the original models. We demonstrate the advantages of our proposed approach on three popular vision tasks, i.e., dense crowd counting, semantic segmentation, and image-level classification. The experimental results demonstrate the superiority of our framework. In particular, for crowd counting on the Mall dataset, the proposed ACSL framework is able to reduce up to 94% parameters (VGG16-Decoder) and 84% FLOPs (ResNet101), while maintaining the same performance of (at times outperforming) the original model.

MFP‐Net: Multi‐scale feature pyramid network for crowd counting

Although deep learning has been widely used for dense crowd counting, it still faces two challenges. Firstly, the popular network models are sensitive to scale variance of human head, human occlusions, and complex background due to repeated utilization of vanilla convolution kernels. Secondly, the vanilla feature fusion often depends on summation or concatenation, which ignores the correlation of different features leading to information redundancy and low robustness to background noise. To address these issues, a multi-scale feature pyramid network (MFP-Net) for dense crowd counting is proposed in this paper. The proposed MFP-Net makes two contributions. Firstly, the feature pyramid fusion module is designed that adopts rich convolutions with different depths and scales, not only to expand the receptive field, but also to improve the inference speed of models by using parallel group convolution. Secondly, a feature attention-aware module is added in the feature fusion stage. The module can achieve local and global information fusion by capturing the importance of the spatial and channel domains to improve model robustness. The proposed MFP-Net is evaluated on five publicly available datasets, and experiments show that the MFP-Net not only provides better crowd counting results than comparative models, but also requires fewer parameters.

SRNet: Scale-Aware Representation Learning Network for Dense Crowd Counting

Huge variations in the scales of people in images create an extremely challenging problem in the task of crowd counting. Currently, many researchers apply multi-column structures to solve the scale variation problem. However, multi-column structures usually have complex structures with large numbers of parameters and are difficult to optimize. To this end, we propose a scale-aware representation learning network (SRNet) that uses a commonly used encoder-decoder framework. An image is converted into deep features by the first ten layers of VGG16 in the encoder. Then, the features are regressed to a crowd density map via the decoder. The decoder mainly consists of two modules: the scale-aware feature learning module (SAM) and the pixel-aware upsampling module (PAM). SAM models the multi-scale features of a crowd at each level with different sizes of receptive fields, and PAM enlarges the spatial resolution and enhances the pixel-level semantic information, thereby improving the overall counting accuracy. We conduct extensive crowd counting experiments on ShanghaiTech Part_A, UCF-QNRF, and UCF_CC_50 datasets. Furthermore, to obtain the locations of each person, we conduct crowd localization experiments on UCF-QNRF and NWPU-Crowd datasets. The qualitative and quantitative results prove the effectiveness of the SRNet in dense crowd counting and crowd localization tasks.

Dense crowd counting based on adaptive scene division

With the rapid development of computer vision and artificial intelligence, crowd counting has attracted significant attention from researchers and many well-known methods were proposed. However, due to interocclusions, perspective distortion, and uneven crowd distribution, crowd counting is still a highly challenging task in crowd analysis. Motivated by granular computing, a novel end-to-end crowd counting network (GrCNet) is proposed to enable the problem of crowd counting to be conceptualized at different levels of granularity, and to map problem into computationally tractable subproblems. It shows that by adaptively dividing the image into granules and then feeding the granules into different counting subnetworks separately, the scale variation range of image is narrowed and the the adaptability of counting algorithm to different scenarios is improved. Experiments on four well-known crowd counting benchmark datasets indicate that GrCNet achieves state-of-the-art counting performance and high robustness in dense crowd counting.

KUMBH MELA: a case study for dense crowd counting and modeling

Dense crowd counting and modeling at different gatherings has ignited a new flame in the visual surveillance research community. There is a high possibility of mishappenings in the form of stampede, mob fighting at these gatherings and the administration is helpless in these scenarios. There is a requirement of analyzing the crowd to prevent these dangerous situations. The proposed work is a case study of Kumbh Mela which models the crowd counting in densely populated images. In the proposed work, the orthographic projection of the crowd is captured using a camera attached to a drone, to reduce the effect of occlusion and scaling which, otherwise, may get introduce during image acquisition process. The captured data is fed to a Convolutional Neural Network for training the model to count head of persons present in the frame. The results obtained from the trained model are validated using geometry and imaging techniques. The proposed model has achieved a mean-absolute-error of 94.3 and a mean-squared-error of 104.6 which seems to outperform the existing state-of-the-art models with respect to the reported performance parameters. The proposed model can be used as a viable solution in applications related to modeling the crowd behavior.

Deep convolution network for dense crowd counting

Estimating the total number of people in a crowded situation is a challenging task due to numerous occlusions and perspective changes existing in crowd images. To address this issue, the authors have proposed a new deep learning framework for accurate and efficient crowd counting here. Inspired by multi-column convolutional neural network (MCNN) and contextual pyramid convolutional neural network (CP-CNN), the authors use a combination of a two branches, convolutional neutral network (CNN) and transposed convolutional layers, to generate a high-quality density map. The two-branch CNN for feature extraction generates a density map that is only a quarter of size of the original image Then a set of transposed convolutional layers and convolutional layers are combined with the network to make up for the detail loss of the density map conducted by stacked pooling. Compared with MCNN and CP-CNN, the authors’ approach employs fewer branches and simpler architecture. Experimental result shows that their approach achieves MAE 80.7 and MSE 131.2 in ShanghaiTech PartA dataset, MAE 15.6 and MSE 26.8 in ShanghaiTech PartB dataset, and MAE Average 7.1 in WorldExpo'10 dataset.

Locate, Size, and Count: Accurately Resolving People in Dense Crowds via Detection.

We introduce a detection framework for dense crowd counting and eliminate the need for the prevalent density regression paradigm. Typical counting models predict crowd density for an image as opposed to detecting every person. These regression methods, in general, fail to localize persons accurate enough for most applications other than counting. Hence, we adopt an architecture that locates every person in the crowd, sizes the spotted heads with bounding box and then counts them. Compared to normal object or face detectors, there exist certain unique challenges in designing such a detection system. Some of them are direct consequences of the huge diversity in dense crowds along with the need to predict boxes contiguously. We solve these issues and develop our LSC-CNN model, which can reliably detect heads of people across sparse to dense crowds. LSC-CNN employs a multi-column architecture with top-down feature modulation to better resolve persons and produce refined predictions at multiple resolutions. Interestingly, the proposed training regime requires only point head annotation, but can estimate approximate size information of heads. We show that LSC-CNN not only has superior localization than existing density regressors, but outperforms in counting as well. The code for our approach is available at https://github.com/val-iisc/lsc-cnn.

DSPNet: Deep scale purifier network for dense crowd counting

Crowd counting has produced considerable concern in recent years. However, crowd counting in highly congested scenes is a challenging problem owing to scale variation. To remedy this issue, we propose a novel deep scale purifier network (DSPNet) that can encode multiscale features and reduce the loss of contextual information for dense crowd counting. Our proposed method has two strong points. First, the DSPNet model consists of a frontend and a backend. The frontend is a conventional deep convolutional neural network, while the unified deep neural network backend adopts a “maximal ratio combining” strategy to learn complementary scale information at different levels. The scale purifier module, which improves scale representations, can effectively fuse multiscale features. Second, DSPNet performs the whole RGB image-based inference to facilitate model learning and decrease contextual information loss. Our customized network is end-to-end and has a fully convolutional architecture. We demonstrate the generalization ability of our approach by cross-scene evaluation. Extensive experiments on three publicly available crowd counting benchmarks (i.e., UCF-QNRF, ShanghaiTech, and UCF_CC_50 datasets) show that our DSPNet delivers superior performance against state-of-the-art methods.

Almost Unsupervised Learning for Dense Crowd Counting

We present an unsupervised learning method for dense crowd count estimation. Marred by large variability in appearance of people and extreme overlap in crowds, enumerating people proves to be a difficult task even for humans. This implies creating large-scale annotated crowd data is expensive and directly takes a toll on the performance of existing CNN based counting models on account of small datasets. Motivated by these challenges, we develop Grid Winner-Take-All (GWTA) autoencoder to learn several layers of useful filters from unlabeled crowd images. Our GWTA approach divides a convolution layer spatially into a grid of cells. Within each cell, only the maximally activated neuron is allowed to update the filter. Almost 99.9% of the parameters of the proposed model are trained without any labeled data while the rest 0.1% are tuned with supervision. The model achieves superior results compared to other unsupervised methods and stays reasonably close to the accuracy of supervised baseline. Furthermore, we present comparisons and analyses regarding the quality of learned features across various models.

Effective use of convolutional neural networks and diverse deep supervision for better crowd counting

In this paper, we focus on the task of estimating crowd count and high-quality crowd density maps. Among crowd counting methods, crowd density map estimation is especially promising because it preserves spatial information which makes it useful for both counting and localization (detection and tracking). Convolutional neural networks have enabled significant progress in crowd density estimation recently, but there are still open questions regarding suitable architectures. We revisit CNNs design and point out key adaptations, enabling plain a signal column CNNs to obtain high resolution and high-quality density maps on all major dense crowd counting datasets. The regular deep supervision utilizes the general ground truth to guide intermediate predictions. Instead, we build hierarchical supervisory signals with additional multi-scale labels to consider the diversities in deep neural networks. We begin by obtaining multi-scale labels based on different Gaussian kernels. These multi-scale labels can be seen as diverse representations in the supervision and can achieve high performance for better quality crowd density map estimation. Extensive experiments demonstrate that our approach achieves the state-of-the-art performance on the ShanghaiTech, UCF_CC_50 and UCSD datasets.

Dense crowd counting from still images with convolutional neural networks

For reasons of public security, modeling large crowd distributions for counting or density estimation has attracted significant research interests in recent years. Existing crowd counting algorithms rely on predefined features and regression to estimate the crowd size. However, most of them are constrained by such limitations: (1) they can handle crowds with a few tens individuals, but for crowds of hundreds or thousands, they can only be used to estimate the crowd density rather than the crowd count; (2) they usually rely on temporal sequence in crowd videos which is not applicable to still images. Addressing these problems, in this paper, we investigate the use of a deep-learning approach to estimate the number of individuals presented in a mid-level or high-level crowd visible in a single image. Firstly, a ConvNet structure is used to extract crowd features. Then two supervisory signals, i.e., crowd count and crowd density, are employed to learn crowd features and estimate the specific counting. We test our approach on a dataset containing 107 crowd images with 45,000 annotated humans inside, and each with head counts ranging from 58 to 2201. The efficacy of the proposed approach is demonstrated in extensive experiments by quantifying the counting performance through multiple evaluation criteria.