Crowd Density Estimation Research Articles

Generating high quality crowd density map based on perceptual loss

High quality crowd density maps preserve a large amount of spatial information of crowd distribution, which provides significant priori information for the field of crowd behavior analysis and anomaly detection. Recent work on crowd density estimation pays more attention to the accuracy of crowd counting, ignoring the quality of crowd density map estimation. Hence, in this paper, we propose an end-to-end crowd density estimation network to generate high quality crowd density map. The original pixel-level Euclidean distance loss function in the Multi-column Convolutional Neural Network (MCNN) is replaced by the perceptual loss network. By optimizing the perceptual loss function that is defined as the differences between high-level semantic features generated by a pre-trained network, high-quality map estimation can be obtained. At the same time the accuracy of crowd counting and the sensitivity to the external environment can be improved. Extensive experiments conducted on challenging datasets validate the proposed method outperforms the state-of-the-art methods in both the crowd counting accuracy and the density estimation quality.

Crowd Estimation Using Electromagnetic Wave Power-Level Measurements: A Proof of Concept

Current crowd density estimation technologies that leverage IR depth perception, video and image processing or WiFi/BLE-based sniffing and probing have privacy and deployment issues. This paper presents a novel method for non-intrusive crowd density estimation that monitors variation in EM radiation within an environment. The human body's electrical and magnetic characteristics can be correlated with variations in available EM energy. This allows for the determination of the number of people within a room. Simulations conducted using Comsol to analyse and measure electromagnetic energy levels inside a room containing human bodies. Experimental analysis provides validation of the simulation results by showing 0.8 dBm drop on the average level of EM energy per person.

Multi-scale dilated convolution of convolutional neural network for crowd counting

Growing numbers of crowd density estimation methods have been developed in scene monitoring, crowd safety and on-site management scheduling. We proposed a method for density estimation of a single static image based on convolutional neural network naming Multi-scale Dilated Convolution of Convolutional Neural Network (Multi-scale-CNN). The proposed method employed the method of density maps regression to learn the mapping relationship between single-image and density maps through convolutional neural network. The adopted network structure is composed of two major components to adapt changes of characters scales in crowd images, a convolutional neural network for the general feature extraction and the other is multi-scale dilated convolution for disposing the scale change problem. It is insufficient for currently study that tackled the multi-column or multi-input convolutional neural networks to solve multi-scale problems. Our method utilizes a single-column network to extract features and combines multi-scale dilated convolution to aggregate multi-scale information to address the shortcomings of two networks. The multi-scale dilated convolution module aggregates multi-scale context information systematically by making use of dilated convolution without reducing the receiving domain, thereby integrate the underlying detail information into the high-level semantic features to promote the perception and counting ability of network for small targets. This paper demonstrates the proposed network structure in ShanghaiTech dataset, UCF_CC_50 dataset and worldexpo’10 dataset, and compares the results with numbers of current mainstream crowd counting algorithms, proves that our method surpasses current state-of-the-art methods and has excellent counting accuracy and robustness. The training and testing codes of our method models can be downloaded at https://github.com/doctorwgd/Multi-scale-CNN.

Scene Invariant Virtual Gates Using DNNs

Understanding where people are located and how they are moving about in an environment is critical for operators of large public spaces such as shopping centers, and large public infrastructures such as airports. Automated analysis of CCTV footage is increasingly being used to address this need through techniques that can count crowd sizes, estimate their density, and estimate the through-put of people into and/or out of a choke-point. A limitation of using CCTV based approaches, however, is the need to train models specific to each view which, for large environments with 100s or 1000s of cameras, can quickly become problematic. While there is some success in developing scene-invariant crowd counting and crowd density estimation approaches, much less attention has been given to developing scene-invariant solutions for through-put estimation. In this paper, we investigate the use of convolutional neural network and long short-term memory architectures to estimate pedestrian through-put from arbitrary CCTV viewpoints. To properly develop and demonstrate our approach, we present a new 22 view database featuring 44 h of pedestrian throughput annotation, containing over 11 000 annotated people; and using this proposed approach we show that we are able to outperform a scene-dependant approach across a diverse set of challenging view-points.

The Evaluation of Crowd Density Estimation with Wi-Fi Signal Band in Closed Space via Ray-Tracing Simulation

The Simulation of Crowd Sensing in Closed Space with the Attenuation Measurement of the Wi-Fi Signal is Proposed here. A Train Car is Used as the Test Environment of Interest. Experimental Validation is Performed for a Certain Range of Input Parameters which Include Permittivity of Human Body, Structure Materials and Sensor Placements. the Simulation Results Show that for all Cases, the Signal Strengths Associated with Masses of Human Body Reduce with Number of Passengers. the Number of Passengers can Be Estimated with Good Accuracy Using Linear Function with the R-Square Value over 0.97 and RMSE Value below 2. the Simulation Results Suggest that the Estimation Accuracy of Number of Passengers will Improve with Higher Number of Passengers and Multiple Transmitters in Neighbor Nodes can Affect the Overall Precision. the Experimental Validation Exhibits the same Trend as that of the Simulation. the Additional Experiment Indicates that the Passenger Body Movement Also has a Considerable Impact on the Estimation Accuracy.

Automated Crowd Controlling System Using Image Processing and Video Processing Technique to Avoid Stamped

Crowd control is needed to prevent the outbreak of disorder and prevent possible stampedes. An automated detection of people crowds from images has become a very important research field. Due to the importance of the topic, many researchers tried to solve this problem using CCTV street cameras. There are still significant problems in managing public pedestrian transport areas such as railway stations, stadiums, shopping malls, and religious gatherings. Using CCTV cameras, some image processing techniques are suitable for an automatic crowd monitoring system. The feasibility of such a system has been tested by analyzing the crowd behavior, crowd density and motion. Traditional measurement techniques, based on manual observations, are not suitable for comprehensive data collection of patterns of density and movement. Real-time monitoring is tedious and tiring, but critical for safety. The author has investigated a number of techniques for crowd density estimation, movement estimation, incident detection and their merits using image processing.

Atrous convolutions spatial pyramid network for crowd counting and density estimation

Scale variation because of perspective distortion is still a challenge for crowd analysis. To address this problem, an atrous convolutions spatial pyramid network (ACSPNet) is proposed to perform crowd counts and density maps for both sparse and congested scenarios. Atrous Convolutions sequenced with increasing atrous rates are utilized to exaggerate the receptive field and maintain the resolution of extracted features. Different rates of atrous convolution blocks in the pyramid are skip-connected to integrate multi-scale information and extent scale perception ability. Atrous Spatial Pyramid Pooling (ASPP) is employed to resample information at different scales and contain global context. We evaluate our ACSPNet on five challenging benchmark crowd counting datasets and our method achieves state-of-the-art mean absolute error (MAE) and mean squared error (MSE) performances.

Detecting high indoor crowd density with Wi-Fi localization: a statistical mechanics approach

We address the problem of detecting highly raised crowd density in situations such as indoor dance events. We propose a new method for estimating crowd density by anonymous, non-participatory, indoor Wi-Fi localization of smart phones. Using a probabilistic model inspired by statistical mechanics, and relying only on big data analytics, we tackle three challenges: (1) the ambiguity of Wi-Fi based indoor positioning, which appears regardless of whether the latter is performed with machine learning or with optimization, (2) the MAC address randomization when a device is not connected, and (3) the volatility of packet interarrival times. The main result is that our estimation becomes more—rather than less—accurate when the crowd size increases. This property is crucial for detecting dangerous crowd density.

Cross-Line Pedestrian Counting Based on Spatially-Consistent Two-Stage Local Crowd Density Estimation and Accumulation

This paper proposes a scalable approach for counting pedestrians crossing a virtual line when the crowd is highly dynamic and possibly extremely dense. The approach mainly consists of two parts: local crowd density estimation and pedestrian counting based on accumulating local densities across the line. To obtain a fine estimation of local crowd densities, we divide the neighborhood at the line into a number of blocks. We enforce spatial consistency between local counts in the blocks and those in the enclosing regions to guarantee consistent estimation of local crowd densities. For scalability to various density levels in crowd density estimation, we propose a two-stage strategy: pre-classification of density levels and subsequent regression with overlapped operational ranges. To count pedestrians crossing the virtual line, we accumulate the crowd densities across the line according to the locally estimated velocities. Extensive experimental results demonstrate the effectiveness of the proposed approach and its scalability to crowdedness.

Abnormal crowd density estimation in aerial images

The unpreceded growth of intelligent surveillance systems has resulted in an urgent need for automatic analysis of the captured scenes. Automatic detection of an abnormal crowd in aerial images can provide useful information to prevent disasters. In fact, aerial images have the advantage of covering a very large view of the people distributed over a scene. Accordingly, we propose a high density crowd detection method in aerial images. In this method, we adapted the bag of words technique using the multiblock local binary pattern as a texture descriptor to extract low-level features. In addition, we also used a three-level classification strategy to reduce confusion between crowd density classes. The experimental results reveal the performance of our method while estimating the crowd density in a challenging context.

Direct generation of level of service maps from images using convolutional and long short-term memory networks

Congestion in transport stations could result in stampede development and deadly crush situations. Closed circuit television (CCTV) cameras enable station managers to monitor the crowd and reduce overcrowding risks. However, identifying congestion conditions is a very laborious task for a human operator who has to monitor multiple locations at the same time. This paper presents a new approach to automated image-based identification of congestion as measured by level of service (LOS), which is the most widely accepted standard for measuring congestion. Existing methods for measuring LOS based on crowd density estimation from images have the disadvantages that, crowd density cannot be estimated accurately. In addition, the calculation of flow parameters involves a complex process, and consequently these parameters are not indicative of congestion in real-time. This paper proposes a novel method based on machine learning to directly classify LOS without calculating flow parameters. In the proposed method, visual features extracted by a deep convolutional neural network are classified using a support vector machine classifier and the classification results are further refined by using a long short-term memory network. A second contribution of this research is to develop a web-based LOS map visualization platform to monitor pedestrian distribution and variation of distribution in real-time. Experimental evaluation at Flinders Street Station in Melbourne shows that this method can achieve an accuracy of 81.9% and efficiency of 0.40 seconds per frame in LOS classification using CCTV images.

Effective use of convolutional neural networks and diverse deep supervision for better crowd counting

In this paper, we focus on the task of estimating crowd count and high-quality crowd density maps. Among crowd counting methods, crowd density map estimation is especially promising because it preserves spatial information which makes it useful for both counting and localization (detection and tracking). Convolutional neural networks have enabled significant progress in crowd density estimation recently, but there are still open questions regarding suitable architectures. We revisit CNNs design and point out key adaptations, enabling plain a signal column CNNs to obtain high resolution and high-quality density maps on all major dense crowd counting datasets. The regular deep supervision utilizes the general ground truth to guide intermediate predictions. Instead, we build hierarchical supervisory signals with additional multi-scale labels to consider the diversities in deep neural networks. We begin by obtaining multi-scale labels based on different Gaussian kernels. These multi-scale labels can be seen as diverse representations in the supervision and can achieve high performance for better quality crowd density map estimation. Extensive experiments demonstrate that our approach achieves the state-of-the-art performance on the ShanghaiTech, UCF_CC_50 and UCSD datasets.

Crowd density estimation in still images using multiple local features and boosting regression ensemble

Crowd density estimation is a challenging research problem in computer vision and has many applications in commercial and defense sectors. Various crowd density estimation methods have been proposed by researchers in the past, but there is an utmost need for accurate, robust and efficient crowd density estimation techniques for its practical implementation. In this paper, we propose a fine-tuned and computationally economical, ensemble regression-based machine learning model for crowd density estimation. The WorldExpo’10 dataset has been used for experimental analysis and model performance evaluation. We extract variety of features in texture-based features such as gray-level co-occurrence matrix, local binary pattern and histogram of oriented gradients, structure-based features such as perimeter pixel and the orientation of pixels, and segment-based handcrafted features from each patch of the image and use an optimum combination of these features as input to the regression model. To achieve optimized memory utilization and faster speed, principal component analysis is employed to reduce the dimensions of the lengthy feature vector. Extensive experiments on different fronts ranging from the model hyperparameter optimization, features optimization and features selection were conducted, and at each step, we selected the most favorable results as input to the optimized model. The performance of the model is evaluated based on two popular metrics, i.e., mean absolute error and mean squared error. The comparative analysis shows that the proposed system outperforms the former methods tested on the WorldExpo’10 dataset.

Crowd density estimation based on classification activation map and patch density level

The task of crowd counting and density map estimation is riddled with many challenges, such as occlusions, non-uniform density, intra-scene and inter-scene variations in scale and perspective. Due to the development of deep learning and large crowd datasets in recent years, most crowd counting methods have achieved notable success. This paper aims to solve crowd density estimation problem for both sparse and dense conditions. To this end, we make two contributions: (1) a network named Patch Scale Discriminant Regression Network (PSDR). Given an input crowd image, it divides the image into patches and sends image patches of different density levels into different regression networks to get the corresponding density maps. It combines all patch density maps to predict the entire density map as the output. (2) A person classification activation map (CAM) method. CAM provides person location information and guides the generation of the entire density map in the final stage. Experiment confirms that CAM allows PSDR to gain another round of performance boost. For instance, on the SmartCity dataset, we achieve (8.6–1.1) MAE and (11.6–1.4) MSE. Our method combining above two methods performs better than state-of-the-art methods.

Crowd detection and counting using a static and dynamic platform: state of the art

Automated object detection and crowd density estimation are popular and important area in visual surveillance research. The last decades witnessed many significant research in this field however, it is still a challenging problem for automatic visual surveillance. The ever increase in research of the field of crowd dynamics and crowd motion necessitates a detailed and updated survey of different techniques and trends in this field. This paper presents a survey on crowd detection and crowd density estimation from moving platform and surveys the different methods employed for this purpose. This review category and delineates several detections and counting estimation methods that have been applied for the examination of scenes from static and moving platforms.

Crowd detection and counting using a static and dynamic platform: state of the art

Automated object detection and crowd density estimation are popular and important area in visual surveillance research. The last decades witnessed many significant research in this field however, it is still a challenging problem for automatic visual surveillance. The ever increase in research of the field of crowd dynamics and crowd motion necessitates a detailed and updated survey of different techniques and trends in this field. This paper presents a survey on crowd detection and crowd density estimation from moving platform and surveys the different methods employed for this purpose. This review category and delineates several detections and counting estimation methods that have been applied for the examination of scenes from static and moving platforms.

LTE Signals for Device-Free Crowd Density Estimation Through CSI Secant Set and SVD

The rising need of crowd monitoring in public spaces, especially for safety purposes, pushes the research community to propose and experiment novel methods of crowd density estimation. This paper focuses on device-free RF sensing, which does not require the monitored people to carry any electronic device. In particular, the paper proposes and assesses the performance of a crowd density estimation system based on the analysis of the variations of Channel State Information (CSI) computed from unencrypted synchronization signals transmitted by a eNodeB and reflected/scattered by people located in the monitored area. The proposed method uses features extracted from the list of singular values of the CSI secant set. This approach allows to reduce the impact of CSI variations due to noise or HW instability, improving the sensitivity to CSI variations caused by human presence. The average accuracy achieved by the proposed approach is 84%, which is comparable with the accuracy achieved with WiFi based crowd density estimation systems.

Crowd Counting in Low-Resolution Crowded Scenes Using Region-Based Deep Convolutional Neural Networks

Crowd counting and density estimation is an important and challenging problem in the visual analysis of the crowd. Most of the existing approaches use regression on density maps for the crowd count from a single image. However, these methods cannot localize individual pedestrian and therefore cannot estimate the actual distribution of pedestrians in the environment. On the other hand, detection-based methods detect and localize pedestrians in the scene, but the performance of these methods degrades when applied in high-density situations. To overcome the limitations of pedestrian detectors, we proposed a motion-guided filter (MGF) that exploits spatial and temporal information between consecutive frames of the video to recover missed detections. Our framework is based on the deep convolution neural network (DCNN) for crowd counting in the low-to-medium density videos. We employ various state-of-the-art network architectures, namely, Visual Geometry Group (VGG16), Zeiler and Fergus (ZF), and VGGM in the framework of a region-based DCNN for detecting pedestrians. After pedestrian detection, the proposed motion guided filter is employed. We evaluate the performance of our approach on three publicly available datasets. The experimental results demonstrate the effectiveness of our approach, which significantly improves the performance of the state-of-the-art detectors.

Crowd counting via scale-adaptive convolutional neural network in extremely dense crowd images

Crowd counting, a high accuracy and high-speed technology, has been applied in new retail, shopping mall, underground, rail station and vehicle surveillance systems. However, due to the inconsistent sizes of human heads, there are a lot of counting errors and instability of the crowd density estimation in extremely dense crowd images. Therefore, a scale-adaptive Convolutional Neural Network (CNN) architecture is proposed by introducing residual network on the basis of multi-column CNN. In the process of model training, joint learning is proposed in this paper. Through alternating training for residual network and multi-column CNN, network parameters with the best accuracy are selected after iteration. Joint learning helps to enhance the modelling ability for massive scale transformation and the scale self-adaptability of the network. The proposed method is experimented on public dense crowd data sets. Experimental results prove that scale-adaptive CNN shows higher counting capability than the current state-of-the-art method.

An FPGA-Based HOG Accelerator with HW/SW Co-Design for Human Detection and Its Application to Crowd Density Estimation

Human detection is important in many applications and has attracted significant attention over the last decade. The Histograms of Oriented Gradients (HOG) as effective local descriptors are used with binary sliding window mechanism to achieve good detection performance. However, the computation of HOG under such framework is about billion times and the pure software implementation for HOG computation is hard to meet the real-time requirement. This study proposes a hardware architecture called One-HOG accelerator operated on FPGA of Xilinx Spartan-6 LX-150T that provides an efficient way to compute HOG such that an embedded real-time platform of HW/SW co-design for application to crowd estimation and analysis is achieved. The One-HOG accelerator mainly consists of gradient module and histogram module. The gradient module is for computing gradient magnitude and orientation; histogram module is for generating a 36-D HOG feature vector. In addition to hardware realization, a new method called Histograms-of-Oriented-Gradients AdaBoost Long-Feature-Vector (HOG-AdaBoost-LFV) human classifier is proposed to significantly decrease the number of times to compute the HOG without sacrificing detection performance. The experiment results from three static image and four video datasets demonstrate that the proposed SW/HW (software/hardware) co-design system is 13.14 times faster than the pure software computation of Dalal algorithm.