Crowd Scene Analysis Research Articles

Crowd Scene Analysis Encounters High Density and Scale Variation.

Crowd scene analysis receives growing attention due to its wide applications. Grasping the accurate crowd location is important for identifying high-risk regions. In this article, we propose a Compressed Sensing based Output Encoding (CSOE) scheme, which casts detecting pixel coordinates of small objects into a task of signal regression in encoding signal space. To prevent gradient vanishing, we derive our own sparse reconstruction backpropagation rule that is adaptive to distinct implementations of sparse reconstruction and makes the whole model end-to-end trainable. With the support of CSOE and the backpropagation rule, the proposed method shows more robustness to deep model training error, which is especially harmful to crowd counting and localization. The proposed method achieves state-of-the-art performance across four mainstream datasets, especially achieves excellent results in highly crowded scenes. A series of analysis and experiments support our claim that regression in CSOE space is better than traditionally detecting coordinates of small objects in pixel space for highly crowded scenes.

A Deep Learning Approach for Real-Time Analysis of Attendees’ Engagement in Public Events

Smart city analytics requires the harnessing and analysis of emotions and sentiments conveyed by images and video footage. In recent years, facial sentiment analysis attracted significant attention for different application areas, including marketing, gaming, political analytics, healthcare, and human computer interaction. Aiming at contributing to this area, we propose a deep learning model enabling the accurate emotion analysis of crowded scenes containing complete and partially occluded faces, with different angles, various distances from the camera, and varying resolutions. Our model consists of a sophisticated convolutional neural network (CNN) that is combined with pooling, densifying, flattening, and Softmax layers to achieve accurate sentiment and emotion analysis of facial images. The proposed model was successfully tested using 3,750 images containing 22,563 faces, collected from a large consumer electronics trade show. The model was able to correctly classify the test images which contained faces with different angles, distances, occlusion areas, facial orientation and resolutions. It achieved an average accuracy of 90.6% when distinguishing between seven emotions (Happiness, smiling, laughter, neutral, sadness, anger, and surprise) in complete faces, and 86.16% accuracy in partially occluded faces. Such model can be leveraged for the automatic analysis of attendees’ engagement level in events. Furthermore, it can open the door for many useful applications in smart cities, such as measuring employees’ satisfaction and citizens’ happiness.

Deep Learning-Based Crowd Scene Analysis Survey.

Recently, our world witnessed major events that attracted a lot of attention towards the importance of automatic crowd scene analysis. For example, the COVID-19 breakout and public events require an automatic system to manage, count, secure, and track a crowd that shares the same area. However, analyzing crowd scenes is very challenging due to heavy occlusion, complex behaviors, and posture changes. This paper surveys deep learning-based methods for analyzing crowded scenes. The reviewed methods are categorized as (1) crowd counting and (2) crowd actions recognition. Moreover, crowd scene datasets are surveyed. In additional to the above surveys, this paper proposes an evaluation metric for crowd scene analysis methods. This metric estimates the difference between calculated crowed count and actual count in crowd scene videos.

Effective crowd counting using multi-resolution context and image quality assessment-guided training

Crowd counting is the challenging task in the crowd scene analysis. To tackle the scale variant issue and to calculate the more accurate result in this target task, this paper designs an effective crowd counting method based on multi-resolution context and image quality assessment-guided training. Specially, a multi-resolution context module is designed to extract the multi-scale context adaptively to enhance the final counting performance through learning the imbalance between different scale paths. An image quality assessment-guided training approach is developed to facilitate the crowd counting network to generate high-quality density map and more accurate counting result. Extensive experiments on benchmarks demonstrate the effectiveness of the proposed method on crowd counting task, the generalization of the proposed method, and the generalization of the developed image quality assessment-guided training approach.

Real-time frequency-based detection of a panic behavior in human crowds

The real-time detection of a panic behavior in a human crowd is of a high interest as it helps alleviating crowd disasters and ensures that timely appropriate action will be taken. However, the fast analysis of video sequences to detect abnormal behaviours is one of the most challenging tasks for computer vision experts. While many research works propose off-line solutions, few studies investigate the real-time analysis of crowded scenes. This may be due to the fact that detecting a panic behaviour is closely related to the analysis of the crowd dynamics, which commonly necessitates heavy computations. In order to alleviate this problem, we propose a real-time panic detection technique that analyzes the crowd movements based on a simple and efficient solution. The key idea of the proposed approach consists of analyzing the interactions between moving edges along the video in the frequency domain. Our contribution is threefold. First, moving edges are considered for analysis along the video. Second, when a panic situation occurs within a human crowd, it leads to interactions between people that are different from those that occur during a normal situation. Therefore, to reveal such a behavior, a new frequency based-feature is proposed. To select the most appropriate frequency domain, the fast fourier transform, the discrete cosine transform and the discrete wavelet transform are investigated. Third, two different formulations of the problem of detecting a panic are explored. The experimental evaluation of the proposed technique shows its outperforming compared to the state-of-the-art approaches in terms of detection rates and execution time.

Multi-Stream Networks and Ground Truth Generation for Crowd Counting

Crowd scene analysis has received a lot of attention recently due to a wide variety of applications, e.g., forensic science, urban planning, surveillance and security. In this context, a challenging task is known as crowd counting [1–6], whose main purpose is to estimate the number of people present in a single image. A multi-stream convolutional neural network is developed and evaluated in this paper, which receives an image as input and produces a density map that represents the spatial distribution of people in an end-to-end fashion. In order to address complex crowd counting issues, such as extremely unconstrained scale and perspective changes, the network architecture utilizes receptive fields with different size filters for each stream. In addition, we investigate the influence of the two most common fashions on the generation of ground truths and propose a hybrid method based on tiny face detection and scale interpolation. Experiments conducted on two challenging datasets, UCF-CC-50 and ShanghaiTech, demonstrate that the use of our ground truth generation methods achieves superior results.

Scale-Recursive Network with point supervision for crowd scene analysis

Crowd scene analysis, and in particular its density estimation, is a challenging task due to the lack of spatial information, scale variation, and the large amount of supervised-learning parameters. In order to address these challenges, we propose a Scale-Recursive encoder–decoder Network with Point Supervision (SRN+PS). On the one hand, an encoder–decoder recurrent structure uses features between adjacent scales to tackle scale variation, and a novel loss function, called the row vector-based counting loss, is proposed to focus on the crowd counting accuracy. On the other hand, we employ an additional point segmentation task in training and combine features learned from the two tasks above. The Euclidean loss, row vector-based counting loss, and two-label focal loss are integrated by a joint training scheme, which improves both the quality of density map estimation and the performance of crowd counting. Finally, we propose a weakly supervised framework based on the SRN structure and the Convolutional Winner-Take-All(CWTA) module. In this framework, most parameters are obtained by unsupervised learning with the exception of a few which are tuned by supervised learning in model training. As a result, our multi-scale structure can obtain salient object sparse spatial features from unsupervised learning. Experiments on the ShanghaiTech, UCF_CC_50 and UCSD datasets demonstrate the effectiveness of our proposed method.

Identifying motion pathways in highly crowded scenes: A non-parametric tracklet clustering approach

Many approaches that address the analysis of crowded scenes rely on using short trajectory fragments, also known as tracklets, of moving objects to identify motion pathways. Typically, such approaches aim at defining meaningful relationships among tracklets. However, defining these relationships and incorporating them in a crowded scene analysis framework is a challenge. In this article, we introduce a robust approach to identifying motion pathways based on tracklet clustering. We formulate a novel measure, inspired by line geometry, to capture the pairwise similarities between tracklets. For tracklet clustering, the recent distance dependent Chinese restaurant process (DD-CRP) model is adapted to use the estimated pairwise tracklet similarities. The motion pathways are identified based on two hierarchical levels of DD-CRP clustering such that the output clusters correspond to the pathways of moving objects in the crowded scene. Moreover, we extend our DD-CRP clustering adaptation to incorporate the source and sink gate probabilities for each tracklet as a high-level semantic prior for improving clustering performance. For qualitative evaluation, we propose a robust pathway matching metric, based on the chi-square distance, that accounts for both spatial coverage and motion orientation in the matched pathways. Our experimental evaluation on multiple crowded scene datasets, principally, the challenging Grand Central Station dataset, demonstrates the state-of-the-art performance of our approach. Finally, we demonstrate the task of motion abnormality detection, both at the tracklet and frame levels, against the normal motion patterns encountered in the motion pathways identified by our method, with competent quantitative performance on multiple datasets.

Gestalt laws based tracklets analysis for human crowd understanding

Crowded scene analysis is a popular research topic due to its great application potentials, such as intelligent video surveillance and crowd density estimation. In this paper, we propose a novel approach to detecting crowd groups and learning semantic regions with a unified hierarchical clustering framework. According to the Gestalt laws of grouping, we propose three priors to define a unified similarity metric to measure the similarities of pairs of original tracklets and pairs of representative tracklets from different crowd groups, so that the short-term crowd groups and the long-term semantic paths commonly composed of several short-term crowd groups can be detected by a bottom-up hierarchical clustering algorithm simultaneously. In order to verify our method at the longer time duration video sequences in the crowded scene, we construct a new crowd database (CASIA crowd database 1) with various crowd densities in real scenes. Extensive experiments on our CASIA crowd database, Collective Motion Database and CUHK database are performed, and the results demonstrate that our approach is effective and reliable for crowd detection and semantic scene understanding in various crowd densities, especially for the crowd analysis in long temporal video clips.

A Survey of Video-Based Crowd Anomaly Detection in Dense Scenes

Population growth has made the probability of incidents at large-scale crowd events higher than ever. In the past decades, automated crowd scene analysis done by computer vision has attracted attention. However, severe occlusions and complex crowd behaviors make such analysis a challenge. As a key aspect of crowd scene analysis, a number of works dealing with dense crowd anomaly detection based on computer vision have been presented. This work is a survey of computer vision techniques for analyzing dense crowd scenes. It covers two aspects: crowd density estimation and abnormal event detection. Some problems and perspectives are discussed at the end.

A Literature Review on Crowd Scene Analysis and Monitoring

A Literature Review on Crowd Scene Analysis and Monitoring

Crowded Scene Analysis: A Survey

Automated scene analysis has been a topic of great interest in computer vision and cognitive science. Recently, with the growth of crowd phenomena in the real world, crowded scene analysis has attracted much attention. However, the visual occlusions and ambiguities in crowded scenes, as well as the complex behaviors and scene semantics, make the analysis a challenging task. In the past few years, an increasing number of works on crowded scene analysis have been reported, covering different aspects including crowd motion pattern learning, crowd behavior and activity analysis, and anomaly detection in crowds. This paper surveys the state-of-the-art techniques on this topic. We first provide the background knowledge and the available features related to crowded scenes. Then, existing models, popular algorithms, evaluation protocols, as well as system performance are provided corresponding to different aspects of crowded scene analysis. We also outline the available datasets for performance evaluation. Finally, some research problems and promising future directions are presented with discussions.

Analyzing motion patterns in crowded scenes via automatic tracklets clustering

Crowded scene analysis is currently a hot and challenging topic in computer vision field. The ability to analyze motion patterns from videos is a difficult, but critical part of this problem. In this paper, we propose a novel approach for the analysis of motion patterns by clustering the tracklets using an unsupervised hierarchical clustering algorithm, where the similarity between tracklets is measured by the Longest Common Subsequences. The tracklets are obtained by tracking dense points under three effective rules, therefore enabling it to capture the motion patterns in crowded scenes. The analysis of motion patterns is implemented in a completely unsupervised way, and the tracklets are clustered automatically through hierarchical clustering algorithm based on a graphic model. To validate the performance of our approach, we conducted experimental evaluations on two datasets. The results reveal the precise distributions of motion patterns in current crowded videos and demonstrate the effectiveness of our approach.

Learning Video Manifolds for Content Analysis of Crowded Scenes

In this paper, we propose a new approach for recognizing group events and abnormality detection in a crowded scene. A manifold learning algorithm with temporal-constraints is proposed to embed a video of a crowded scene in a low-dimensional space. Our low dimensional representation of a video preserves the spatial temporal property of a video as well as the characteristic of the video. Recognizing video events and abnormality detection in a crowded scene is achieved by studying the video trajectory in the manifold space. We evaluate our proposed method on the state-of-the-art public data-sets containing different crowd events. Qualitative and quantitative results show the promising performance of the proposed method.

Learning Video Manifolds for Content Analysis of Crowded Scenes

Learning Video Manifolds for Content Analysis of Crowded Scenes