Abnormal Crowd Behavior Research Articles

Nonlinear analysis of pedestrian flow Reynolds number in video scenes

Abstract Research on crowd motion state plays an essential role for public safety and security. This paper aims to investigate the chaotic characteristics of pedestrian flow as a dynamical system. Firstly, pedestrian flow Reynolds number is proposed, which is a novel feature descriptor derived from Hydrodynamics, to describe crowd motion state. Secondly, the calculation method of pedestrian flow Reynolds number in video is put forward to characterize the motion state of the pedestrian flow. Thirdly, nonlinear time series analysis tools, including time delay embedding and largest Lyapunov exponent are applied to verify the chaos of pedestrian flow’s motion. Experiments are performed on different data sets and the result that all the largest Lyapunov exponent are positive could indeed demonstrate the complexity and chaos of crowd motion. Meanwhile, it turns out that pedestrian flow Reynolds number put forward in the paper can effectively characterize the motion state of pedestrian flow. Our work paves a new way for research on crowd turbulence. It could potentially be applied to pattern analysis of crowd abnormal behavior analysis, crowd motion understanding, which can be used to improve the efficiency of public security management.

An Analysis Method of Crowd Abnormal Behavior for Video Service Robot

Abnormal behavior analysis in crowded scenes is a challenging task in the computer vision field. Determining how to quickly and accurately locate abnormal behavior in a crowd, especially in airports, shopping malls, stations, and other practical application scenarios, has become the key to the timely handling of mass incidents. The authors have developed an embedded video analysis robot capable of local processing for crowd anomaly detection in application scenarios. In recent years, many methods of crowd anomaly detection have been proposed by scholars, but the performance of these methods when applied to video service robots is unsatisfactory. In this paper, an anomaly analysis method based on improved k-means is proposed. This method combines mean shift and the k-means classification method to achieve rapid and accurate crowd anomaly detection. A quantitative experimental evaluation was carried out on a video service robot, and the results demonstrate that the method is effective for the detection of abnormal behavior on multiple and publicly available video sequences.

Detecting abnormal crowd behaviors based on the div-curl characteristics of flow fields

This study proposes a divergence-curl-driven framework for the perception of crowd motion states. In this framework, the characteristics of a flow field, divergence and curl, are used to analyze crowd states. As a collective motion, the movement of a pedestrian crowd shows coherent structural properties. By using the methods of fluid mechanics and the feature visualization of flow fields, a physical characteristic descriptor of crowd motion is established that can model the motion state in a crowd flow field. Given the significance of the temporal comparison of motion states for detecting changes in crowds, a method based on the temporal context of motion is presented to measure changes in the distribution of the physical characteristic descriptors of crowd motion. This method can be used to calculate differences in the distribution of the flow field’s physical characteristics between each state and measure these subtle continuous changes on the sample points, thereby obtaining a quantified metric of changes in a crowd’s motion state. Experiments on crowd event datasets demonstrate the effectiveness of our proposed framework for detecting crowd state changes and abnormal activity.

Effective Crowd Anomaly Detection Through Spatio-temporal Texture Analysis

Abnormal crowd behaviors in high density situations can pose great danger to public safety. Despite the extensive installation of closed-circuit television (CCTV) cameras, it is still difficult to achieve real-time alerts and automated responses from current systems. Two major breakthroughs have been reported in this research. Firstly, a spatial-temporal texture extraction algorithm is developed. This algorithm is able to effectively extract video textures with abundant crowd motion details. It is through adopting Gabor-filtered textures with the highest information entropy values. Secondly, a novel scheme for defining crowd motion patterns (signatures) is devised to identify abnormal behaviors in the crowd by employing an enhanced gray level co-occurrence matrix model. In the experiments, various classic classifiers are utilized to benchmark the performance of the proposed method. The results obtained exhibit detection and accuracy rates which are, overall, superior to other techniques.

Dual-channel CNN for efficient abnormal behavior identification through crowd feature engineering

This research has been investigating an automatic and online crowd anomaly detection model by exploring a novel compound image descriptor generated from live video streams. A dual-channel convolutional neural network (DCCNN) has been set up for efficiently processing scene-related and motion-related crowd information inherited from raw frames and the compound descriptor instances. The novelty of the work stemmed from the creation of the spatio-temporal cuboids in online (or near real-time) manner through dynamically extracting local feature tracklets within the temporal space while handling the foreground region-of-interests (i.e., moving targets) through the exploration of Gaussian Mixture Model in the spatial space. Hence, the extracted foreground blocks can effectively eliminate irrelevant backgrounds and noises from the live streams for reducing the computational costs in the subsequent detecting phases. The devised compound feature descriptor, named as spatio-temporal feature descriptor (STFD), is capable of characterizing the crowd attributes through the measures such as collectiveness, stability, conflict and density in each online generated spatio-temporal cuboid. A STFD instance registers not only the dynamic variation of the targeted crowd over time based on local feature tracklets, but also the interaction information of neighborhoods within a crowd, e.g., the interaction force through the K-nearest neighbor (K-NN) analysis. The DCCNN developed in this research enables online identification of suspicious crowd behaviors based on analyzing the live-feed images and their STFD instances. The proposed model has been developed and evaluated against benchmarking techniques and databases. Experimental results have shown substantial improvements in terms of detection accuracy and efficiency for online crowd abnormal behavior identification.

Towards Intelligent Crowd Behavior Understanding Through the STFD Descriptor Exploration

Realizing the automated and online detection of crowd anomalies from surveillance CCTVs is a research-intensive and application-demanding task. This research proposes a novel technique for detecting crowd abnormalities through analyzing the spatial and temporal features of input video signals. This integrated solution defines an image descriptor (named spatio-temporal feature descriptor—STFD) that reflects the global motion pattern of crowds over time. A designed convolutional neural network (CNN) has then been adopted to classify dominant or large-scale crowd abnormal behaviors. The work reported has focused on: (1) detecting moving objects in online (or near real-time) manner through spatio-temporal segmentations of crowds identified by the similarity of group trajectory structures in the temporal space and the foreground blocks based on the Gaussian mixture model in the spatial space; (2) dividing multiple clustered groups based on the spectral clustering methods through treating image pixels from segmented regions as dynamic particles; (3) creating STFD descriptor instances by calculating corresponding attributes such as collectiveness, stability, conflict and crowd density for individuals (particles) in the corresponding groups; (4) inputting generated STFD descriptor instances into the devised CNN to detect suspicious crowd behaviors. For the test and evaluation of the devised models and techniques, the PETS database has been selected as the primary experimental data sets. Results against benchmarking models and systems have shown promising advancements of this novel approach in terms of accuracy and efficiency for crowd anomaly detection.

Energy Level-Based Abnormal Crowd Behavior Detection.

The change of crowd energy is a fundamental measurement for describing a crowd behavior. In this paper, we present a crowd abnormal detection method based on the change of energy-level distribution. The method can not only reduce the camera perspective effect, but also detect crowd abnormal behavior in time. Pixels in the image are treated as particles, and the optical flow method is adopted to extract the velocities of particles. The qualities of different particles are distributed as different value according to the distance between the particle and the camera to reduce the camera perspective effect. Then a crowd motion segmentation method based on flow field texture representation is utilized to extract the motion foreground, and a linear interpolation calculation is applied to pedestrian’s foreground area to determine their distance to the camera. This contributes to the calculation of the particle qualities in different locations. Finally, the crowd behavior is analyzed according to the change of the consistency, entropy and contrast of the three descriptors for co-occurrence matrix. By calculating a threshold, the timestamp when the crowd abnormal happens is determined. In this paper, multiple sets of videos from three different scenes in UMN dataset are employed in the experiment. The results show that the proposed method is effective in characterizing anomalies in videos.

Detection of Mass Panic using Internet of Things and Machine Learning

The increase of emergency situations that cause mass panic in mass gatherings, such as terrorist attacks, random shooting, stampede, and fires, sheds light on the fact that advancements in technology should contribute in timely detecting and reporting serious crowd abnormal behaviour. The new paradigm of the ‘Internet of Things’ (IoT) can contribute to that. In this study, a method for real-time detection of abnormal crowd behaviour in mass gatherings is proposed. This system is based on advanced wireless connections, wearable sensors and machine learning technologies. It is a new crowdsourcing approach that considers humans themselves as the surveillance devices that exist everywhere. A sufficient number of the event’s attendees are supposed to wear an electronic wristband which contains a heart rate sensor, motion sensors and an assisted-GPS, and has a wireless connection. It detects the abnormal behaviour by detecting heart rate increase and abnormal motion. Due to the unavailability of public bio-dataset on mass panic, dataset of this study was collected from 89 subjects wearing the above-mentioned wristband and generating 1054 data samples. Two types of data collected were: firstly, the data of normal daily activities and secondly, the data of abnormal activities resembling the behaviour of escape panic. Moreover, another abnormal dataset was synthetically generated to simulate panic with limited motion. In our proposed approach, two-phases of data analysis are done. Phase-I is a deep machine learning model that was used to analyze the sensors’ collected readings of the wristband and detect if the person has indeed panicked in order to send alerting signals. While phase-II data analysis takes place in the monitoring server that receives the alerting signals to conclude if it is a mass panic incident or a false positive case. Our experiments demonstrate that the proposed system can offer a reliable, accurate, and fast solution for panic detection. This experiment uses the Hajj pilgrimage as a case study.

The detecting of abnormal crowd activities based on motion vector

Aiming at the crowd in high-definition video motion state of sudden changes in rapid detection of abnormal crowd behavior problem, this paper proposes a kind of abnormal behavior crowd detection method based on motion vector. This algorithm is established upon the Social Force Model, first, extracts the motion vector in the code stream of the high-definition compressed videos, computes the interaction force in the social force model and rapidly draws the characteristics of the moving crowd; then according to the algorithm, we perform the bag of words approach and histogram statistics on the intensity and angle of the interaction force flow; finally we analyze two histograms to distinguish the moving state of the crowd and fulfill the detection of the abnormal crowd movement. The simulation experiment shows the method compared with the traditional social force model, in the 1024 × 768HD video frame processing speed on the average increase of 30% in average, the discrimination of abnormal frame advance 35 frames, the recall to an average increase of 22%.

Crowd Anomaly Detection Using Motion Based Spatio-Temporal Feature Analysis

<p>Recently, the demand for surveillance system is increasing in real time application to enhance the security system. These surveillance systems are mainly used in crowded places such as shopping malls, sports stadium etc. In order to support enhance the security system, crowd behavior analysis has been proven a significant technique which is used for crowd monitoring, visual surveillance etc. For crowd behavior analysis, motion analysis is a crucial task which can be achieved with the help of trajectories and tracking of objects. Various approaches have been proposed for crowd behavior analysis which has limitation for densely crowded scenarios, a new object entering the scene etc. In this work, we propose a new approach for abnormal crowd behavior detection. Proposed approach is a motion based spatio-temporal feature analysis technique which is capable of obtaining trajectories of each detected object. We also present a technique to carry out the evaluation of individual object and group of objects by considering relational descriptors based on their environmental context. Finally, a classification is carried out for detection of abnormal or normal crowd behavior by following patch based process. In the results, we have reported that proposed model is able to achieve better performance when compared to existing techniques in terms of classification accuracy, true positive rate, and false positive rate.</p>

Extracting foreground ensemble features to detect abnormal crowd behavior in intelligent video-surveillance systems

Public safety is a matter of national security and people’s livelihoods. In recent years, intelligent video-surveillance systems have become important active-protection systems. A surveillance system that provides early detection and threat assessment could protect people from crowd-related disasters and ensure public safety. Image processing is commonly used to extract features, e.g., people, from a surveillance video. However, little research has been conducted on the relationship between foreground detection and feature extraction. Most current video-surveillance research has been developed for restricted environments, in which the extracted features are limited by having information from a single foreground; they do not effectively represent the diversity of crowd behavior. This paper presents a general framework based on extracting ensemble features from the foreground of a surveillance video to analyze a crowd. The proposed method can flexibly integrate different foreground-detection technologies to adapt to various monitored environments. Furthermore, the extractable representative features depend on the heterogeneous foreground data. Finally, a classification algorithm is applied to these features to automatically model crowd behavior and distinguish an abnormal event from normal patterns. The experimental results demonstrate that the proposed method’s performance is both comparable to that of state-of-the-art methods and satisfies the requirements of real-time applications.

A Method of Detecting Abnormal Crowd Behavior Events Applied in Air Patrol Robot

When the ground or air patrol robot monitors a certain area, one of the important intelligent functions is to monitoring and alerting unusual event of the monitored area. This paper proposes a method via the number of people in the monitored area to detect the environment is safety or not. We use mature methods to formulate the counting problem and the number of people in each frame can be calculated with Gaussian Process Regression model. We suppose that this application may provides an important technical support for enhancing the patrol robot monitoring effect.

Detection and localization of crowd behavior using a novel tracklet-based model

In this paper, two novel descriptors are introduced to detect and localize abnormal behaviors in crowded scenes. The first proposed descriptor is based on the orientation and magnitude of short trajectories extracted by tracking interest points in spatio-temporal 3D patches. The proposed descriptor employs a novel simplified Histogram of Oriented Tracklets (sHOT), which is shown to be very effective in the task of crowd abnormal behavior detection. In this scheme, abnormal behaviors are detected at different levels, namely spatio-temporal level and frame level. By combining the first proposed descriptor and the dense optical flow model, we propose our second framework which is able to localize the abnormal behavior areas in video sequences. The evaluation of our simple but yet effective descriptors on different state-of-the-art datasets, namely UCSD, UMN and Violence in Crowds yields very promising results in abnormality detection and outperforming different former state-of-the-art descriptors.

A Method of Detecting Abnormal Crowd Behavior Events Applied in Air Patrol Robot

A Method of Detecting Abnormal Crowd Behavior Events Applied in Air Patrol Robot

A thermodynamics-inspired feature for anomaly detection on crowd motions in surveillance videos

Identification of abnormal behaviors in surveillance videos of crowds plays an important role in public security monitoring. However, detecting abnormal crowd behaviors is challenging in that movements of individuals are usually random and unpredictable, and the occlusions caused by over-crowding make the task more difficult. In this paper, we introduce thermodynamic micro-statistics theory to detect and localize abnormal behaviors in crowded scenes based on Boltzmann Entropy. For this purpose, the scene of interest is modeled as moving particles turned out from a general optical flow algorithm. The particles are grouped into a set of prototypes according to their speeds and directions of moving, and a histogram is established to figure out how the particles distribute over the prototypes. Here, Boltzmann Entropy is computed from the histogram for each video clip to characterize the chaos degree of crowd motion. By means of such feature extraction, the crowd motion patterns can be represented as a time series. We find that when most people behave anomaly in an area under surveillance, the corresponding entropy value will increase remarkably in comparison with those of normal cases. This motives us to make use of Boltzmann Entropy to distinguish the collective behaviors of people under emergent circumstances from their normal behaviors by evaluating how significantly the current feature value fits into the Gaussian model of normal cases. We validate our method extensively for anomaly detection and localization. The experimental results show promising performance compared with the state of the art methods.

A classification method based on streak flow for abnormal crowd behaviors

An abnormal crowd behavior would harm social public security. Different abnormal crowd behaviors would bring about different harms and subject to different attentions of social public. The higher the harm of the abnormal crowd behavior was, the higher the attention of the social public would be. Therefore, in this paper, a classification method, using crowd density estimation, is proposed for abnormal crowd behaviors. First, in order to enhance the accuracy of the abnormal crowd behavior detection, streak flow, based on fluid mechanics, is introduced to improve the algorithm proposed by Hassner et al. Second, the abnormal crowd behavior in a video scene is detected, the crowd density is estimated and the abnormal behavior is classified on the basis of the crowd density. Lastly, many challenging real-world surveillance videos are used to validate the effectiveness and feasibility of our method for abnormal behavior classification.

Congestion detection of pedestrians using the velocity entropy: A case study of Love Parade 2010 disaster

Gatherings of large human crowds often result in crowd disasters such as the Love Parade Disaster in Duisburg, Germany on July 24, 2010. To avoid these tragedies, video surveillance and early warning are becoming more and more significant. In this paper, the velocity entropy is first defined as the criterion for congestion detection, which represents the motion magnitude distribution and the motion direction distribution simultaneously. Then the detection method is verified by the simulation data based on AnyLogic software. To test the generalization performance of this method, video recordings of a real-world case, the Love Parade disaster, are also used in the experiments. The velocity histograms of the foreground object in the videos are extracted by the Gaussian Mixture Model (GMM) and optical flow computation. With a sequential change-point detection algorithm, the velocity entropy can be applied to detect congestions of the Love Parade festival. It turned out that without recognizing and tracking individual pedestrian, our method can detect abnormal crowd behaviors in real-time.

A Classification Method Based on Multi-Features for Abnormal Crowd Behaviors

An abnormal crowd behavior would harm social public security. Different abnormal crowd behaviors would bring about different harms and subject to different attentions of social public. The higher the harm of the abnormal crowd behavior was, the higher the attention of the social public would be. Therefore, in this paper, a classification method, using crowd density estimation and crowd intensity, is proposed for abnormal crowd behaviors. In order to describe the abnormal crowd behavior more reasonably, we shoot some videos. Finally, the effectiveness and accuracy of our method for abnormal behavior classification is validated by the experimental results.

Abnormal crowd behavior detection by using the particle entropy

The crowd distribution information is the crucial information for abnormal behaviors detection in the crowd scenes. In this paper, we firstly refer to the definition of the entropy and propose an algorithm effectively and accurately representing the crowd distribution information in the crowd scenes. The proposed algorithm not only avoids unstable foreground extraction, but also owns low computational complexity. To detect the abnormal crowd behaviors, we use the Gaussian Mixture Model (GMM) over the normal crowd behaviors to predict the abnormal crowd behaviors since GMM usually can deal well with the unbalanced problem. In this paper we simultaneously use the crowd distribution information and the crowd speed information to estimate the parameters of GMM over the normal crowd behaviors and predict abnormal crowd behaviors. Experiment conducted on publicly available dataset consisting of gathering and dispersion events validates that the proposed approach can preeminently reflect the crowd distribution information. In addition, experiments conducted on publicly UMN dataset demonstrate that the proposed abnormal crowd behavior detection method has an excellent performance and outperforms the state-of-the-art methods.

An Abnormal Crowd Behavior Detection Algorithm Based on Fluid Mechanics

Abnormal crowd behavior detection is an advanced topic researched in fields of computer vision and digital image processing. The problems such as diversity of monitoring scene, different crowd density and mutual occlusion among crowds etc result in a low recognition rate for abnormal crowd behavior detection. In order to solve these problems, this paper combines a streakline model based on fluid dynamics with an abnormal behavior detection method presented by Hassner et al., and proposes a modified algorithm to improve the recognition accuracy of abnormal crowd behavior. Finally, the validity and accuracy of the algorithm are verified via a large amount of challenging real-world surveillance videos.