Crowd Behavior Analysis Research Articles

Identification of crowd behaviour patterns using stability analysis

Crowd behaviour analysis and management have become a significant research problem for the last few years because of the substantial growth in the world population and their security requirements. There are numerous unsolved problems like crowd flow modelling and crowd behaviour detection, which are still open in this area, seeking great attention from the research community. Crowd flow modelling is one of such problems, and it is also an integral part of an intelligent surveillance system. Modelling of crowd flow has now become a vital concern in the development of intelligent surveillance systems. Real-time analysis of crowd behavior needs accurate models that represent crowded scenarios. An intelligent surveillance system supporting a good crowd flow model will help identify the risks in a wide range of emergencies and facilitate human safety. Mathematical models of crowd flow developed from real-time video sequences enable further analysis and decision making. A novel method identifying eight possible crowd flow behaviours commonly seen in the crowd video sequences is explained in this paper. The proposed method uses crowd flow localisation using the Gunnar-Farneback optical flow method. The Jacobian and Hessian matrix analysis along with corresponding eigenvalues helps to find stability points identifying the flow patterns. This work is carried out on 80 videos taken from UCF crowd and CUHK video datasets. Comparison with existing works from the literature proves our method yields better results.

VR Toolkit for Identifying Group Characteristics

Visualising crowds is a key pedestrian dynamics topic, with significant research efforts aiming to improve the current state-of-the-art. Sophisticated visualisation methods are a standard for modern commercial models, and can improve crowd management techniques and sociological theory development. These models often define standard metrics, including density and speed. However, modern visualisation techniques typically use desktop screens. This can limit the capability of a user to investigate and identify key features, especially in real time scenarios such as control centres. Virtual reality (VR) provides the opportunity to represent scenarios in a fully immersive environment, granting the user the ability to quickly assess situations. Furthermore, these visualisations are often limited to the simulation model that has generated the dataset, rather than being source-agnostic. In this paper we implement an immersive, interactive toolkit for crowd behaviour analysis. This toolkit was built specifically for use within VR environments and was developed in conjunction with commercial users and researchers. It allows the user to identify locations of interest, as well as individual agents, showing characteristics such as group density, individual (Voronoi) density and speed. Furthermore, it was used as a data-extraction tool, building individual fundamental diagrams for all scenario agents, and predicting group status as a function of local agent geometry. Finally, this paper presents an evaluation of the toolkit made by crowd behaviour experts.

Motion pattern-based crowd scene classification using histogram of angular deviations of trajectories

Automated crowd behaviour analysis and monitoring is a challenging task due to the unpredictable nature of the crowd within a particular scene and across different scenes. The prior knowledge of the type of scene under consideration is a crucial mid-level information, which could be utilized to develop robust crowd behaviour analysis systems. In this paper, we propose an approach to automatically detect the type of a crowded scene based on the global motion patterns of the objects within the scene. Three different types of scenes whose global motion pattern characteristics vary from uniform to non-uniform are considered in this work, namely structured, semi-structured, and unstructured scenes, respectively. To capture the global motion pattern characteristics of an input crowd scene, we first extract the motion information in the form of trajectories using a key-point tracker and then compute the average angular orientation feature of each trajectory. This paper utilizes these angular features to introduce a novel feature vector, termed as Histogram of Angular Deviations (HAD), which depicts the distribution of the pair-wise angular deviation values for each trajectory vector. Since angular deviation information is resistant to changes in scene perspectives, we consider it as a key feature for distinguishing the scene types. To evaluate the effectiveness of the proposed HAD-based feature vector in classifying the crowded scenes, we build a crowd scene classification model by training the classical machine learning algorithms on the publicly available Collective Motion Database. The experimental results demonstrate the superior crowd classification performance of the proposed approach as compared to the existing methods. In addition to this, we propose a technique based on quantizing the angular deviation values to reduce the feature dimension and subsequently introduce a novel crowd scene structuredness index to quantify the structuredness of an input crowded scene based on its HAD.

FSC-Set: Counting, Localization of Football Supporters Crowd in the Stadiums

Counting the number of people in a crowd has gained attention in the last decade. Due to its benefit to many applications such as crowd behavior analysis, crowd management, and video surveillance systems, etc. Counting crowded scenes, like stadiums, represents a challenging task due to the inherent occlusions and density of the crowd inside and outside the stadiums. Finding a pattern to control thousands of people and counting them is a challenging task. With the introduction of Convolutional Neural Networks (CNN), enables performing this task with acceptable performance. The accuracy of a CNN-based method is related to the size of data used for training. The availability of the dataset is sparse. In particular, there is no dataset in the literature that can be used for training applications for crowd scene. This paper proposes two main contributions including a new dataset for crowd counting, and a CNN-based method for counting the number of people and generating the crowd density maps. The proposed dataset for Football Supporters Crowd (FSC-Set) is composed of 6000 annotated images (manually) of different types of scenes that contain thousands of people gathering in or around the stadiums. FSC-Set contains more than 1.5 Million individuals. The collected images are captured under varying Fields of Views (FOV), illuminations, resolutions, and scales. The proposed dataset can also be utilized for other applications, such as individual’s localization and face detection as well as team recognition from supporter images. Further, we propose a CNN-based method named FSCNet for crowd counting exploiting context-aware attention, spatial-wise attention, and channel-wise attention modules. The proposed method is evaluated on our established FSC-Set and other existing datasets then compared to state-of-the-art methods. The obtained results show satisfactory performances on all the datasets. The dataset is made publicly available and can be requested using the following link: <uri xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">https://sites.google.com/view/fscrowd-dataset/</uri>

Group Behavior Pattern Recognition Algorithm Based on Spatio-Temporal Graph Convolutional Networks

With the rapid growth of population, more diverse crowd activities, and the rapid development of socialization process, group scenes are becoming more common, so the demand for modeling, analyzing, and understanding group behavior data in video is increasing. Compared with the previous work on video content analysis, factors such as the increasing number of people in the group video and the more complex scene make the analysis of group behavior in video face great challenges. Therefore, a group behavior pattern recognition algorithm based on spatio-temporal graph convolutional network is proposed in this paper, aiming at group density analysis and group behavior recognition in the video. A crowd detection and location method based on density map regression-guided classification was designed. Finally, a crowd behavior analysis method based on density grade division was designed to complete crowd density analysis and video group behavior detection. In addition, this paper also proposes to extract spatio-temporal features of crowd posture and density by using the double-flow spatio-temporal map network model, so as to effectively capture the differentiated movement information among different groups. Experimental results on public datasets show that the proposed method has high accuracy and can effectively predict group behavior.

Hybrid Feature-Assisted Neural Model for Crowd Behavior Analysis

Hybrid Feature-Assisted Neural Model for Crowd Behavior Analysis

Machine Learning Based Crowd Behaviour Analysis and Prediction

Crowd behaviour has been widely known to have the ability to forecast the events a crowd could create. Crowd management can become extremely efficient if situations such as riots, mob lynching, traffic jams, accidents, stampede, etc. could be predicted beforehand. To this end many researchers have made their contributions in the past and there is still immense work being carried out currently. All the researches worked with different algorithms and techniques to analyze images or videos of crowd scenes for counting the number of people in the crowd, predicting the behaviour of the crowd and classifying an image or video as normal or abnormal crowd event. This paper, hence, is directed towards underlining the some of the major researches in this field, the approaches and algorithms adopted by them and their comparisons. Overall, this paper reviews the past researches and presents a summary of the techniques and strategies employed. At the end of this paper is the future scope of work possible in the field of crowd behaviour analysis, prediction and crowd counting.

Coherent group detection in still image

Collective behaviors of coherent groups convey semantic relations among individuals in a crowd scene. Detecting coherent groups is primitive for crowd behavior analysis and practically useful in crowd surveillance. However, classically, crowd analysis in still image is focused on crowd counting estimation or crowd segmentation only. In this paper, we present a novel framework that merges these two classical approaches to achieve a higher level of crowd understanding, i.e., detecting coherent groups within a crowd in a still image. Essentially, in addition to crowd counting estimation, our work can infer crowd segments at both image-level and coherent group level. Extensive experiments and analysis of crowd scene images with a variety of crowd densities demonstrate the efficacy of the proposed framework. The proposed framework also shows its potential application, especially in crowd understanding.

Crowd emotion evaluation based on fuzzy inference of arousal and valence

Crowd behavior analysis is an important research topic in the field of video surveillance and public safety management. Crowd emotion has a strong relation with the crowd behavior. However, it is very challenging to predict crowd emotion using conventional emotion clues of humans from the video surveillance data, e.g. facial expression or body gesture. To tackle this challenge, this paper presents a crowd emotion evaluation method using fuzzy inference according to the arousal-valence model of the crowd movement. Specifically, the enthalpy, magnitude variance, confusion index and crowd density are extracted to describe crowd emotion. The enthalpy value and magnitude variance are taken as the input of the fuzzy inference system of arousal. And the confusion index and crowd density are used as the input of the fuzzy system of valence. The arousal value and valence value are the output respectively. Through establishing the relationship between arousal, valence and crowd features, the fuzzy rules are constructed to infer the emotion in the crowd scene. Experimental results show that the proposed method can effectively evaluate the arousal and valence in crowd emotion.

Recent trends in crowd analysis: A review

When overpopulated cities face frequent crowded events like strikes, demonstrations, parades or other sorts of people gatherings, they are confronted to multiple security issues. To mitigate these issues, security forces are often involved to monitor the gatherings and to ensure the security of their participants. However, when access to technology is limited, the security forces can quickly become overwhelmed. Fortunately, more and more important smart cities are adopting the concept of intelligent surveillance systems. In these situations, intelligent surveillance systems require the most advanced techniques of crowd analysis to monitor crowd events properly. In this review, we explore various studies related to crowd analysis. Crowd analysis is commonly broken down into two major branches: crowd statistics and crowd behavior analysis. When crowd statistics determines the Level Of Service (LoS) of a crowded scene, crowd behavior analysis describes the motion patterns and the activities that are observed in a scene. One of the hottest topics of crowd analysis is anomaly detection. Although a unanimous definition of anomaly has not yet been met, each of crowd analysis subtopics can be subjected to abnormality. The purpose of our review is to find subareas, in crowd analysis, that are still unexplored or that seem to be rarely addressed through the prism of Deep Learning.

Visual Analysis and Recognition of Crowd Behavior by Principal Component Analysis

The analysis of the human behavior from video is a wide field of the vision by computer. In this work, we are presenting mainly a new approach and method of detects behavior or abnormal events continuous of crowd in the case of the dangerous situations. These scenes are characterized by the presence of a great number of people in the camera’s field of vision. A major problem is the development of an autonomous approach for the management of a great number of anomalies which is almost impossible to carry out by operators. We present in this paper an approach for the anomalies detection, the visual sequences of the video are detected like behavior normal or abnormal based on the measurement and the extraction of the points by the optical flow, then calculations of the distance between the matrices of covariance of the distributions of the vectors of movement calculated on the consecutive reinforcements.

Monitoring Crowd Movement for Anomaly Detection Using Scale Invariant Feature Transform

In order to detect the potentially dangerous arrears and the situation of a crowd in public security systems, the automated analysis of crowd monitoring using surveillance video is playing vital role. Even though many works are focused on the analysis related to the crowd behavior analysis, complexity in algorithm, real time working module and predefined rigid automatically selection rules are the major problems in the behavior analysis crowd detection. This work proposed a real time algorithm to detect the global anomalies in Scale Invariant Feature Transform(SIFT) based on holistic approach. Significantly deviation in the normal behavior from the previously stored data set, that is people running away from the crowd or suddenly gathering into a particular point were consider an the anomalies are the framework of the approach. The experimental result shows that, compared with the existing methods, the proposed method could able to run in real time and have less complexity in algorithm.

Understanding Individual Mobility Pattern and Portrait Depiction Based on Mobile Phone Data

With the arrival of the big data era, mobile phone data have attracted increasing attention due to their rich information and high sampling rate. Currently, researchers have conducted various studies using mobile phone data. However, most existing studies have focused on macroscopic analysis, such as urban hot spot detection and crowd behavior analysis over a short period. With the development of the smart city, personal service and management have become very important, so microscopic portraiture research and mobility pattern of an individual based on big data is necessary. Therefore, this paper first proposes a method to depict the individual mobility pattern, and based on the long-term mobile phone data (from 2007 to 2012) of volunteers from Beijing as part of project Geolife conducted by Microsoft Research Asia, more detailed individual portrait depiction analysis is performed. The conclusions are as follows: (1) Based on high-density cluster identification, the behavior trajectories of volunteers are generalized into three types, and among them, the two-point-one-line trajectory and evenly distributed behavior trajectory were more prevalent in Beijing. (2) By integrating with Google Maps data, five volunteers’ behavior trajectories and the activity patterns of individuals were analyzed in detail, and a portrait depiction method for individual characteristics comprehensively considering their attributes, such as occupation and hobbies, is proposed. (3) Based on analysis of the individual characteristics of some volunteers, it is discovered that two-point-one-line individuals are generally white-collar workers working in enterprises or institutions, and the situation of a single cluster mainly exists among college students and home freelancer. The findings of this study are important for individual classification and prediction in the big data era and can also provide useful guidance for targeted services and individualized management of smart cities.

Efficient Mining of Hotspot Regional Patterns with Multi-Semantic Trajectories

Due to the explosive growth in trajectory data with multi-semantic labels, mining hotspots based on extracted sequential patterns from multi-semantic trajectories are an emerging need in various applications of crowd behavior analysis, such as location-based advertising, business location, and urban planning. However, most existing regional pattern mining methods only focus on temporal continuity, spatial compactness, and a single semantics. Outdated or nonhot regions may be discovered if the timeliness, multi-semantic features, and user activity of trajectory patterns are not taken into consideration. To address this issue, this paper studies a hotspot regional multi-semantic trajectory pattern mining problem, with the aim of identifying the hotspot pattern of a social crowd in the trajectory data. Specifically, 1) we propose a new spatiotemporal density scheme to quantify the frequency and heat of a specific pattern in space, and to discover all regions where movement behaviors occur densely and have high heat; 2) we develop a trajectory mining algorithm, effective hotspot regional multi-semantic trajectory pattern mining (tHRM), which effectively reveals the hot movement patterns in a region that are not necessarily dominant in intensity. Experiments using real trajectory data sets show that tHRM is able to discover regional hotspot regional patterns, which are difficult to discover with state-of-the-art regional pattern mining methods. Furthermore, tHRM runs 3-4 times faster, and the mining quality is an order of magnitude higher than that of the rival methods.

Revisiting crowd behaviour analysis through deep learning: Taxonomy, anomaly detection, crowd emotions, datasets, opportunities and prospects

Crowd behaviour analysis is an emerging research area. Due to its novelty, a proper taxonomy to organise its different sub-tasks is still missing. This paper proposes a taxonomic organisation of existing works following a pipeline, where sub-problems in last stages benefit from the results in previous ones. Models that employ Deep Learning to solve crowd anomaly detection, one of the proposed stages, are reviewed in depth, and the few works that address emotional aspects of crowds are outlined. The importance of bringing emotional aspects into the study of crowd behaviour is remarked, together with the necessity of producing real-world, challenging datasets in order to improve the current solutions. Opportunities for fusing these models into already functioning video analytics systems are proposed.

Multi-scale crowd feature detection using vision sensing and statistical mechanics principles

Crowd behaviour analysis using vision has been subject to many different approaches. Multi-purpose crowd descriptors are one of the more recent approaches. These descriptors provide an opportunity to compare and categorize various types of crowds as well as classify their respective behaviours. Nevertheless, the automated calculation of descriptors which are expressed as measurements with accurate interpretation is a challenging problem. In this paper, analogies between human crowds and molecular thermodynamics systems are drawn for the measurement of crowd behaviour. Specifically, a novel descriptor is defined and measured for crowd behaviour at multiple scales. This descriptor uses the concept of Entropy for evaluating the state of crowd disorder. By results, the descriptor Entropy does indeed appear to capture the desired outcome for crowd entropy while utilizing easily detectable image features. Our new approach for machine understanding of crowd behaviour is promising, while it offers new complementary capabilities to the existing crowd descriptors, for example, as will be demonstrated, in the case of spectator crowds. The scope and performance of this descriptor are further discussed in detail in this paper.

Motion estimation of high density crowd using fluid dynamics

ABSTRACT Motion estimation (ME) being a fundamental process of crowd behavior analysis experienced real challenges at high densities due to visual ambiguities and occlusion problems etc. Various surveys reported in the past years summarize conventional ME methods for crowd behaviors at low/medium densities. In this paper, we focus on state-of-the-art fluid dynamics (FD) ME methods developed over the last one and the half-decade for high-density crowd analysis. A detailed discussion is provided on the development of FD ME methods explaining the strengths and weaknesses and viability of FD ME methods for anomaly detection at high crowd densities. Comprehensive experiments are performed comparing the performance of conventional and FD ME at varying crowd densities. Experimentation results show that conventional ME methods fail at high-density crowd whereas FD ME methods could estimate motion only at the global level. Still, research is required to meet the challenges of local ME at high crowd densities.

A novel framework and concept-based semantic search Interface for abnormal crowd behaviour analysis in surveillance videos

Monitoring continuously captured surveillance videos is a challenging and time consuming task. To assist this issue, a new framework is introduced that applies anomaly detection, semantic annotation and provides a concept-based search interface. In particular, novel optical flow based features are used for abnormal crowd behaviour detection. Then, processed surveillance videos are annotated using a new semantic metadata model based on multimedia standards using Semantic Web technologies. In this way, globally inter-operable metadata about abnormal crowd behaviours are generated. Finally, for the first time, based on crowd behaviours, a novel concept-based semantic search interface is proposed. In the proposed interface, along with search results (video segments), statistical data about crowd behaviours are also presented. With extensive user studies, it is demonstrated that the proposed concept-based semantic search interface enables efficient search and analysis of abnormal crowd behaviours. Although there are existing works to achieve (a) crowd anomaly detection, (b) semantic annotation and (c) semantic search interface, none of the existing works combine these three system components in a novel framework like the one proposed in this paper. In each system component, we introduce contributions to the field as well as use the Semantic Web technologies to combine and standardize output of different system components; output of the anomaly detection is automatically annotated with metadata and stored to a semantic database. When continuous surveillance videos are processed, only the semantic database is updated. Finally, the user interface queries the updated database for searching/analyzing surveillance videos without changing any coding. Thus, the framework supports re-usability. This paper explains and evaluates different components of the framework.

Analysis of crowd behavior through pattern virtualization.

Abstract The study of the concentration of individuals in public places such as squares, shopping malls, parks, gardens, etc., is an open study field in the different disciplines of science, that leads to the need of having systems that allow to forecast and to predict eventualities in uncontrolled situations, as it is the case of an earthquake. From that assumption, artificial intelligence, as a branch of computational sciences, studies the human behavior in a virtual way in order to obtain simulations based on social, psychological, neuro-scientific areas, among others, with the purpose of linking these theories to the area of artificial intelligence. This paper presents a way to generate virtual multitudes with heterogeneous behaviors, in such a way that the individuals that form the multitude present different behaviors.

Review of pedestrian tracking: Algorithms and applications

Pedestrian tracking is a hotspot and a difficult topic in computer vision research. Through the tracking of pedestrians in video materials, trajectories can be extracted to support the analysis of individual or collected behavior dynamics. In this review, we first discuss the difference between pedestrian tracking and pedestrian detection. Then we summarize the development of traditional tracking algorithms and deep learning-based tracking algorithms, and introduce classic pedestrian dynamic models. In the end, typical applications, including intelligent monitoring, congestion analysis, and anomaly detection are introduced systematically. With the rising use of big data and deep learning techniques in the area of computer vision, the research on pedestrian tracking has made a leap forward, which can support more accurate, timely extraction of behavior patterns and then to facilitate large-scale dynamic analysis of individual or crowd behavior.