Crowd Analysis Research Articles

A Wide Area Multiview Static Crowd Estimation System Using UAV and 3D Training Simulator

Crowd size estimation is a challenging problem, especially when the crowd is spread over a significant geographical area. It has applications in monitoring of rallies and demonstrations and in calculating the assistance requirements in humanitarian disasters. Therefore, accomplishing a crowd surveillance system for large crowds constitutes a significant issue. UAV-based techniques are an appealing choice for crowd estimation over a large region, but they present a variety of interesting challenges, such as integrating per-frame estimates through a video without counting individuals twice. Large quantities of annotated training data are required to design, train, and test such a system. In this paper, we have first reviewed several crowd estimation techniques, existing crowd simulators and data sets available for crowd analysis. Later, we have described a simulation system to provide such data, avoiding the need for tedious and error-prone manual annotation. Then, we have evaluated synthetic video from the simulator using various existing single-frame crowd estimation techniques. Our findings show that the simulated data can be used to train and test crowd estimation, thereby providing a suitable platform to develop such techniques. We also propose an automated UAV-based 3D crowd estimation system that can be used for approximately static or slow-moving crowds, such as public events, political rallies, and natural or man-made disasters. We evaluate the results by applying our new framework to a variety of scenarios with varying crowd sizes. The proposed system gives promising results using widely accepted metrics including MAE, RMSE, Precision, Recall, and F1 score to validate the results.

Group Behavior Pattern Recognition Algorithm Based on Spatio-Temporal Graph Convolutional Networks

With the rapid growth of population, more diverse crowd activities, and the rapid development of socialization process, group scenes are becoming more common, so the demand for modeling, analyzing, and understanding group behavior data in video is increasing. Compared with the previous work on video content analysis, factors such as the increasing number of people in the group video and the more complex scene make the analysis of group behavior in video face great challenges. Therefore, a group behavior pattern recognition algorithm based on spatio-temporal graph convolutional network is proposed in this paper, aiming at group density analysis and group behavior recognition in the video. A crowd detection and location method based on density map regression-guided classification was designed. Finally, a crowd behavior analysis method based on density grade division was designed to complete crowd density analysis and video group behavior detection. In addition, this paper also proposes to extract spatio-temporal features of crowd posture and density by using the double-flow spatio-temporal map network model, so as to effectively capture the differentiated movement information among different groups. Experimental results on public datasets show that the proposed method has high accuracy and can effectively predict group behavior.

Crowd estimation using key‐point matching with support vector regression

The crowd behaviour understanding and density estimation are some of the fast-growing fields in video surveillance. There are many techniques (detection and regression) that are used as the method of crowd analysis and estimation. In the present approach, SVR (support vector regression) is used as the basic analysis technique and the novel key-point matching with SURF (speedup robust feature) is used as the feature extractor for moving objects in the video. The traditional linear regression methods used mainly key-point as one of the statistical features instead of matching with consecutive frames, but we used the magnitude of the optical flow for foreground object extraction instead of inter-frame difference. The combination of the optical flow of foreground objects and key-point matching generates new features apart from conventional features such as areas and corners. In this new approach, key-point pairing with linear regression is tested with the PETS2009 dataset, and performance is compared with the existing approaches.

Comparative Analysis of Convolutional Neural Network Architectures for Real Time COVID-19 Facial Mask Detection

The late 2019 outbreak of Coronavirus Disease (COVID-19) had an indelible imprint on the humanity. The world is recovering from the outbreak but there is danger of a second wave of the outbreak. To get rid of the outbreak it is necessary to prevent the viral transmission and it is need of the hour to maintain social distancing and wear masks in public areas. The governments are providing strict guidelines to wear masks in public places. It is not manually feasible to check if people are wearing masks or not. In this paper, process of detecting face masks in public places is automated using Convolutional Neural Networks by performing comparative analysis on Sequential bi-layered CNN, VGG-16 CNN and MobileNetV2 CNN architectures. Among these three architectures MobileNetV2 outperformed with a performance accuracy of 99.2%. The efficient Deep Learning architecture of detecting face masks can be achieved with the help of IoT (Internet of Things) devices and cameras, of those who are not following guidelines in public places. Such a system is very useful in post outbreak period and can be installed in public places such as Railway Stations, Airports, Parks, Schools, colleges, offices etc. to track and ensure wearing of masks by people. The contribution of this paper is not to reel-off the finding from the original paper on Face Mask detection with various architectures rather to provide results on the efficiency of using the MobileNetV2 architecture in comparison with Sequential CNN and VGG-16 architectures for crowd analysis mask detection.

From ‘the people’ to the crowd: The push for independence in Catalonia

This article examines the sociological value of Elias Canetti’s work on crowds and power. It explores crowd action and imagery in the push for Catalan independence through the analysis of materials published on Twitter by Tsunami Democràtic, which emerged to coordinate the response to the sentencing of Catalan political leaders after the unilateral declaration of independence. It then goes on to discuss how a crowd-based approach offers a supplementary perspective to contemporary studies of populism, on the one hand, and to accounts that primarily focus on the role of social media in organizing political protest movements, on the other. An analysis of crowds not only avoids both methodological holism and methodological individualism. It also helps to understand why so many people were mobilized beyond the power of concepts, ideologies and discourse.

SOCIAL DISTANCING DETECTOR USING DEEP LEARNING

This paper presents a methodology for detecting social distance using deep learning and computer vision between people to control the spread of covid-19. This application is developed to give alerts to people for maintaining social distance in crowded places. By using pre-recorded video as input and the open-source object detection pretrained model using the YOLOv3 algorithm We can tell if people are following social distancing or not and based on that we are creating red or green bounding boxes over it. It is also working on web cameras, CCTV, etc, and can detect people in real-time. This may help authorities to redesign the layout of public places or to take precautionary actions to mitigate high-risk zones.it can be used in other fields also like autonomous vehicles, human action recognition, crowd analysis.

The knowledge domain of crowd dynamics: Anatomy of the field, pioneering studies, temporal trends, influential entities and outside-domain impact

The literature of pedestrian, crowd and evacuation dynamics is captured in its approximate full scope and is analysed at various levels using scientometric indicators of its underlying articles (N≈6200). The analyses provide new insight into the structural make-up of this field, its distribution across various disciplines, its temporal and historical patterns of development, as well as its pioneering and influential entities (i.e., articles and authors). The analysis establishes that the field has exerted a high degree of influence beyond its borders while identifying these areas of influence. Studies with greatest impact within and beyond the borders of crowd dynamics literature; as well as, pioneering but neglected studies of the field are identified. It is determined that the beginning of the twenty-first century marks the most important milestone of this field, an intellectual turning point that set the field to become a standalone research domain. Temporal analysis indicates certain paradigm shifts in crowed dynamics research during the past decade. It identifies two streams of activities labelled “empirical methods” and “crowd counting/visual crowd analysis” as the two youngest and currently hottest research streams of this field. Outcomes suggest that further interactions and collaborations between the computer vision and the mainstream of crowd researchers could be warranted. This could lead to the next generations of data-driven crowd models and prevent the field from going to a state of stagnation. It is also hoped that these outcomes contribute to enhancing the quality (i.e., specificity and inclusiveness) of document referencing in crowd dynamics papers.

Adaptive colour restoration and detail retention for image enhancement

Computer vision-based crowd understanding and analysis technology has been widely used in public safety due to the rapid growth of population and the frequent occurrence of various accidents. Improving imaging quality is the key to improve the performance of crowd analysis, density estimation, target recognition, segmentation, and detection in computer vision tasks. Due to the complex imaging environment such as fog and low illumination, some images taken in outdoor environment often have the problems of colour distortion, lack of details, and the poor imaging quality, which affect the subsequent visual tasks. To improve the imaging quality and visual effect, an adaptive colour restoration and detail retention-based method is proposed for image enhancement. First, to overcome the problem of colour distortion caused by low illumination and fog, a multi-channel fusion based adaptive image colour restoration method is proposed. To make the enhancement result more consistent with human observation, the detail retention-based method is applied to enhance the details. Experimental results demonstrate that the authors' results are effective and outperform the compared methods both in visual and objective evaluations.

Towards Privacy Paradigm Shift Due to the Pandemic: A Brief Perspective

With the advent of the pandemic (e.g., novel corona virus disease 2019 (COVID-19)), a tremendous amount of data about individuals are collected by the health authorities on daily basis for curbing the disease’s spread. The individuals’ data collection/processing at a massive scale for community well-being with the help of digital solutions (e.g., mobile apps for mobility and proximity analysis, contact tracing through credit card usage history, facial recognition through cameras, and crowd analysis using cellular networks data etc.) raise several privacy concerns. Furthermore, the privacy concerns that are arising mainly due to the fine-grained data collection has hindered the response to tackle this pandemic in many countries. Hence, acquiring/handling individuals data with privacy protection has become a vibrant area of research in these pandemic times. This paper explains the shift in privacy paradigm due to the pandemic (e.g., COVID-19) which involves more and detailed data collection about individuals including locations and demographics. We explain technical factors due to which the people’s privacy is at higher risk in the COVID-19 time. In addition, we discuss privacy concerns in different epidemic control measures (ECMs) (e.g., contact tracing, quarantine monitoring, and symptoms reporting etc.) employed by the health authorities to tackle this disease. Further, we provide an insight on the data management in the ECMs with privacy protection. Finally, the future prospects of the research in this area tacking into account the emerging technologies are discussed. Through this brief article, we aim to provide insights about the vulnerability to user’s privacy in pandemic times, likely privacy issues in different ECMs adopted by most countries around the world, how to preserve user’s privacy effectively in all phases of the ECMs considering relevant data in loop, and conceptual foundations of ECMs to fight with future pandemics in a privacy preserving manner.

Coherent group detection in still image

Collective behaviors of coherent groups convey semantic relations among individuals in a crowd scene. Detecting coherent groups is primitive for crowd behavior analysis and practically useful in crowd surveillance. However, classically, crowd analysis in still image is focused on crowd counting estimation or crowd segmentation only. In this paper, we present a novel framework that merges these two classical approaches to achieve a higher level of crowd understanding, i.e., detecting coherent groups within a crowd in a still image. Essentially, in addition to crowd counting estimation, our work can infer crowd segments at both image-level and coherent group level. Extensive experiments and analysis of crowd scene images with a variety of crowd densities demonstrate the efficacy of the proposed framework. The proposed framework also shows its potential application, especially in crowd understanding.

Comparison between Recurrent Networks and Temporal Convolutional Networks Approaches for Skeleton-Based Action Recognition.

Action recognition plays an important role in various applications such as video monitoring, automatic video indexing, crowd analysis, human-machine interaction, smart homes and personal assistive robotics. In this paper, we propose improvements to some methods for human action recognition from videos that work with data represented in the form of skeleton poses. These methods are based on the most widely used techniques for this problem—Graph Convolutional Networks (GCNs), Temporal Convolutional Networks (TCNs) and Recurrent Neural Networks (RNNs). Initially, the paper explores and compares different ways to extract the most relevant spatial and temporal characteristics for a sequence of frames describing an action. Based on this comparative analysis, we show how a TCN type unit can be extended to work even on the characteristics extracted from the spatial domain. To validate our approach, we test it against a benchmark often used for human action recognition problems and we show that our solution obtains comparable results to the state-of-the-art, but with a significant increase in the inference speed.

Automatic Emotion Recognition for Groups: A Review

This review aims to summarize and describe research on the topic of automatic group emotion recognition. In recent years, the topic of emotion analysis of groups or crowds has gained interest, with studies performing emotion detection in different contexts, using different datasets and modalities (such as images, video, audio, social media messages), and taking different approaches. Articles are included after an innovative search method, including Dense Query Extraction and automatic cross-referencing. Discussed are the types of groups and emotion models considered in automatic emotion recognition research, common datasets for all modalities, general approaches taken, and reported performances. These performances are discussed, followed by an analysis of the application possibilities of the discussed methods. To ensure clear, replicable, and comparable studies, we suggest research should test on multiple, common datasets and report on multiple metrics, when possible. Implementation details and code should be made available where possible. An area of interest for future work is to build systems with more real-world application possibilities, coping with changing group sizes, different emotional subgroups, and changing emotions over time, while having a higher robustness and working with datasets with reduced biases.

Recent trends in crowd analysis: A review

When overpopulated cities face frequent crowded events like strikes, demonstrations, parades or other sorts of people gatherings, they are confronted to multiple security issues. To mitigate these issues, security forces are often involved to monitor the gatherings and to ensure the security of their participants. However, when access to technology is limited, the security forces can quickly become overwhelmed. Fortunately, more and more important smart cities are adopting the concept of intelligent surveillance systems. In these situations, intelligent surveillance systems require the most advanced techniques of crowd analysis to monitor crowd events properly. In this review, we explore various studies related to crowd analysis. Crowd analysis is commonly broken down into two major branches: crowd statistics and crowd behavior analysis. When crowd statistics determines the Level Of Service (LoS) of a crowded scene, crowd behavior analysis describes the motion patterns and the activities that are observed in a scene. One of the hottest topics of crowd analysis is anomaly detection. Although a unanimous definition of anomaly has not yet been met, each of crowd analysis subtopics can be subjected to abnormality. The purpose of our review is to find subareas, in crowd analysis, that are still unexplored or that seem to be rarely addressed through the prism of Deep Learning.

Abnormal crowd density estimation in aerial images based on the deep and handcrafted features fusion

Abnormal crowd analysis has always been an interesting research area since it ensures people’s safety and prevents crowd disasters in large scale events. In particular, crowd density estimation in aerial images has the advantage of monitoring open areas by covering a very large view and reaching difficult access areas. In this context, we proposed a new method for crowd density estimation in aerial images in order to detect crowded areas showing abnormal densities. The proposed method consists of two phases: an offline phase and an inference phase. The offline phase aimed to generate a crowd model using a fusion of relevant deep and handcrafted features selected via the minimum-redundancy maximum-relevance (mRMR) technique. However, in the inference phase, we made use of the already generated model to classify the aerial images patches into four classes: None, Sparse, Medium and Dense. Our main contributions lie in the fact we relied on the fusion of semantic and low-level information to encode crowd density patches, as well as, the definition of the most salient crowd features to reduce confusion between crowd density classes. The experimental evaluation of the proposed method proves the validity of these contributions as well as the effectiveness and efficiency of our method compared to the state-of-the-art reference methods.

Characterization of different crowd behaviors using novel deep learning framework

Crowd behavior understanding is recognized as a complex problem due to unpredictable behavior of humans and complex interactions of individuals in groups. For crowd managers, it is crucial to understand the crowd dynamics to manage the crowd efficiently and effectively. Current practice of crowd management is based on manual analysis of the scene. Such manual analysis of the scene is a tedious job and usually prone to errors due to limited human capabilities. Therefore, the task of automatizing crowd analysis has received tremendous attention from the research community during the recent years. In this paper, we propose a deep model framework that automatically characterizes different crowd behaviors based on motion and appearance. We first extract dense trajectories from the input video segment and then generate trajectory image by projecting trajectories on to image plane. Trajectory image effectively captures relative motion in the scene. We use stack of trajectory images to train deep convolutional network that learns compact and powerful representation of motion in the scene. We evaluate our approach on UCF, CUHK, and Crowd-11 benchmark datasets. From the experiment results, we demonstrate, both in quantitative and qualitative ways, that the proposed framework outperforms other existing methods by a great margin.

Video Processing Using Deep Learning Techniques: A Systematic Literature Review

Studies show lots of advanced research on various data types such as image, speech, and text using deep learning techniques, but nowadays, research on video processing is also an emerging field of computer vision. Several surveys are present on video processing using computer vision deep learning techniques, targeting specific functionality such as anomaly detection, crowd analysis, activity monitoring, etc. However, a combined study is still unexplored. This paper aims to present a Systematic Literature Review (SLR) on video processing using deep learning to investigate the applications, functionalities, techniques, datasets, issues, and challenges by formulating the relevant research questions (RQs). This systematic mapping includes 93 research articles from reputed databases published between 2011 and 2020. We categorize the deep learning technique for video processing as CNN, DNN, and RNN based. We observe significant advancements in video processing between 2017 and 2020, primarily due to the advent of AlexNet, ResNet, and LSTM based deep learning techniques. The prominent fields of video processing research are observed as human action recognition, crowd anomaly detection, and behavior analysis. This SLR is a helpful guide for the researchers to explore the recent literature, available datasets, and existing deep learning techniques for video processing.

Crowd Density Estimation by Using Attention Based Capsule Network and Multi-Column CNN

We propose a strategy that focuses on estimating the number of people in a crowd, one of the aims of crowd analysis, using static images or video images. While manual feature extraction was not performed with pixel and regression-based methods in the first studies on crowd analysis, recent studies use Convolutional Neural Networks (CNN) based models. However, it is still difficult to extract spatial information such as position, orientation, posture, and angular value for crowd estimation from a density map. This study uses capsule networks and routing by agreement algorithm as an attention module. Our proposed approach consists of both CNN and capsule network-based attention modules in a two-column deep neural network architecture. We evaluate our proposed approach compared with other state-of-the-art methods using three well-known datasets: UCF-QNRF, UCF_CC_50, UCSD, ShangaiTech Part A, and WorldExpo’10.

The Use of Input Words Vectorsto Detect Crowd Region in Images

Human crowd analysis has common utilizations from the urban engineering and traffic management to law enforcement. They all need a crowd for first being detected, and is the issue that has been dealt with in the present study. Considering an image, the algorithm that has been proposed in this paper performs a segmentation of that image to crowd and non-crowd areas. The fundamental concept is capturing two main characteristics of the crowd: (a) on a narrower scale, its main elements have to appear like humans (only weakly so, as a result of the low resolution, dressing variations, occlusion, and so on), whereas (b) on the wider scale, the crowd intrinsically includes elements of the redundant appearance. The proposed approach makes use of that through the utilization of underlying statistical framework which has been based on the quantized features of the SURF. The two previously mentioned characteristics of the crowds have been obtained through the resultant statistical model responses’ feature vector, which describe the level of crowd-like appearances around the location of an image with the increase of the spatial level around it.

CrowdGAN: Identity-Free Interactive Crowd Video Generation and Beyond.

In this paper, we introduce a novel yet challenging research problem, interactive crowd video generation, committed to producing diverse and continuous crowd video, and relieving the difficulty of insufficient annotated real-world datasets in crowd analysis. Our goal is to recursively generate realistic future crowd video frames given few context frames, under the user-specified guidance, namely individual positions of the crowd. To this end, we propose a deep network architecture specifically designed for crowd video generation that is composed of two complementary modules, each of which combats the problems of crowd dynamic synthesis and appearance preservation respectively. Particularly, a spatio-temporal transfer module is proposed to infer the crowd position and structure from guidance and temporal information, and a point-aware flow prediction module is presented to preserve appearance consistency by flow-based warping. Then, the outputs of the two modules are integrated by a self-selective fusion unit to produce an identity-preserved and continuous video. Unlike previous works, we generate continuous crowd behaviors beyond identity annotations or matching. Extensive experiments show that our method is effective for crowd video generation. More importantly, we demonstrate the generated video can produce diverse crowd behaviors and be used for augmenting different crowd analysis tasks, i.e., crowd counting, anomaly detection, crowd video prediction. Code is available at https://github.com/Icep2020/CrowdGAN.

Adaptive weighted crowd receptive field network for crowd counting

Crowd counting plays an important role in crowd analysis and monitoring. To this end, we propose a novel method called Adaptive Weighted Crowd Receptive Field Network (AWRFN) for crowd counting to estimate the number of people and the spatial distribution of input crowd images. The proposed AWRFN is composed of four modules: backbone, crowd receptive field block (CRFB), recurrent block (RB), and channel attention block (CAB). Backbone utilizes the first ten layers of VGG16 to extract base features of input images. CRFB is a multi-branch architecture simulating a real human visual system for further obtaining refined and discriminative crowd features. RB generates strong semantic and global information by recurrently stacking convolutional layers with the same parameters. CAB outputs appropriate weights to supervise each channel of the feature maps output from CRFB, which uses the outputs of RB as guidance. Different from previous works using Euclidean Loss, we employ L1_Smooth Loss to train our network in an end-to-end fashion. To demonstrate the effectiveness of our proposed method, we implement AWRFN on two representative datasets including the ShanghaiTech dataset and the UCF_CC_50 dataset. The experimental results prove that our method is both effective and robust compared with the state-of-the-art approaches.