Crowd Analysis Research Articles

Tasks of visual crowd analysis in intelligent video surveillance systems

Modern urban population growth creates challenges for public safety, in particular due to crowds. This stimulates the development of new crowd management methods that require automated analysis. Visual crowd analysis based on computer vision technologies is a key tool for solving these problems. The development of deep learning has significantly improved the monitoring systems used for urban surveillance, social distancing control, transportation and event management. However, crowd analysis remains challenging due to occlusions, scale variations, unpredictable movement patterns, and complex behavior. To overcome these challenges, new algorithms, models, and large-scale datasets are needed to enable real-time analysis. The main tasks include people counting, object detection, motion analysis, behavior recognition, and anomaly detection. Deep neural networks and transfer learning significantly increase the accuracy and adaptability of such systems, which helps to improve public safety and manage the flow of people.

A SUPERVISED ONLINE CROWD ANALYSIS NETWORK WITH DUAL-TASK DEEP LEARNING IDEA IN HEALTHCARE APPLICATION

A SUPERVISED ONLINE CROWD ANALYSIS NETWORK WITH DUAL-TASK DEEP LEARNING IDEA IN HEALTHCARE APPLICATION

Learning Models in Crowd Analysis: A Review

Learning Models in Crowd Analysis: A Review

Object detection and crowd analysis using deep learning techniques: Comprehensive review and future directions

Object detection using deep learning has attracted considerable interest from researchers because of its competency in performing state-of-the-art tasks, including detection, observation, and action recognition. Deep Learning (DL)-based object detection models extract features directly from data, which is more efficient and effective than traditional methods requiring handcrafted features. Further, DL also effectively tackles spatiotemporal challenges by leveraging techniques; hence, researchers can develop better object detection models and implement more efficient strategies for object recognition. Moreover, optimizing these models improves performance and recognizes objects within videos or images. This survey comprises of an overview of related review papers and DL-based Object Detection (OD) algorithms. Object detection algorithms are presented as two classifications, namely Two-stage and One-stage methods, with Convolutional Neural Network (CNN) as the backbone. OD’s applications are examined here, and crowd analysis has been extensively studied and researched for potential applications. Most of the papers in object detection rely on Convolutional Neural Networks (CNNs) (28%), whereas crowd analysis papers are distributed as follows: 24% in counting, 25% in categorizing, and 25% in analyzing individual behaviors, and 27% in others. In recent years, deep learning has significantly advanced object detection capabilities to provide effective solutions for various applications, including crowd analysis.

Survey of Techniques and Applications in Object Recognition

Abstract: Object detection, which is closely related to video analysis and image comprehension, has received attention in years from researchers. Traditional approaches to object detection rely on crafted features and shallow trainable architectures. However, these methods face challenges. Often reach a plateau, in performance improvement. To overcome these limitations complex ensembles are often created by combining low-level image features with high-level context from object detectors and scene classifiers. The emergence of learning has introduced powerful tools that can learn semantics high level and deeper features. These tools address the limitations of architectures by introducing network structures training strategies and optimization functions. This paper provides a review of learning-based frameworks for object detection. The review starts with an overview of the history of learning with a particular emphasis on the Convolutional Neural Network (CNN) as an exemplary tool. It then delves into generic object detection architectures exploring modifications and effective strategies for improving detection performance. Recognizing the characteristics of detection tasks, the paper also briefly surveys several tasks such, as salient object detection, face detection, crowd analysis, and pedestrian detection.

Generalized Characteristic Function Loss for Crowd Analysis in the Frequency Domain.

Typical approaches that learn crowd density maps are limited to extracting the supervisory information from the loosely organized spatial information in the crowd dot/density maps. This paper tackles this challenge by performing the supervision in the frequency domain. More specifically, we devise a new loss function for crowd analysis called generalized characteristic function loss (GCFL). This loss carries out two steps: 1) transforming the spatial information in density or dot maps to the frequency domain; 2) calculating a loss value between their frequency contents. For step 1, we establish a series of theoretical fundaments by extending the definition of the characteristic function for probability distributions to density maps, as well as proving some vital properties of the extended characteristic function. After taking the characteristic function of the density map, its information in the frequency domain is well-organized and hierarchically distributed, while in the spatial domain it is loose-organized and dispersed everywhere. In step 2, we design a loss function that can fit the information organization in the frequency domain, allowing the exploitation of the well-organized frequency information for the supervision of crowd analysis tasks. The loss function can be adapted to various crowd analysis tasks through the specification of its window functions. In this paper, we demonstrate its power in three tasks: Crowd Counting, Crowd Localization and Noisy Crowd Counting. We show the advantages of our GCFL compared to other SOTA losses and its competitiveness to other SOTA methods by theoretical analysis and empirical results on benchmark datasets. Our codes are available at https://github.com/wbshu/Crowd_Counting_in_the_Frequency_Domain.

Crowd Analysis and Tracking for Individual Rescue Operation Using Machine Learning

Abstract: Crowd management and rescue operations in densely populated areas pose a significant challenge, particularly in the context of identifying and aiding individuals in distress. By harnessing machine learning, our approach continuously analyzes specific crowd segments, focusing on tracking various hand gestures and their corresponding actions. This data-driven system enables swift gesture recognition and provision of essential support to individuals in need within these defined crowd areas. The paper presents the methodology, experimental results, and future directions for this innovative approach, which holds significant potential for enhancing public safety and optimizing emergency responses in high-density settings

NEW ERA OF VISION TO ENVISION USING YOLO

We provide a new approach to crowd counting in this research that makes use of the You Only Look Once (YOLO) algorithm. We show how well YOLO performs in precisely identifying and counting people in congested environments. Our method seeks to overcome the difficulties associated with crowd observation and analysis in real time. Urban planning, public safety, and event management are just a few of the many areas where crowd counting is important and dynamic. Occlusions, size variations, and congested settings are only a few of the challenges that traditional approaches frequently face in providing precise and timely crowd estimates. This study presents a unique method for accurate and efficient crowd counting that leverages the You Only Look Once (YOLO) algorithm. Through the utilization of YOLO's sophisticated object identification capabilities, our suggested approach provides a reliable way to compute crowd sizes in real-time situations. We provide an in-depth examination of the difficulties involved in crowd counting and show, by intensive testing and assessment on many datasets, how successful our YOLO-based method is. Our findings demonstrate the People counting, Image-based crowd estimation, Video analytics, Data preprocessing, Performance evaluation. Keywords— Crowd counting, Object detection, YOLO (You Only Look Once), Deep learning, Computer vision, Convolutional neural networks (CNNs), Image processing, Crowd analysis, Real-time monitoring, Surveillance systems,

Scale Attentive Aggregation Network for Crowd Counting and Localization in Smart City

Recent years have witnessed a remarkable proliferation of applications in smart cities. Crowd analysis is a crucial subject, and it incorporates two subtasks in smart city systems, i.e. , crowd counting and crowd localization. Nevertheless, the presence of adverse intrinsic factors, i.e. , scale variation and background noise severely degrades the performance of counting and localization. Although great efforts have been made on separate research on counting and localization, few works are capable of performing both tasks at the same time. To this aim, the scale attentive aggregation network (SA 2 Net) is proposed to solve the problems of scale variation and background noise in crowd counting and localization tasks synchronously. Specifically, the SA 2 Net has two vital modules, namely multiscale feature aggregator (MFA) module and background noise suppressor (BNS) module. The MFA module is designed in a four-pathway structure, and it aggregates the multiscale feature so as to facilitate the correlation between different scales. The BNS module utilizes the contextual information between the input keys matrix and self-attention matrix to suppress the background noise. Furthermore, a global consistency loss combined with the Euclidean loss is utilized to optimize the network in counting and localization tasks. Extensive experimental results prove that the SA 2 Net outperforms the state-of-the-art competitors both subjectively and objectively.

Audio–visual representation learning for anomaly events detection in crowds

In recent years, anomaly events detection in crowd scenes attracts many researchers’ attentions, because of its importance to public safety. Existing methods usually exploit visual information to analyze whether any abnormal events have occurred due to only visual sensors are generally equipped in public places. However, when an abnormal event in crowds occurs, sound information may be discriminative to assist the crowd analysis system to determine whether there is an abnormality. Compared with vision information that is easily occluded, audio signals have a certain degree of penetration. Thus, this paper attempt to exploit multi-modal learning for modeling the audio and visual signals simultaneously. To be specific, we design a two-branch network to model different types of information. The first is a typical 3D CNN model to extract temporal appearance feature from video clips. The second is an audio CNN for encoding Log Mel-Spectrogram of audio signals. Finally, by fusing the above features, the more accurate prediction will be produced. We conduct the experiments on SHADE dataset, a synthetic audio–visual dataset in surveillance scenes, and find introducing audio signals effectively improves the performance of anomaly events detection and outperforms other state-of-the-art methods. Furthermore, we will release the code and the pre-trained models as soon as possible.

Real-Time Spatial-Temporal Depth Separable CNN for Multi-Functional Crowd Analysis in Videos

Crowd behavior prediction (CBP) and crowd counting (CC) are the essential functions of vision-based crowd analysis (CA), which play a crucial role in controlling crowd disasters. The CA using different models for the CBP and the CC will increase computational overheads and have synchronization issues. The state-of-the-art approaches utilized deep convolutional architectures to exploit spatial-temporal features to accomplish the objective, but such models suffer from computational complexities during convolution operations. Thus, to sort out the issues as mentioned earlier, this paper develops a single deep model which performs two functionalities of CA: CBP and CC. The proposed model uses multilayers of depth-wise separable CNN (DSCNN) to extract fine-grained spatial-temporal features from the scene. The DSCNN can minimize the number of matrix multiplications during convolution operation compared to traditional CNN. Further, the existing datasets are available to accomplish the single functionality of CA. In contrast, the proposed model needs a dual-tasking CA dataset which should provide the ground-truth labels for CBP and CC. Thus, a dual functionality CA dataset is prepared using a benchmark crowd behavior dataset, i.e. MED. Around 41[Formula: see text]000 frames have been manually annotated to obtain ground-truth crowd count values. This paper also demonstrates an experiment on the proposed multi-functional dataset and outperforms the state-of-the-art methods regarding several performance metrics. In addition, the proposed model processes each test frame at 3.40 milliseconds, and thus is easily applicable in real-time.

Scale-Context Perceptive Network for Crowd Counting and Localization in Smart City System

The task of crowd counting and localization is to predict the count and position of people in a crowd, which is a practical and essential sub-task in crowd analysis and smart city systems. However, the inherent problems of scale variation and background disturbance restrain their performance. While recent researches focus on studying counting and localization independently, a few works are capable of executing both tasks simultaneously. To this end, we propose a Scale-Context Perceptive Network (SCPNet) to jointly tackle the crowd counting and localization tasks in a unified framework. Specifically, a scale perceptive (SP) module with a local-global branch schema is designed to capture multiscale information. Meanwhile, a context perceptive (CP) module, by the channel-spatial self-attention mechanism, is derived to suppress the background disturbance. Furthermore, a novel hierarchical scale loss function that combines the Euclidean loss function and structural similarity loss function is designed to prompt the proposed model to fulfill the counting and localization simultaneously. Extensive experiments on challenging crowd datasets prove the superiority of the proposed SCPNet compared with the state-of-the-art competitors in both objective and subjective evaluations.

Smart Surveillance: A Review & Survey Through Deep Learning Techniques for Detection & Analysis

Big data applications are consuming most of the space in industry and research area. Among the widespread examples of big data, the role of video streams from CCTV cameras is equally important as other sources like social media data, sensor data, agriculture data, medical data and data evolved from space research. Surveillance videos have a major contribution in unstructured big data. CCTV cameras are implemented in all places where security having much importance. Manual surveillance seems tedious and time consuming. Security can be defending in different terms in different contexts like theft identification, violence detection, chances of explosion etc. In crowded public places the term security covers almost all type of abnormal events. Among them violence detection is difficult to handle since it involves group activity. The anomalous or abnormal activity analysis in a crowd video scene is very difficult due to several real world constraints. The paper includes a deep rooted survey which starts from object recognition, action recognition, crowd analysis and finally violence detection in a crowd environment. Majority of the papers reviewed in this survey are based on deep learning technique. Various deep learning methods are compared in terms of their algorithms and models. The main focus of this survey is application of deep learning techniques in detecting the exact count, involved persons and the happened activity in a large crowd at all climate conditions. Paper discusses the underlying deep learning implementation technology involved in various crowd video analysis methods. Real time processing, an important issue which is yet to be explored more in this field is also considered. Not many methods are there in handling all these issues simultaneously. The issues recognized in existing methods are identified and summarized. Also, future direction is given to reduce the obstacles identified. The survey provides a bibliographic summary of papers from Science-direct, IEEE Xplore and ACM digital library.

Convolutional neural network for human crowd analysis: a review

Convolutional neural network for human crowd analysis: a review

Visual crowd analysis: Open research problems

AbstractOver the last decade, there has been a remarkable surge in interest in automated crowd monitoring within the computer vision community. Modern deep‐learning approaches have made it possible to develop fully automated vision‐based crowd‐monitoring applications. However, despite the magnitude of the issue at hand, the significant technological advancements, and the consistent interest of the research community, there are still numerous challenges that need to be overcome. In this article, we delve into six major areas of visual crowd analysis, emphasizing the key developments in each of these areas. We outline the crucial unresolved issues that must be tackled in future works, in order to ensure that the field of automated crowd monitoring continues to progress and thrive. Several surveys related to this topic have been conducted in the past. Nonetheless, this article thoroughly examines and presents a more intuitive categorization of works, while also depicting the latest breakthroughs within the field, incorporating more recent studies carried out within the last few years in a concise manner. By carefully choosing prominent works with significant contributions in terms of novelty or performance gains, this paper presents a more comprehensive exposition of advancements in the current state‐of‐the‐art.

관광해설 유튜브 댓글 분석을 통한 소비자 구매여정 연구

Purpose This study attempted to explain the decision-making process of YouTube tour interpretation by applying the consumer decision journey(CDJ) to the field of tourism research. In addition, it purpose to empirically understand the impact of YouTube tour interpretation on the tourism destination decision-making process as online tourism information during the pandemic, and make theoretical and empirical suggestions accordingly.
 Methods The impact of YouTube tour interpretation content on CDJ was proposed through literature research, and comments on tour interpretation YouTube contents were used to study the visitor decision-making process of the CDJ model using text mining techniques. Visit Seoul TV, the Seoul Tourism Foundation's official YouTube channel, selected ‘healing tour contents in the room’, and text mining and word cloud analysis conducted.
 Results As a result, comments were spread each stages on the CDJ of visitors and were most distributed in the active evaluation stage, and the length of the comments was longer as the CDJ progressed. In the word crowd analysis, expression of gratitude in the 1st stage, positive evaluation in the 2nd stage, the 3rd stage of visit intention, and the 4th stage of revisit intention were characterized, and the frequency of audit expression was high over stages 1, 2 and 3. Through this study, among the various touch points of CDJ.
 Conclusion This study provided practical implications for the need to find ways to lead to the overall management and active evaluation of potential visitors' CDJ and academic implications for using unstructured data and connecting theoretical models among social media research in the tourism field.

IGROWL: Improved Group Detection With Link Prediction

One of the main challenges robots need to overcome is crowd analysis. Crowd analysis deals with the detection of individuals and interaction groups as well as the recognition of their activities. This paper focuses on the detection of conversational groups, where there have been a number of approaches addressing this problem in both supervised and unsupervised ways. Supervised bottom-up approaches primarily relied on pairwise affinity matrices and were limited to static, third-person views. In this work, we present our approach based on Graph Neural Networks (GNNs) to the problem of interaction group detection, called improved Group Detection With Link Prediction (iGROWL). iGROWL utilises the fact that interaction groups exist in certain inherent spatial configurations and improves its predecessor, GROWL, by introducing an ensemble learning-based sample balancing technique to the algorithm. Our results show that iGROWL outperforms other state-of-the-art methods by 16.7% and 26.4% in terms of F1-score when evaluated on the Salsa Poster Session and Cocktail Party datasets, respectively. Moreover, we show that sample balancing with GNNs is not trivial, but consistent results can be achieved by employing ensemble learning.

Artificial Intelligence Aided Crowd Analytics in Rail Transit Station

Crowd analysis and management is a key area of study for transit agencies seeking to optimize their operations and to facilitate safety risk management activities. Key features of crowd analytics include passenger flow volume, crowd density, and walking speed. This study proposes a generalized artificial intelligence (AI)-based crowd analytics model framework for rail transit stations, by analyzing and visualizing crowd analysis data from video records of high-density crowds. Specifically, we propose a generalized AI-aided methodological framework (AI-Crowd) for calculating flow volume, crowd density, and walking speed. You Only Look Once (YOLO) and Deep SORT are integrated into the model framework to detect and track each individual’s dynamic position. Camera calibration is utilized to transform detected trajectories into a real-world coordinate system. Methods for calculating crowd dynamic metrics are formulated based on the data. To validate the model framework, several video records from a platform scenario at a major rail transit station are used. The model’s pedestrian counting accuracy can reach 95% and the fundamental diagrams of density–speed are shown to be consistent with empirical studies. Further crowd analysis of a stair scenario and a transferring passage scenario using the proposed model framework shows some differentiations in walking behavior. The methodology has further practical applications, such as monitoring social distancing.

Postures anomaly tracking and prediction learning model over crowd data analytics.

Innovative technology and improvements in intelligent machinery, transportation facilities, emergency systems, and educational services define the modern era. It is difficult to comprehend the scenario, do crowd analysis, and observe persons. For e-learning-based multiobject tracking and predication framework for crowd data via multilayer perceptron, this article recommends an organized method that takes e-learning crowd-based type data as input, based on usual and abnormal actions and activities. After that, super pixel and fuzzy c mean, for features extraction, we used fused dense optical flow and gradient patches, and for multiobject tracking, we applied a compressive tracking algorithm and Taylor series predictive tracking approach. The next step is to find the mean, variance, speed, and frame occupancy utilized for trajectory extraction. To reduce data complexity and optimization, we applied T-distributed stochastic neighbor embedding (t-SNE). For predicting normal and abnormal action in e-learning-based crowd data, we used multilayer perceptron (MLP) to classify numerous classes. We used the three-crowd activity University of California San Diego, Department of Pediatrics (USCD-Ped), Shanghai tech, and Indian Institute of Technology Bombay (IITB) corridor datasets for experimental estimation based on human and nonhuman-based videos. We achieve a mean accuracy of 87.00%, USCD-Ped, Shanghai tech for 85.75%, and IITB corridor of 88.00% datasets.

Safe campus: An intelligent campus video surveillance system for crowd analysis

Nigerian campuses comprises of diverse culture, ethnicity and religion. Controlling these campuses is a very big deal. Cameras are installed to help security personnel in carrying out these enormous tasks. However, the installed closed circuit television [CCTV] cameras are only used for evidence sourcing rather than prevention of campus vices. With the use of appropriate technique, campus vices can be prevented thus securing the campus. In this research, an Artificial Intelligent (AI) video surveillance system was developed. The system captures, analyses video for any abnormal behaviour and alerts relevant personnel for appropriate required action. The scheme uses crowd surge a crowd analyzer for videos with vulnerability and threats. The result shows that when deployed at strategic flagged locations early campus vices are detected and reported to relevant personnel whom take appropriate actions and measures to curb escalation of the vices.