Real-time Crowd Research Articles

Estimating crowd density with edge intelligence based on lightweight convolutional neural networks

Crowd stampedes and incidents are critical threats to public security that have caused countless deaths during the past few decades. To avoid crowd stampedes, real-time crowd density estimation can help monitor crowd movements, and thus support a timely evacuation strategy development. In previous studies, scholars and engineers developed multiple video-based crowd density estimation algorithms based on deep neural networks. The excessive computational complexity of deep learning algorithms exacerbated the algorithm’s efficiency, causing unacceptable real-time performance. In the Internet of Things era, deploying the crowd density estimation task with edge computing is an advanced strategy to maintain the real-time performance of the entire system. Considering the limited computational resources on the edge devices, deep learning-based crowd density estimation algorithms normally cannot be handled. To fulfill the deployment on the edge device, the algorithms need to be optimized with a smaller model size. Therefore, this paper proposes a lightweight Convolutional Neural Networks (CNN) based crowd density estimation model by combining the modified MobileNetv2 and the dilated convolution. Public crowd image data sets are used to conduct experiments for evaluating the performance of the proposed algorithm in terms of accuracy and inference speed. The results show that our model achieves much better inference speed accompanied by a slight increase in accuracy. The proposed method of this study can enhance the performance of the crowd monitoring system, and therefore help avoid crowd stampedes and incidents.

A Real-Time Crowd Monitoring and Management System for Social Distance Classification and Healthcare Using Deep Learning.

Coronavirus born COVID-19 disease has spread its roots in the whole world. It is primarily spread by physical contact. As a preventive measure, proper crowd monitoring and management systems are required to be installed in public places to limit sudden outbreaks and impart improved healthcare. The number of new infections can be significantly reduced by adopting social distancing measures earlier. Motivated by this notion, a real-time crowd monitoring and management system for social distance classification is proposed in this research paper. In the proposed system, people are segregated from the background using the YOLO v4 object detection technique, and then the detected people are tracked by bounding boxes using the Deepsort technique. This system significantly helps in COVID-19 prevention by social distance detection and classification in public places using surveillance images and videos captured by the cameras installed in these places. The performance of this system has been assessed using mean average precision (mAP) and frames per second (FPS) metrics. It has also been evaluated by deploying it on Jetson Nano, a low-cost embedded system. The observed results show its suitability for real-time deployment in public places for COVID-19 prevention by social distance monitoring and classification.

Identification of Anomalous Behavioral Patterns in Crowd Scenes

Real time crowd anomaly detection and analyses has become an active and challenging area of research in computer vision since the last decade. The emerging need of crowd management and crowd monitoring for public safety has widen the countless paths of deep learning methodologies and architectures. Although, researchers have developed many sophisticated algorithms but still it is a challenging and tedious task to manage and monitor crowd in real time. The proposed research work focuses on detection of local and global anomaly detection of crowd. Fusion of spatial-temporal features assist in differentiation of feature trained using Mask R-CNN with Resnet101 as a backbone architecture for feature extraction. The data from, BIWI Walking Pedestrian dataset and the Crowds-By-Examples (CBE) dataset and Self-Generated dataset has been used for experimentation. The data deals with different situations like one set of data deals with normal situations like people walking and acting individually, in a group or in a dense crowd. The other set of data contains images four unique anomalies like fight, accident, explosion and people behaving normally. The simulated results show that in terms of precision and recall, our system performs well with Self-Generated dataset. Moreover, our system uses an early stopping mechanism, which allows our system to outperform to make our model efficient. That is why, on 89th epoch our system starts generating finest results.

Real-time crowd counting via lightweight scale-aware network

Accurate crowd counting in congested scenes is still challenging, and its inference speed is also essential in intelligent edge devices. To tackle this issue, we propose a novel lightweight scale-aware network (LSANet) for high-accuracy real-time crowd counting, which consists of four structures: local extractor (LE), scale feature extraction module (SFEM), scale feature fusion module (SFFM), and density map regressor (DMR). Specifically, we devise the SFEM and SFFM carefully to reinforce the learning capability of scale representation, which significantly improves the counting accuracy. Besides, the multi-level intersection-over-union (MIoU) loss function is proposed to reduce background false recognition and estimation errors. Extensive experiments on four mainstream datasets demonstrate that our method achieves the optimal trade-off between effectiveness and efficiency compared with previous lightweight methods.

Real-Time Collaborative Planning with the Crowd

Planning is vital to a wide range of domains, including robotics, military strategy, logistics, itinerary generation and more, that both humans and computers find difficult. Collaborative planning holds the promise of greatly improving performance on these tasks by leveraging the strengths of both humans and automated planners. However, this requires formalizing the problem domain and input, which must be done by hand, a priori, restricting its use in general real-world domains. We propose using a real-time crowd of workers to simultaneously solve the planning problem, formalize the domain, and train an automated system. As plans are developed, the system is able to learn the domain, and contribute larger segments of work.

A survey on deep learning-based real-time crowd anomaly detection for secure distributed video surveillance

A survey on deep learning-based real-time crowd anomaly detection for secure distributed video surveillance

Advances and Trends in Real Time Visual Crowd Analysis.

Real time crowd analysis represents an active area of research within the computer vision community in general and scene analysis in particular. Over the last 10 years, various methods for crowd management in real time scenario have received immense attention due to large scale applications in people counting, public events management, disaster management, safety monitoring an so on. Although many sophisticated algorithms have been developed to address the task; crowd management in real time conditions is still a challenging problem being completely solved, particularly in wild and unconstrained conditions. In the proposed paper, we present a detailed review of crowd analysis and management, focusing on state-of-the-art methods for both controlled and unconstrained conditions. The paper illustrates both the advantages and disadvantages of state-of-the-art methods. The methods presented comprise the seminal research works on crowd management, and monitoring and then culminating state-of-the-art methods of the newly introduced deep learning methods. Comparison of the previous methods is presented, with a detailed discussion of the direction for future research work. We believe this review article will contribute to various application domains and will also augment the knowledge of the crowd analysis within the research community.

Real Time Crowd Navigation from First Principles of Probability Theory

Constructing realistic and real time human-robot interaction models is a core challenge in crowd navigation. In this paper we derive a robot-agent interaction density from first principles of probability theory; we call our approach “first order interacting Gaussian processes” (foIGP). Furthermore, we compute locally optimal solutions—with respect to multi-faceted agent “intent” and “flexibility”—in near real time on a laptop CPU. We test on challenging scenarios from the ETH crowd dataset and show that the safety and efficiency statistics of foIGP is competitive with human safety and efficiency statistics. Further, we compute the safety and efficiency statistics of dynamic window avoidance, a physics based model variant of foIGP, a Monte Carlo inference based approach, and the best performing deep reinforcement learning algorithm; foIGP outperforms all of them.

MobileCount: An efficient encoder-decoder framework for real-time crowd counting

In this work, we propose a computation-efficient encoder-decoder architecture, named MobileCount, which is specifically designed for high-accuracy real-time crowd counting on mobile or embedded devices with limited computation resources. For the encoder part, MobileNetV2 is tailored in order to significantly reduce FLOPs at a little cost of performance drop, which has 4 bottleneck blocks preceded by a max pooling layer of stride 2. The design of decoder is motivated by Light-weight RefineNet, which further boosts counting performance with only a 10% increase of FLOPs. In comparison with state-of-the-arts, our proposed network is able to achieve comparable counting performance with 1/10 FLOPs on a number of benchmarks. At last, we propose a multi-layer knowledge distillation method to further boost the performance of MobileCount without increasing its FLOPs.

Near Real-time Crowd Counting using Deep Learning Approach

Abstract In the current digital era, at many places crowd counting mechanisms still rely on old-fashioned methods such as maintaining registers, making use of people counters and sensors based counting at entrance. These methods fail in the places where the movement of people is completely random, highly variable and dynamic. These methods are time consuming and tedious. The proposed system is developed for situations where emergency evacuations are required such as fire outbreaks, calamitous events, etc. and making informed decisions on the basis of the number of people such as food, water, detecting congestion, etc. A deep convolution neural network (DCNN) based system can be used for near real-time crowd counting. The system uses NVIDIA GPU processor to exploit the parallel computing framework to achieve swift and agile processing of the video feed taken through a camera. This work contributes towards constructing a model to detect heads captured by CCTV camera. The model is trained extensively by providing several scenarios such as overlapping heads, partial visibility of heads etc. This system provides significant accuracy in estimating the head count in dense population in reasonably less amount of time.

A Privacy-Aware Crowd Management System for Smart Cities and Smart Buildings

Cities are growing at a dizzying pace and they require improved methods to manage crowded areas. Crowd management stands for the decisions and actions taken to supervise and control densely populated spaces and it involves multiple challenges, from recognition and assessment to application of actions tailored to the current situation. To that end, Wi-Fi-based monitoring systems have emerged as a cost-effective solution for the former one. The key challenge that they impose is the requirement to handle large datasets and provide results in near real-time basis. However, traditional big data and event processing approaches have important shortcomings while dealing with crowd management information. In this paper, we describe a novel system architecture for real-time crowd recognition for smart cities and smart buildings that can be easily replicated. The described system proposes a privacy-aware platform that enables the application of artificial intelligence mechanisms to assess crowds' behavior in buildings employing sensed Wi-Fi traces. Furthermore, the present paper shows the implementation of the system in two buildings, an airport and a market, as well as the results of applying a set of classification algorithms to provide crowd management information.

CrowdEst: a method for estimating (and not simulating) crowd evacuation parameters in generic environments

Evacuation plans have been historically used as a safety measure for the construction of buildings. The existing crowd simulators require fully-modeled 3D environments and enough time to prepare and simulate scenarios, where the distribution and behavior of the crowd needs to be controlled. In addition, its population, routes or even doors and passages may change, so the 3D model and configurations have to be updated accordingly. This is a time-consuming task that commonly has to be addressed within the crowd simulators. With that in mind, we present a novel approach to estimate the resulting data of a given evacuation scenario without actually simulating it. For such, we divide the environment into smaller modular rooms with different configurations, in a divide-and-conquer fashion. Next, we train an artificial neural network to estimate all required data regarding the evacuation of a single room. After collecting the estimated data from each room, we develop a heuristic capable of aggregating per-room information so the full environment can be properly evaluated. Our method presents an average error of 5% when compared to evacuation time in a real-life environment. Our crowd estimator approach has several advantages, such as not requiring to model the 3D environment, nor learning how to use and configure a crowd simulator, which means any user can easily use it. Furthermore, the computational time to estimate evacuation data (inference time) is virtually zero, which is much better even when compared to the best-case scenario in a real-time crowd simulator.

Determination of Bus Crowding Coefficient Based on Passenger Flow Forecasting

To improve bus passengers’ degree of comfort, it is necessary to determine the real-time crowd coefficient in the bus. With this concern, this paper employed the RBF Neural Networks approach to predict the number of passengers in the bus based on historical data. To minimize the impact of the randomness of passenger flow on the determination of bus crowd coefficient, a cloud model-based bus crowd coefficient identification method was proposed. This paper first selected the performance measurements for determining bus crowd coefficient and calculated the digital characteristics of the cloud model based on the boundary values of the selected performance measures under six Levels-of-Service (LOSs). Then the subclouds obtained under the six LOSs were synthesized into a standard cloud. According to the predicted number of passengers in the bus, the passenger density and loading frequency were calculated, which were imported into the cloud generator to set up the bus crowd coefficient identification model. By calculating the crowd degrees of identification cloud and template cloud at each site, this paper determined the crowed coefficient of each bus station. Finally, this paper took the bus line No. 10 in Dalian city as case study to verify the proposed model. It was found that the crowd coefficients of the selected route ranged from 60.265 to 109.825, and the corresponding LOSs ranged between C and F. The method of discriminating bus crowding coefficient can not only effectively determine the congestion coefficient, but also effectively avoid the fuzziness and randomness of the crowding coefficient judgment in the bus, which has strong theoretical and practical significance.

Real-Time Large Crowd Rendering with Efficient Character and Instance Management on GPU

Achieving the efficient rendering of a large animated crowd with realistic visual appearance is a challenging task when players interact with a complex game scene. We present a real-time crowd rendering system that efficiently manages multiple types of character data on the GPU and integrates seamlessly with level-of-detail and visibility culling techniques. The character data, including vertices, triangles, vertex normals, texture coordinates, skeletons, and skinning weights, are stored as either buffer objects or textures in accordance with their access requirements at the rendering stage. Our system preserves the view-dependent visual appearance of individual character instances in the crowd and is executed with a fine-grained parallelization scheme. We compare our approach with the existing crowd rendering techniques. The experimental results show that our approach achieves better rendering performance and visual quality. Our approach is able to render a large crowd composed of tens of thousands of animated instances in real time by managing each type of character data in a single buffer object.

The impact of position errors on crowd simulation

In large crowd events, there is always a potential possibility that a stampede accident will occur. The accident may cause injuries or even death. Approaches for controlling crowd flows and predicting dangerous congestion spots would be a boon to on-site authorities to manage the crowd and to prevent fatal accidents. One of the most popular approaches is real-time crowd simulation based on position data from personal Global Positioning System (GPS) devices. However, the accuracy of spatial data varies for different GPS devices, and it is also affected by an environment in which an event takes place. In this paper, we would like to assess the effect of position errors on stampede prediction. We propose an Automatic Real-time dEtection of Stampedes (ARES) method to predict stampedes for large events. We implement three different stampede assessment methods in Menge framework and incorporate position errors. Our analysis suggests that the probability of simulated stampede changes significantly with the increase of the magnitude of position errors, which cannot be eliminated entirely with the help of classic techniques, such as the Kalman filter. Thus, it is our position that novel stampede assessment methods should be developed, focusing on the detection of position noise and the elimination of its effect.

Perceptual evaluation of maneuvering motion illusion for virtual pedestrians

Crowd simulations span a wide spectrum of application domains, most notably video games, evacuation scenarios, and the movie industry. However, it is not obligatory that all virtual populace applications have the primary objective of realistic simulation. In most instances, it is necessary and sufficient that viewers perceive the crowd as plausible. Even for a crowd consisting of agents navigating on linear trajectories without any maneuvers, visual motion illusion elicited by these trajectories might appear to be a natural consequence, causing them to be perceived as wriggling rather than straight. In this respect, we evaluate in this study whether simulated 3D human agents walking with constant, collision-free velocities, induce such a maneuvering motion illusion, aiming toward an efficient real-time crowd simulation. For this purpose, we recorded videos of virtual human crowds with different parameter combinations, such as the agent walking speed, crowd density, camera tilt angle, and camera distance. These videos were watched by human subjects who were instructed to mark the virtual agents who they thought had changed their gait directions. The analyzed results revealed that participants claimed the presence of maneuvering virtual agents in the videos, even though there were none in any of them. Spatial grouping of the markings highlighted that the participants mainly focused on the central area of the simulation environment, and spatiotemporal analysis of the click data also showed stronger evidence to such an illusion (see accompanying video). Furthermore, we found that all of the referred parameters have statistically significant main effects on the number of marked agents per watched video.

Modeling crowd dynamics through coarse-grained data analysis.

Understanding and predicting the collective behaviour of crowds is essential to improve the efficiency of pedestrian flows in urban areas and minimize the risks of accidents at mass events. We advocate for the development of crowd traffic management systems, whereby observations of crowds can be coupled to fast and reliable models to produce rapid predictions of the crowd movement and eventually help crowd managers choose between tailored optimization strategies. Here, we propose a Bi-directional Macroscopic (BM) model as the core of such a system. Its key input is the fundamental diagram for bi-directional flows, i.e. the relation between the pedestrian fluxes and densities. We design and run a laboratory experiments involving a total of 119 participants walking in opposite directions in a circular corridor and show that the model is able to accurately capture the experimental data in a typical crowd forecasting situation. Finally, we propose a simple segregation strategy for enhancing the traffic efficiency, and use the BM model to determine the conditions under which this strategy would be beneficial. The BM model, therefore, could serve as a building block to develop on the fly prediction of crowd movements and help deploying real-time crowd optimization strategies.

A Feedback Control-Based Crowd Dynamics Management in IoT System

The development of technologies related to the Internet of Things (IoT) provides a new perspective on applications pertaining to smart cities. Smart city applications focus on resolving issues facing people in everyday life, and have attracted a considerable amount of research interest. The typical issue encountered in such places of daily use, such as stations, shopping malls, and stadiums is crowd dynamics management. Therefore, we focus on crowd dynamics management to resolve the problem of congestion using IoT technologies. Real-time crowd dynamics management can be achieved by gathering information relating to congestion and propose less crowded places. Although many crowd dynamics management applications have been proposed in various scenarios and many models have been devised to this end, a general model for evaluating the control effectiveness of crowd dynamics management has not yet been developed in IoT research. Therefore, in this paper, we propose a model to evaluate the performance of crowd dynamics management applications. In other words, the objective of this paper is to present the proof-of-concept of control effectiveness of crowd dynamics management. Our model uses feedback control theory, and enables an integrated evaluation of the control effectiveness of crowd dynamics management methods under various scenarios. We also provide extensive numerical results to verify the effectiveness of the model.

Cioppino: Multi-Tenant Crowd Management

Embedding human computation in systems for data analysis improves the quality of the analysis, but can significantly impact the end-to-end cost and performance of the system. Recent work in crowdsourcing systems attempts to optimize for performance, but focuses on single applications running homogeneous tasks. In this work, we introduce Cioppino, a system that accounts for human factors that can affect performance when running multiple applications in parallel. Cioppino uses a queueing model to represent the pool of workers, and leverages techniques for autoscaling used in cloud computing to adaptively adjust the pool size. Its model also accounts for worker abandonment, and automatically shifts workers between applications to improve performance and match workers with tasks they enjoy most. Our evaluation of Cioppino in simulation on traces extracted from a realtime crowd system running on Amazon’s Mechanical Turk demonstrates a 19X reduction in cost, a 20% increase in throughput, and a 2X increase in worker preference for assigned tasks as compared to state-of-the-art crowd management strategies.

Engineering social media driven intelligent systems through crowdsourcing

PurposeThe purpose of this paper is to explore implicit crowdsourcing, leveraging social media in real-time scenarios for intelligent systems.Design/methodology/approachA case study using an illustrative example system, which systematically used a custom social media platform for automated financial news analysis and summarisation was developed, evaluated and discussed. Literature review related to crowdsourcing and collective intelligence in intelligent systems was also conducted to provide context and to further explore the case study.FindingsIt was shown how, and that useful intelligent systems can be constructed from appropriately engineered custom social media platforms which are integrated with intelligent automated processes. A recent inter-rater agreement measure for evaluating quality of implicit crowd contributions was also explored and found to be of value.Practical implicationsThis paper argues that when social media platforms are closely integrated with other automated processes into a single system, this may provide a highly worthwhile online and real-time approach to intelligent systems through implicit crowdsourcing. Key practical issues, such as achieving high-quality crowd contributions, challenges of efficient workflows and real-time crowd integration into intelligent systems, were discussed. Important ethical and related considerations were also covered.Originality/valueA contribution to existing theory was made by proposing how social media Web platforms may benefit crowdsourcing. As opposed to traditional crowdsourcing platforms, the presented approach and example system has a set of social elements that encourages implicit crowdsourcing. Instances of crowdsourcing with existing social media, such as Twitter, often also called crowd piggybacking, have been used in the past; however, using an entirely custom-built social media system for implicit crowdsourcing is relatively novel and has several advantages. Some of the discussion in context of intelligent systems construction are novel and contribute to the existing body of literature in this field.