Crowd Surveillance Research Articles

Comprehensive examination of regional railway passenger behavior and dwell time components: Insights from video-based observations in Victoria, Australia

In understudied regional railway operations, this study explores passenger boarding and alighting patterns and how station design impacts them, particularly dwell times. Despite extensive metropolitan and suburban train research, regional railways have been overlooked. This study investigates regional rail passenger flow and dwell time to bridge this gap. This article studies dwell times and passenger boarding and alighting at two regional stations in Victoria, Australia, using CCTV data. The objective is to identify insights that might improve regional railway services' efficiency and user experience and advocate for sector-specific solutions. Analysis indicates distinct station boarding and alighting features, highlighting the discovery of the ‘blinded phenomenon’ for train conductors particularly in the afternoon peak (PMP). The results of the study showed that PMP services, which prioritise alighting passengers, had higher dwell times than the morning peak (AMP) services, which emphasise boarding passengers. Obstructed views make it difficult for train conductors to monitor passenger alighting, prolonging dwell times. Better human resource strategies, artificial intelligence for crowd surveillance, and strategic CCTV system deployment to streamline operations and improve passenger experience on regional railways are proposed in the paper, laying the groundwork for future research and operational changes in this vital transportation sector.

Crowd Scene Anomaly Detection in Online Videos

Abstract. The prevalence of surveillance cameras in public places has led to an extremely pressing need for effective position and crowd monitoring, as well as anomaly detection. This paper tends to exhibit an incorporated approach that combines state-of-the-art computer vision techniques for comprehensive crowd surveillance. The main features of our approach are summarized into four steps: (a) Object detection and tracking; (b) Geometric rectification for positioning; (c) Motion extraction; and (d) Anomaly detection. First, this uses YOLOv5's Convolutional Neural Network (CNN) model in making efficient detection of objects, focusing on spotting individuals within crowded scenes. After detection, a strong mechanism for tracking is established with the help of the DeepSORT algorithm, which can track the person across frames. It must gain the people's position in the video frame and analyze motion data with the guarantee of capture of camera-scene geometry. Each frame thus gets converted from the 3D perspective to a 2D bird's eye view within the surveillance video, giving a guarantee of capture of the geometry of a camera scene. Motion anomaly detection is addressed through statistical methods, with Kernel Density Estimation (KDE) being employed to identify deviations from normal motion patterns. Extensive experiments conducted on different online crowd scene video datasets validate the effectiveness of the proposed anomaly detection mechanism. Overall, this integrated approach proposes a promising solution to crowd surveillance, further development of object detection, tracking, and anomaly analysis for monitoring public spaces.

Revolutionizing crowd surveillance through voice-driven face recognition empowering rapid identification: towards development of sustainable smart cities

Recent global efforts to create sustainable smart cities have significantly transformed society and improved the lives of people. Nowadays, crowd surveillance (CS) has become essential in sustainable smart cities and society to protect public safety and security. In this regard, the face-based human detection system has received considerable attention because it is recognized as an emerging method in crowd surveillance applications. Thus, in this work, a new method for real-time identification of people for a crowd surveillance system (CSS) that uses facial and speech recognition technology has been introduced. In traditional CS systems, human operators are frequently used by crowd surveillance systems to watch and evaluate video feeds. Human error and operator weariness may result in lost opportunities or slow replies, which reduce the system’s efficacy. Certain procedures, including the initial identification and monitoring of people in video feeds, can be automated using a voice-activated system. To address the issues with the present CSS, a new framework Voice-Activated Face Recognition (VAFR) is proposed in this work. The proposed framework combines the speech and face recognition models for crowd surveillance. Experimental and simulation studies have been performed to analyze the performance of the proposed VAFR framework. The proposed framework uses the Viola-Jones algorithm for face identification and the Conformer architecture for speech analysis, reaching a noteworthy 99.8% accuracy rate in live video feeds. In addition, the ethical and safety aspect of the proposed VAFR system is presented.

Drone path planning and dynamic arena-target allocation for crowd surveillance

The paper addresses the problem of surveillance of dynamic crowds in a bounded arena using multiple drones. The strategy proposes a prediction-based path planning approach for drones, where the target’s future position is predicted and used to generate a traverse path. The procedure for grouping numerous human beings in order to deal with a dense crowd scenario is also described. An algorithm to dynamically divide and allocate the arena to drones is also proposed to ensure equal workload distribution among all drones in the framework. The paper also discusses relevant concepts like detecting anomalous crowd activity and searching for any missing target in the arena. The proposed algorithms are validated on MATLAB environment numerically and ROS/Gazebo physics engine as a part of simulation-in-the-loop (SITL).

Anomaly Detection in Crowd Surveillance using Edge Computing: A Comprehensive Survey

Abstract: As urban environments continue to witness the proliferation of surveillance systems, ensuring public safety and security has become an increasingly challenging endeavor. Anomaly detection in crowded areas has emerged as a crucial task in these settings, aimed at identifying suspicious activities or potential threats. To meet the demands of real-time monitoring and analysis, edge computing has gained prominence as a critical technology. This survey paper provides a comprehensive overview of the state-of-the-art in anomaly detection, its use in crowd surviellance and the role played by edge computing in anomaly detection .The paper reviews an extensive body of literature encompassing various techniques and methodologies employed for anomaly detection in crowded scenes. It explores the evolution of traditional video-based approaches, such as motion analysis and object tracking, and the recent advancements leveraging deep learning, including various models in machine learning. These techniques are examined for their applicability in crowd surveillance scenarios.Various commonly used datasets to measure the quality of anomaly detection are also explored, along with their attributes and descriptions.The survey analyzes how edge computing solutions, such as edge AI accelerators and edge devices, enable faster and more context-aware processing of video data, while also addressing issues related to bandwidth constraints, privacy concerns, and scalability; paving way for a further research in harnessing the power of edge computing in crowd surveillance anomaly detection.

Digitally improving UK police surveillance and incidence response using real-time crowd reporting app: Digipolice

Overtime, evolving technologies has made ways of monitoring and controlling crime and action in the United Kingdom (UK) become more challenging which needs optimization with the help of technologies. Also with the rate of crime across cities in the UK like mass shootings, murder and gang gun battles, the UK Police system within localities are overwhelmed or may not efficiently cope with crime alerts and incidences coming in at increasing frequencies. However looking in the area of real time crowd reporting, there is need to leverage technology to enhance public safety, improve incident response, and foster community engagement. This necessitated the aim of this study to develop a digital policing surveillance app for crowd reporting in the UK. By utilizing digital policing surveillance apps, the aim is to improve crowd management capabilities by enabling real-time reporting of incidents and observations from both the public and law enforcement personnel. This was positioned to facilitate better situational awareness, efficient resource allocation, and effective response strategies. To achieve the objectives, Python Programming Language 3.8 were deployed together with a Django Framework and a Microsoft Visual Studio Build Tools. In the DP system, the Model-View-Controller (MVC) pattern, which divides an application into three interconnected components—the model, the view, and the controller—was used for the architectural design of the system. For the database, Django's ORM makes it simple to support many databases, and in this project straightforward SQLite3 database were utilised which is ideal for small-footprint applications. Using WebSockets to establish real-time, bidirectional communication between web clients and servers, the user interface and intelligence board for the system were developed. A clean designed frontend design system with Bootstrap version 5—an open-source, reusable design framework created by Twitter—was selected for its easy user interface. The system was put through 50 concurrent load tests with actual users swarming the app since the DP app was subjected to hundreds of people complaining in real time. 2. The host computer had a Core i5 vPro CPU running at 4GHz and 4GB of RAM. The result shows that 50 users successfully generated an average of 29.7 requests per second. The system scaled up nicely to 50 users and reached its highest response time of 2600 milliseconds (2.6 seconds) in total. A dramatic rise in activity and a longer response time occurred after there were more than 50 users. The system could not operate at its peak efficiency with 100 users because response times were inconsistent and exceeded 7000 milliseconds (7 seconds) for each request. The effectiveness of the online deployment was evaluated with respectable results, and the system requirements also underwent successful testing. The appropriate investigation of ethical issues was done to meet requirements. DigiPolice has the potential to develop into a cutting-edge programme for real-time crowd surveillance and incident reporting. UK Police can employ it to as advancement to its “My Police” with as fast-track real-time and collaborative crowd reporting enhancements.

Deep learning and multi-modal fusion for real-time multi-object tracking: Algorithms, challenges, datasets, and comparative study

Real-time multi-object tracking (MOT) is a complex task involving detecting and tracking multiple objects. After the objects are detected, they are assigned markers, and their trajectories are tracked in real-time. The scientific community is intrigued by the possibilities of utilizing MOT technology in the context of smart cities. Their primary focus lies in the domains of intelligent transportation, detection of vehicles and pedestrians, crowd surveillance, and public safety. Deep learning techniques have been developed in recent years to effectively tackle the challenges of real-time MOT tasks and enhance tracking performance. Environmental perception within smart traffic applications relies heavily on sensor data fusion. In traffic scenarios, a thoughtful approach involves utilizing a combination of sensors and cameras to detect and track targets while gathering valuable data effectively. However, it faces challenges when it comes to detecting and tracking objects that are in motion, have complex changes in appearance, or are in crowded scenes. This paper explores the foundational standard for real-time Multiple Object Tracking tasks. We prioritize the examination of quantitative measures by conducting a comprehensive analysis of widely utilized benchmark datasets and metrics. This study also investigates established embedding techniques and multi-modal fusion methods within real-time multi-target tracking algorithms. Each strategy will be classified and assessed according to a predefined set of principles. The paper presents a comprehensive analysis and visual representation of various MOT strategies. Finally, this paper aims to present an overview of the current challenges faced by the MOT mission, as well as the potential objectives that lie ahead.

Frequency-guidance Collaborative Triple-branch Network for single image dehazing

Image dehazing is a crucial pre-processing step in many high-level vision applications, e.g., crowd surveillance, automatic driving, and remote sensing. Recently, learning-based methods have achieved promising performance by designing various convolutional neural networks (CNNs). However, most existing CNNs are based on a single feature branch and cannot fully utilize the low-frequency and high-frequency features of haze, which inevitably affects the quality of the restored image. To this end, we develop an effective Frequency-guidance Collaborative Triple-branch Network (FCT-Net) for image dehazing. It casts dehazing as a dual-frequency component restoration task and deals with pertinent features based on a triple-branch-like architecture. Specifically, our FCT-Net can be decomposed into four components: low-frequency branch (LFB), high-frequency branch (HFB), progressive fusion branch (PFB), and image reconstruction tail (IRT). The LFB aims to mine more contextual information with dual-path feature modulation block (DFMB), while the HFB aims to recover the missing structural detail via mixed attention residual block (MARB). PFB is a fusion branch built between LFB and HFB, where different feature streams (low- and high-frequency information) can be integrated together through continuous selective progressive fusion (SPF) modules. Finally, our IRT exploits all these derived features to generate the clear haze-free image. Furthermore, we utilize a multi-strategy loss to optimize our FCT-Net from the perspectives of pixel domain and feature domain, which can achieve better image recovery quality. Extensive experiments on different benchmark datasets show that our FCT-Net is able to get comparable results with other state-of-the-art methods.

A multi-channel soft biometrics framework for seamless border crossings

As the number of passengers at border entry points such as airports and rail stations increases, so does the demand for seamless, secure, and fast biometric technologies for verification purposes. Although fingerprints are currently useful biometric technologies, they are intrusive and slow down the end-to-end verification process, increasing the chances of tampering. Emerging as an alternative technology, soft biometrics have proven successful for non-intrusive and rapid verification. Soft biometrics consists of a large set of features from three different modalities of the human body, including the face, body, and essential & auxiliary attachments. This paper proposes a multi-channel soft biometrics framework that leverages soft biometrics technology over traditional biometrics. The framework encapsulates four distinct components: ApparelNet, which verifies essential and auxiliary attachments; A-Net, which measures anthropometric soft biometrics; OneDetect, which predicts global soft biometrics; and RSFS, which develops a set of highly relevant and supportive soft biometrics for verification. The proposed framework addresses several critical limitations of existing biometrics technologies during the verification process at border entry points, such as intrusive behavior, response time, biometric tampering, and privacy issues. The proposed multi-channel soft biometrics framework has been evaluated using several benchmark datasets in the field, such as Front-view Gait (FVG), Pedestrian Attribute Recognition At Far Distance (PETA), and Multimedia and Vision (MMV) Pedestrian. Using heterogeneous datasets enables the testing of each framework component or channel against numerous constrained and unconstrained scenarios. The outcome of the envisioned multi-channel soft biometrics framework is presented based on distinct outcomes from each channel, but it remains focused on determining a single cumulative verification score for verification at border control. In addition, this multi-channel soft biometrics framework has extended applications in several fields, including crowd surveillance, the fashion industry, and e-learning.

Autonomous pedestrian detection for crowd surveillance using deep learning framework

Autonomous pedestrian detection for crowd surveillance using deep learning framework

Abnormal Crowd Detection in Public Places using OpenCV and Deep Learning

Abstract: The rapid development of object detection algorithm as led to its widespread application in security, such as facial recognition and crowd surveillance. However, real-time tracking of an individual is very challenging, especially in crowded places where the person might be in part or entirely occluded for some period. Hence, this paper objective is to create abnormal activity detection in public places focusing only on people. This system does not just detect a person in real-time but in addition, uses the information it as learned to track the trajectory of the person until they exit the frame of the camera. The system uses the algorithm called YOLO for the person detection. The system was able to successfully detect the abnormal activity in public places and detect crowded group of people in live

Artificial intelligence for Novel Corona Virus (COVID-19) Pandemic

COVID-19 outbreak has created havoc around the world and has brought life to a disturbing halt claiming thousands of lives worldwide and infected cases rising every day. With technological advancements in Artificial Intelligence (AI), AI-based platforms can be used to deal with COVID-19 pandemic and accelerate the processes ranging from crowd surveillance to medical diagnosis. This paper renders a response to battle the virus through various AI techniques by making use of its subsets such as Machine Learning (ML), Deep learning (DL) and Natural Language Processing (NLP). A survey of promising AI methods which could be used in various applications to facilitate the processes in this pandemic along potential of AI and challenges imposed are discussed thoroughly. This paper relies on the findings of the most recent research publications and journals on COVID-19 and suggests numerous relevant strategies. A case study on the impact of COVID-19 in various economic sectors is also discussed. The potential research challenges and future directions are also presented in the paper.

Joint topology control and routing in a UAV swarm for crowd surveillance

Aerial surveillance using unmanned aerial vehicles (UAVs) provides an on-demand and cost-effective solution to smart-city monitoring needs, owing to their three-dimensional positioning adjustment and autonomy. The optimal deployment of a UAV swarm, also known as a flying ad hoc network (FANET), to achieve on-demand coverage of mobile ground targets (MGTs) is challenging in terms of controlling UAV mobility to maximize coverage while maintaining quality of service. Data routing from UAVs to a base station (BS) without awareness of the updated topology causes link breakages, excessive retransmissions, high congestion, and energy holes. Therefore, we propose a joint topology control and routing (JTCR) protocol comprising three modules to perform crowd surveillance. The first JTCR module provides virtual force-based mobility control (VFMC), which controls the mobility of UAVs to track MGTs, ensuring stable bi-connectivity. The second module provides energy-efficient mobility-aware fuzzy clustering that clusters the FANET to aggregate the sensed data to each cluster head (CH) by utilizing the UAV mobility provided by the VFMC. The third module provides topology-aware Q-routing, which routes the aggregated data from CH UAVs to the BS by selecting an optimal path in terms of delay, path stability, and energy consumption. According to our performance study, the proposed JTCR outperforms existing routing protocols in terms of tracking-coverage rate, connectivity rate, the number of retransmissions, packet delivery ratio, end-to-end delay, and energy consumption. This is mainly enabled by the realistic mobility control of the UAV swarm at the reasonable cost of control overhead.

A comparative study on surveillance and privacy regulations (the UAE vs. the USA and the EU)

Surveillance is becoming the norm in today’s life, especially with the pandemic of COVID-19. Surveillance of public crowds and activity is a controversial issue that can contradict the privacy of individuals (Federal Decree-Law No. (5) of 2012). This paper presents a comparative study of surveillance and privacy regulations and law in the UAE compared to the USA and the EU. The objective of this comparison is to highlight the amendments that have been adopted to improve laws and regulations, the need for further improvement, and the strengths and weaknesses in each of these countries. A discussion of different acts adopted in these countries and comparing them can help security experts to cooperate with legislators in order to rectify shortcomings and improve the acts adopted in their respective countries. Furthermore, we think that such a comparison can help system developers to find an easier way to accommodate the differences in security measures that they have to tackle and incorporate when they are serving customers in these countries and especially in the UAE. A legal framework has been proposed in order to define the maturity level of regulations adopted by a government in regard to surveillance and privacy laws and acts.

Estimating Interpersonal Distance and Crowd Density with a Single-Edge Camera

For public safety and physical security, currently more than a billion closed-circuit television (CCTV) cameras are in use around the world. Proliferation of artificial intelligence (AI) and machine/deep learning (M/DL) technologies have gained significant applications including crowd surveillance. The state-of-the-art distance and area estimation algorithms either need multiple cameras or a reference object as a ground truth. It is an open question to obtain an estimation using a single camera without a scale reference. In this paper, we propose a novel solution called E-SEC, which estimates interpersonal distance between a pair of dynamic human objects, area occupied by a dynamic crowd, and density using a single edge camera. The E-SEC framework comprises edge CCTV cameras responsible for capturing a crowd on video frames leveraging a customized YOLOv3 model for human detection. E-SEC contributes an interpersonal distance estimation algorithm vital for monitoring the social distancing of a crowd, and an area estimation algorithm for dynamically determining an area occupied by a crowd with changing size and position. A unified output module generates the crowd size, interpersonal distances, social distancing violations, area, and density per every frame. Experimental results validate the accuracy and efficiency of E-SEC with a range of different video datasets.

Pedestrian trajectory prediction with convolutional neural networks

Predicting the future trajectories of pedestrians is a challenging problem that has a range of application, from crowd surveillance to autonomous driving. In literature, methods to approach pedestrian trajectory prediction have evolved, transitioning from physics-based models to data-driven models based on recurrent neural networks. In this work, we propose a new approach to pedestrian trajectory prediction, with the introduction of a novel 2D convolutional model. This new model outperforms recurrent models, and it achieves state-of-the-art results on the ETH and TrajNet datasets. We also present an effective system to represent pedestrian positions and powerful data augmentation techniques, such as the addition of Gaussian noise and the use of random rotations, which can be applied to any model. As an additional exploratory analysis, we present experimental results on the inclusion of occupancy methods to model social information, which empirically show that these methods are ineffective in capturing social interaction.

Estimation of Crowd Density from UAV Images based on Deep Learning

Crowd monitoring is necessary to improve safety and controllable movements to minimize risk, especially in high crowded events, such as Kumbh Mela, political rallies, sports event etc. In this current digital age mostly crowd monitoring still relies on outdated methods such as keeping records, using people counters manually, and using sensors to count people at the entrance. These approaches are futile in situations where people's movements are completely unpredictable, highly variable, and complex. Crowd surveillance using unmanned aerial vehicles (UAVs), can help us solve these problems. The proposed paper uses a UAV on which an IP Camera will be attached to get media, we then use a convolutional neural network to learn a regression model for crowd counting, the model will be trained extensively by using three widely used crowd counting datasets, ShanghaiTech part A and part B, UCF-CC 50 and UCF-QNRF.

A Wide Area Multiview Static Crowd Estimation System Using UAV and 3D Training Simulator

Crowd size estimation is a challenging problem, especially when the crowd is spread over a significant geographical area. It has applications in monitoring of rallies and demonstrations and in calculating the assistance requirements in humanitarian disasters. Therefore, accomplishing a crowd surveillance system for large crowds constitutes a significant issue. UAV-based techniques are an appealing choice for crowd estimation over a large region, but they present a variety of interesting challenges, such as integrating per-frame estimates through a video without counting individuals twice. Large quantities of annotated training data are required to design, train, and test such a system. In this paper, we have first reviewed several crowd estimation techniques, existing crowd simulators and data sets available for crowd analysis. Later, we have described a simulation system to provide such data, avoiding the need for tedious and error-prone manual annotation. Then, we have evaluated synthetic video from the simulator using various existing single-frame crowd estimation techniques. Our findings show that the simulated data can be used to train and test crowd estimation, thereby providing a suitable platform to develop such techniques. We also propose an automated UAV-based 3D crowd estimation system that can be used for approximately static or slow-moving crowds, such as public events, political rallies, and natural or man-made disasters. We evaluate the results by applying our new framework to a variety of scenarios with varying crowd sizes. The proposed system gives promising results using widely accepted metrics including MAE, RMSE, Precision, Recall, and F1 score to validate the results.

Artificial Intelligence for the Novel Corona Virus (COVID-19) Pandemic

COVID-19 outbreak has created havoc around the world and has brought life to a disturbing halt claiming thousands of lives worldwide and infected cases rising every day. With technological advancements in Artificial Intelligence (AI), AI-based platforms can be used to deal with COVID-19 pandemic and accelerate the processes ranging from crowd surveillance to medical diagnosis. This paper renders a response to battle the virus through various AI techniques by making use of its subsets such as Machine Learning (ML), Deep learning (DL) and Natural Language Processing (NLP). A survey of promising AI methods which could be used in various applications to facilitate the processes in this pandemic along potential of AI and challenges imposed are discussed thoroughly. This paper relies on the findings of the most recent research publications and journals on COVID-19 and suggests numerous relevant strategies. A case study on the impact of COVID-19 in various economic sectors is also discussed. The potential research challenges and future directions are also presented in the paper.

IoT-based crowd monitoring system: Using SSD with transfer learning

The constantly developing urbanization and the emergence of smart cities require better security surveillance and crowd monitoring systems. The growing availability of the Internet of Things (IoT) devices in public and private organizations also provide intelligent and secure surveillance solutions for real-time monitoring in public spaces. This article introduces an IoT-based crowd surveillance system that uses a deep learning model to detect and count people using an overhead view perspective. The Single Shot Multibox Detector (SSD) model with Mobilenetv2 as the basic network is used for the detection of people. The detection model’s accuracy is enhanced with a transfer learning approach. Two virtual lines are defined to count how many people are leaving and entering the scene. In order to assess performance, experiments are performed using different video clips. Results indicate that transfer learning increases the overall detection performance of the system with an accuracy of 95%.