Surveillance Scenarios Research Articles

MIMO-Uformer: A Transformer-Based Image Deblurring Network for Vehicle Surveillance Scenarios

Motion blur is a common problem in the field of surveillance scenarios, and it obstructs the acquisition of valuable information. Thanks to the success of deep learning, a sequence of CNN-based architecture has been designed for image deblurring and has made great progress. As another type of neural network, transformers have exhibited powerful deep representation learning and impressive performance based on high-level vision tasks. Transformer-based networks leverage self-attention to capture the long-range dependencies in the data, yet the computational complexity is quadratic to the spatial resolution, which makes transformers infeasible for the restoration of high-resolution images. In this article, we propose an efficient transformer-based deblurring network, named MIMO-Uformer, for vehicle-surveillance scenarios. The distinct feature of the MIMO-Uformer is that the basic-window-based multi-head self-attention (W-MSA) of the Swin transformer is employed to reduce the computational complexity and then incorporated into a multi-input and multi-output U-shaped network (MIMO-UNet). The performance can benefit from the operation of multi-scale images by MIMO-UNet. However, most deblurring networks are designed for global blur, while local blur is more common under vehicle-surveillance scenarios since the motion blur is primarily caused by local moving vehicles. Based on this observation, we further propose an Intersection over Patch (IoP) factor and a supervised morphological loss to improve the performance based on local blur. Extensive experiments on a public and a self-established dataset are carried out to verify the effectiveness. As a result, the deblurring behavior based on PSNR is improved at least 0.21 dB based on GOPRO and 0.74 dB based on the self-established datasets compared to the existing benchmarks.

Exploring Techniques for Abnormal Event Detection in Video Surveillance

This study explores and evaluates the effectiveness of various abnormal event detection techniques in video surveillance, addressing challenges such as intrusions, accidents, and suspicious activities. Through a systematic review of related papers, the study reveals the prevalence of traditional methods like background subtraction and motion detection despite their limitations in complex scenarios. It highlights the increasing use of deep learning techniques, particularly CNNs and RNNs, which show promise but require substantial labeled data. The findings underscore the importance of selecting proper detection techniques based on specific surveillance scenarios and emphasize the need for extensive labeled datasets for deep learning methods. The originality of this study lies in its comprehensive review and comparison of various abnormal event detection techniques, providing valuable insights and practical implications for advancing video surveillance systems.

Night time Surveillance in Communication Using YOLOv3

Nighttime surveillance presents significant challenges due to low visibility, varying lighting conditions, and interference from artificial light sources. Hence, this study proposed an effective object detection framework using the YOLOv3 model to enhance real-time monitoring in night surveillance applications, specifically within communication and security systems. YOLOv3's architecture, with its multi-scale detection and use of predefined anchor boxes, enables robust detection of objects under low-light environments and amidst light interference from vehicles and streetlights. The proposed system is tested on a night surveillance dataset, where it demonstrates high precision and speed in identifying objects, making it suitable for real-time applications. With Mean Average Precision (mAP) 87.9%, YOLOv3 effectively balances detection accuracy with inference time, ensuring minimal latency in live surveillance feeds. The results indicate that YOLOv3 outperforms traditional models such as Faster RCNN, particularly in detecting small objects under poor illumination. This approach offers a reliable solution for enhancing communication and security systems in nighttime surveillance scenarios.

Acceptance of AI-powered facial recognition technology in surveillance scenarios: Role of trust, security, and privacy perceptions

The study examines the roles of various layers of trust, as well as privacy and security concerns, in shaping the acceptance of AI-powered facial recognition technology (FRT) in three surveillance scenarios—public spaces, hospitals, and schools. Based on survey data from 575 U S. participants, we found that the context in which FRT is deployed shapes people's perceptions and acceptance of the technology. People perceived greater safety gains in schools and greater privacy risks in public spaces. Trust in officials, familiarity with FRT, and perceived security benefits positively predicted acceptance, while distrust and perceived privacy risks negatively predicted acceptance. These findings offer insights for stakeholders of FRT, policymakers, and organizations that seek to implement AI-powered surveillance, emphasizing the need to address public trust and privacy concerns.

UAV Swarm Target Identification and Quantification Based on Radar Signal Independency Characterization

Radar surveillance of noncooperative UAV swarm is challenging and is involved in many critical surveillance scenarios. The multimodality property of dynamic UAV swarm targets presents larger radar signature complexity and elevates the radar detection difficulty. The swarm unit number ambiguity from dense UAV grouping also inhibits radar monitoring accuracy. Inspired by the coherent integration essence of swarm target signals, this paper proposes a radar signal processing framework based on complex valued independent component analysis (cICA) for swarm target identification and quantification. The target detection threshold is determined from pure clutter signals after cICA processing. A customized clustering algorithm is applied on independent components for swarm target quantification. Target detection and quantification methods are verified with various multimodality UAV swarm flight plans. The results indicate that the detection performance of the proposed method is comparable with conventional CFAR algorithms with better stability performance. The target quantification procedure could estimate swarm unit numbers with acceptable numerical deviations. More discussions are given on the relevance between quantification accuracy and swarm configurations with respect to signal independency mechanisms. Efficiency discussions reveal the bottleneck of the proposed method for future optimization works.

Indoor Infrared Sensor Layout Optimization for Elderly Monitoring Based on Fused Genetic Gray Wolf Optimization (FGGWO) Algorithm.

With the increasing aging of the global population, the efficiency and accuracy of the elderly monitoring system become crucial. In this paper, a sensor layout optimization method, the Fusion Genetic Gray Wolf Optimization (FGGWO) algorithm, is proposed which utilizes the global search capability of Genetic Algorithm (GA) and the local search capability of Gray Wolf Optimization algorithm (GWO) to improve the efficiency and accuracy of the sensor layout in elderly monitoring systems. It does so by optimizing the indoor infrared sensor layout in the elderly monitoring system to improve the efficiency and coverage of the sensor layout in the elderly monitoring system. Test results show that the FGGWO algorithm is superior to the single optimization algorithm in monitoring coverage, accuracy, and system efficiency. In addition, the algorithm is able to effectively avoid the local optimum problem commonly found in traditional methods and to reduce the number of sensors used, while maintaining high monitoring accuracy. The flexibility and adaptability of the algorithm bode well for its potential application in a wide range of intelligent surveillance scenarios. Future research will explore how deep learning techniques can be integrated into the FGGWO algorithm to further enhance the system's adaptive and real-time response capabilities.

A Hybrid Neuromorphic Object Tracking and Classification Framework for Real-Time Systems.

Deep learning inference that needs to largely take place on the "edge" is a highly computational and memory intensive workload, making it intractable for low-power, embedded platforms such as mobile nodes and remote security applications. To address this challenge, this article proposes a real-time, hybrid neuromorphic framework for object tracking and classification using event-based cameras that possess desirable properties such as low-power consumption (5-14 mW) and high dynamic range (120 dB). Nonetheless, unlike traditional approaches of using event-by-event processing, this work uses a mixed frame and event approach to get energy savings with high performance. Using a frame-based region proposal method based on the density of foreground events, a hardware-friendly object tracking scheme is implemented using the apparent object velocity while tackling occlusion scenarios. The frame-based object track input is converted back to spikes for TrueNorth (TN) classification via the energy-efficient deep network (EEDN) pipeline. Using originally collected datasets, we train the TN model on the hardware track outputs, instead of using ground truth object locations as commonly done, and demonstrate the ability of our system to handle practical surveillance scenarios. As an alternative tracker paradigm, we also propose a continuous-time tracker with C++ implementation where each event is processed individually, which better exploits the low latency and asynchronous nature of neuromorphic vision sensors. Subsequently, we extensively compare the proposed methodologies to state-of-the-art event-based and frame-based methods for object tracking and classification, and demonstrate the use case of our neuromorphic approach for real-time and embedded applications without sacrificing performance. Finally, we also showcase the efficacy of the proposed neuromorphic system to a standard RGB camera setup when simultaneously evaluated over several hours of traffic recordings.

An efficient weapon detection system using NSGCU-DCNN classifier in surveillance

Weapon detection systems play a significant role in ensuring public safety, specifically in surveillance scenarios. But, poor performance has been shown by the prevailing research methodologies towards occluded and customized weapons. Thus, to resolve this issue, a Non-linear and Scalar Growing Cosine Unit based Deep Convolutional Neural Network (NSGCU-DCNN)-based efficient weapon detection system in the surveillance video is proposed. Primarily, the input video is converted into frames. The pre-processing process is applied for each frame. By utilizing Gaussian Kernelized Double Plateau Histogram Equalization (GKDPHE), contrast enhancement is performed grounded on the frame’s brightness. After that, for each frame, the motions are estimated by using the Frobenius Norm-based Three Step Search (FN-TSS) algorithm. Thereafter, by utilizing Weibull Distributed Viola Jones (WDVJ), the objects are detected, and the detected objects are cropped for further analysis. Next, the edge information of each object is determined by the Canny Edge Detection (CED) algorithm. Then, the frame is further converted into a 3x3 block. Afterward, the silhouette coefficient is calculated between the frames and the mask images. Further, the features are extracted and selected from the silhouette estimated frame. The NSGCU-DCNN classifies the weapons grounded on the selected features. The experimental outcomes exhibited that when contrasted with the prevailing techniques, the proposed mechanism withstands high-accuracy rates.

2D human skeleton action recognition with spatial constraints

AbstractHuman actions are predominantly presented in 2D format in video surveillance scenarios, which hinders the accurate determination of action details not apparent in 2D data. Depth estimation can aid human action recognition tasks, enhancing accuracy with neural networks. However, reliance on images for depth estimation requires extensive computational resources and cannot utilise the connectivity between human body structures. Besides, the depth information may not accurately reflect actual depth ranges, necessitating improved reliability. Therefore, a 2D human skeleton action recognition method with spatial constraints (2D‐SCHAR) is introduced. 2D‐SCHAR employs graph convolution networks to process graph‐structured human action skeleton data comprising three parts: depth estimation, spatial transformation, and action recognition. The initial two components, which infer 3D information from 2D human skeleton actions and generate spatial transformation parameters to correct abnormal deviations in action data, support the latter in the model to enhance the accuracy of action recognition. The model is designed in an end‐to‐end, multitasking manner, allowing parameter sharing among these three components to boost performance. The experimental results validate the model's effectiveness and superiority in human skeleton action recognition.

Human height estimation using AI-assisted computer vision for intelligent video surveillance system

In urban areas, technological advancements have led to an increased focus on height as a critical human characteristic for surveillance purposes. Face recognition often encounters challenges due to occlusion and masks, necessitating the use of height, build, and torso. Accurately estimating human height in surveillance scenarios is complex due to camera calibration, posture variations, and movement patterns. This research introduces a novel human height estimation method for surveillance systems, along with a dedicated dataset. The process begins with camera calibration to rectify lens distortions. A deep learning-based YOLOv7-Occlusion Aware (YOLOv7-OA) target detection technique is employed to precisely locate individuals within the frame. The study assesses the impact of camera height and deflection angle on height estimation across different areas of the field of vision (FOV). The proposed method yields a mean absolute error of 0.02 cm to 0.8 cm across various FOV zones, surpassing the previous 1.39 cm benchmark findings.

Nonconvex distributed feedback optimization for aggregative cooperative robotics

Distributed aggregative optimization is a recently emerged framework in which the agents of a network want to minimize the sum of local objective functions, each one depending on the agent decision variable (e.g., the local position of a team of robots) and an aggregation of all the agents’ variables (e.g., the team barycenter). In this paper, we address a distributed feedback optimization framework in which agents implement a local (distributed) policy to reach a steady-state minimizing an aggregative cost function. We propose AGGREGATIVE TRACKING FEEDBACK, i.e., a novel distributed feedback optimization law in which each agent combines a closed-loop gradient flow with a consensus-based dynamic compensator reconstructing the missing global information. By using tools from system theory, we prove that AGGREGATIVE TRACKING FEEDBACK steers the network to a stationary point of an aggregative optimization problem with (possibly) nonconvex objective function. The effectiveness of the proposed method is validated through numerical simulations on a multi-robot surveillance scenario.

LidPose: Real-Time 3D Human Pose Estimation in Sparse Lidar Point Clouds with Non-Repetitive Circular Scanning Pattern.

In this paper, we propose a novel, vision-transformer-based end-to-end pose estimation method, LidPose, for real-time human skeleton estimation in non-repetitive circular scanning (NRCS) lidar point clouds. Building on the ViTPose architecture, we introduce novel adaptations to address the unique properties of NRCS lidars, namely, the sparsity and unusual rosetta-like scanning pattern. The proposed method addresses a common issue of NRCS lidar-based perception, namely, the sparsity of the measurement, which needs balancing between the spatial and temporal resolution of the recorded data for efficient analysis of various phenomena. LidPose utilizes foreground and background segmentation techniques for the NRCS lidar sensor to select a region of interest (RoI), making LidPose a complete end-to-end approach to moving pedestrian detection and skeleton fitting from raw NRCS lidar measurement sequences captured by a static sensor for surveillance scenarios. To evaluate the method, we have created a novel, real-world, multi-modal dataset, containing camera images and lidar point clouds from a Livox Avia sensor, with annotated 2D and 3D human skeleton ground truth.

Anomaly Detection for Video Surveillance

Anomaly detection in video surveillance is critical for identifying abnormal behaviours or events that deviate from typical patterns. This abstract explores various techniques used to detect anomalies in surveillance videos, including traditional methods like motion detection and background subtraction, as well as advanced approaches such as deep learning and anomaly scoring algorithms. By leveraging these techniques, surveillance systems can automatically detect suspicious activities such as trespassing, theft, or violence, enabling timely intervention and improved security measures. Highlighting the challenges in anomaly detection, such as dealing with complex scenes, occlusions and discusses ongoing research efforts to enhance the accuracy and efficiency of anomaly detection systems in real-world surveillance scenarios. Key Words: anomaly, suspicious, pattern, motion, detection, security measure

Strategies for Optimized UAV Surveillance in Various Tasks and Scenarios: A Review

This review paper provides insights into optimization strategies for Unmanned Aerial Vehicles (UAVs) in a variety of surveillance tasks and scenarios. From basic path planning to complex mission execution, we comprehensively evaluate the multifaceted role of UAVs in critical areas such as infrastructure inspection, security surveillance, environmental monitoring, archaeological research, mining applications, etc. The paper analyzes in detail the effectiveness of UAVs in specific tasks, including power line and bridge inspections, search and rescue operations, police activities, and environmental monitoring. The focus is on the integration of advanced navigation algorithms and artificial intelligence technologies with UAV surveillance and the challenges of operating in complex environments. Looking ahead, this paper predicts trends in cooperative UAV surveillance networks and explores the potential of UAVs in more challenging scenarios. This review not only provides researchers with a comprehensive analysis of the current state of the art, but also highlights future research directions, aiming to engage and inspire readers to further explore the potential of UAVs in surveillance missions.

Integration of Fire Detection and Personnel Accountability System

Fire incidents in public spaces pose significant risks to life and property. Traditional fire detection systems often suffer from delays and inaccuracies, necessitating the development of advanced solutions. This paper presents a comprehensive system for integrating fire detection and personnel accountability in surveillance scenarios. With the increasing concerns about safety in public spaces, the need for effective systems to detect fires and ensure personnel safety has become paramount. Traditional methods rely on manual intervention or basic sensors, often resulting in delayed or inaccurate alerts. To address these challenges, our system employs advanced image processing and machine learning to enhance safety measures. By analyzing surveillance camera feeds, it detects changes in pixel intensity, smoke patterns, and heat signatures to identify potential fire incidents. Machine learning models, such as convolutional neural networks (CNNs), are trained to accurately classify fire and non-fire regions in video frames. Additionally, the system tracks personnel presence and movement, ensuring accountability within the surveillance area. Modular and scalable, our system integrates seamlessly with existing infrastructure, providing reliable fire detection and personnel monitoring. Experimental results demonstrate its effectiveness, contributing to enhanced safety in public spaces and facilitating prompt emergency responses.

Edge HPC Architectures for AI-Based Video Surveillance Applications

The introduction of artificial intelligence (AI) in video surveillance systems has significantly transformed security practices, allowing for autonomous monitoring and real-time detection of threats. However, the effectiveness and efficiency of AI-powered surveillance rely heavily on the hardware infrastructure, specifically high-performance computing (HPC) architectures. This article examines the impact of different platforms for HPC edge servers, including x86 and ARM CPU-based systems and Graphics Processing Units (GPUs), on the speed and accuracy of video processing tasks. By using advanced deep learning frameworks, a video surveillance system based on YOLO object detection and DeepSort tracking algorithms is developed and evaluated. This study thoroughly assesses the strengths, limitations, and suitability of different hardware architectures for various AI-based surveillance scenarios.

NRPerson: A Non-Registered Multi-Modal Benchmark for Tiny Person Detection and Localization

In recent years, the detection and localization of tiny persons have garnered significant attention due to their critical applications in various surveillance and security scenarios. Traditional multi-modal methods predominantly rely on well-registered image pairs, necessitating the use of sophisticated sensors and extensive manual effort for registration, which restricts their practical utility in dynamic, real-world environments. Addressing this gap, this paper introduces a novel non-registered multi-modal benchmark named NRPerson, specifically designed to advance the field of tiny person detection and localization by accommodating the complexities of real-world scenarios. The NRPerson dataset comprises 8548 RGB-IR image pairs, meticulously collected and filtered from 22 video sequences, enriched with 889,207 high-quality annotations that have been manually verified for accuracy. Utilizing NRPerson, we evaluate several leading detection and localization models across both mono-modal and non-registered multi-modal frameworks. Furthermore, we develop a comprehensive set of natural multi-modal baselines for the innovative non-registered track, aiming to enhance the detection and localization of unregistered multi-modal data using a cohesive and generalized approach. This benchmark is poised to facilitate significant strides in the practical deployment of detection and localization technologies by mitigating the reliance on stringent registration requirements.

Dynamic Object Detection in Surveillance Videos using Temporal Convolutional Networks and Federated Learning in Edge Computing Environments

This research addresses the importance of advancing dynamic object detection in surveillance videos by introducing a novel framework that integrates Temporal Convolutional Networks (TCNs) and Federated Learning (FL) within edge computing environments. This research is motivated by the critical need for real-time threat response, enhanced security measures, and privacy preservation in dynamic surveillance scenarios. Leveraging TCNs, the system captures temporal dependencies, providing a comprehensive understanding of object movements. FL ensures decentralized model training, mitigating privacy concerns associated with centralized approaches. Current challenges in real-time processing, privacy preservation, and adaptability to dynamic environments are addressed through innovative solutions. Model optimization techniques optimize TCN efficiency, ensuring real-time processing. Advanced privacy-preserving mechanisms secure FL, addressing privacy concerns. Transfer learning and data augmentation enhance adaptability to dynamic scenarios. The proposed system not only addresses existing challenges but also contributes to the evolution of surveillance technology. Comprehensive metrics, including accuracy, sensitivity, specificity, and real-time processing metrics, provide a thorough evaluation of the system's performance. This research introduces an approach to dynamic object detection, combining TCN and FL in edge computing environments. Results show accuracy exceeding 97%, sensitivity and specificity at 97% and 98%, and F1 score reaching 96%.

A deep neural network for vehicle detection in aerial images

This research paper highlights the significance of vehicle detection in aerial images for surveillance systems, focusing on deep learning methods that outperform traditional approaches. However, the challenge of high computation complexity due to diverse vehicle appearances persists. The motivation behind this study is to highlight the crucial role of vehicle detection in aerial images for surveillance systems, emphasizing the superior performance of deep learning methods compared to traditional approaches. To address this, a lightweight deep neural network-based model is developed, striking a balance between accuracy and efficiency enabling real-time operation. The model is trained and evaluated on a standardized dataset, with extensive experiments demonstrating its ability to achieve accurate vehicle detection with significantly reduced computation costs, offering a practical solution for real-world aerial surveillance scenarios.

面向对海监视的舰船目标跟踪与航迹融合数据集

Real-time trajectory association and trajectory fusion in maritime surveillance pose great challenges and remain hot issues in security, regional situation monitoring, and long-range precision strikes for both military and civilian applications. High-quality datasets play a pivotal role in advancing research in target tracking and fusion technologies within this domain. This paper addresses the data requirements for technological research in target tracking and fusion, as well as the limitations of currently available datasets, including data scarcity, inadequate scene design specificity, uniform data formats, and incomplete data descriptions. We used simulation software to emulate multi-sensor multi-target detection data in complex scenarios, so as to present a dataset tailored for typical maritime surveillance scenarios—targeting ships using 2D radar and Electronic Support Measures (ESM) sensors. The simulation software comprises a scenario generator and a sensor simulator, creating a mature target tracking scenario simulation environment with realistic detection data modeling capabilities. The dataset includes data from 2D radar and ESM sensors, covering typical maritime ship categories, supporting configurations with radiation sources. It is designed for a variety of scenarios such as high-speed motion, dense traffic, multi-sensor data fusion, specific ship detection, and cross-positioning. The dataset contains a total of 368,155 target tracks from 101 ships, spanning a duration of 15,000 seconds. The data format conforms to actual equipment reporting scenarios, while the detection error model accurately reflects real-world conditions. Accuracy assessment and data completeness are ensured through various methods including normality testing of data error, scenario testing of detection rate and false alarm rate, as well as field research. This dataset can provide fundamental data for algorithmic research and validation in the ship's target tracking, trajectory fusion and other related areas.