Detection In Surveillance Videos Research Articles

Two–Stage Detection and Localization of Inter–Frame Tampering in Surveillance Videos Using Texture and Optical Flow

Surveillance cameras provide security and protection through real-time monitoring or through the investigation of recorded videos. The authenticity of surveillance videos cannot be taken for granted, but tampering detection is challenging. Existing techniques face significant limitations, including restricted applicability, poor generalizability, and high computational complexity. This paper presents a robust detection system to meet the challenges of frame duplication (FD) and frame insertion (FI) detection in surveillance videos. The system leverages the alterations in texture patterns and optical flow between consecutive frames and works in two stages; first, suspicious tampered videos are detected using motion residual–based local binary patterns (MR–LBPs) and SVM; second, by eliminating false positives, the precise tampering location is determined using the consistency in the aggregation of optical flow and the variance in MR–LBPs. The system is extensively evaluated on a large COMSATS Structured Video Tampering Evaluation Dataset (CSVTED) comprising challenging videos with varying quality of tampering and complexity levels and cross–validated on benchmark public domain datasets. The system exhibits outstanding performance, achieving 99.5% accuracy in detecting and pinpointing tampered regions. It ensures the generalization and wide applicability of the system while maintaining computational efficiency.

Real-Time Smoke Detection in Surveillance Videos Using an Enhanced RT-DETR Framework with Triplet Attention and HS-FPN

This study addresses the urgent need for an efficient and accurate smoke detection system to enhance safety measures in fire monitoring, industrial safety, and urban surveillance. Given the complexity of detecting smoke in diverse environments and under real-time constraints, our research aims to solve challenges related to low-resolution imagery, limited computational resources, and environmental variability. This study introduces a novel smoke detection system that utilizes the real-time detection Transformer (RT-DETR) architecture to enhance the speed and precision of video analysis. Our system integrates advanced modules, including triplet attention, ADown, and a high-level screening-feature fusion pyramid network (HS-FPN), to address challenges related to low-resolution imagery, real-time processing constraints, and environmental variability. The triplet attention mechanism is essential for detecting subtle smoke features, often overlooked due to their nuanced nature. The ADown module significantly reduces computational complexity, enabling real-time operation on devices with limited resources. Furthermore, the HS-FPN enhances the system’s robustness by amalgamating multi-scale features for reliable detection across various smoke types and sizes. Evaluation using a diverse dataset showcased notable improvements in average precision (AP50) and frames per second (FPS) metrics compared to existing state-of-the-art networks. Ablation studies validated the contributions of each component in achieving an optimal balance between accuracy and operational efficiency. The RT-DETR-based smoke detection system not only meets real-time requirements for applications like fire monitoring, industrial safety, and urban surveillance but also establishes a new performance benchmark in this field.

Temporally enhanced abnormal behavior detection based on multi-channel coupling

Abnormal behavior detection in surveillance videos based on deep learning has becomea hot topic in current research. However, due to the complexity and variability of crowd movement and external environment, detecting abnormal behavior faces great challenges. Current abnormal behavior recognition models have limitations in feature extraction and pay insufficient attention to dynamic temporal features. To address these problems, a spatiotemporal enhancement anomaly detection method based on multi-channel coupling is proposed. Based on the SlowFast network, a multi-channel coupled temporal enhancement module and a spatial enhancement module are introduced respectively to extract more discriminative static and dynamic feature information. A large number of experiments are carried out on three benchmark datasets (Violent Flow, Hockey Fight, and Real-life Violence Situations). The results show that the prediction accuracy of the proposed abnormal behavior recognition method reaches 95.3%、97.3%and respectively 94%, thus verifying the effectiveness of the method.

Anomaly Detection in Video Surveillance: A Comparative Analysis of Deep Learning Models

Anomaly detection in video surveillance is critical for enhancing security and public safety across various applications, including traffic monitoring, public spaces, and industrial settings. Traditional methods often struggle with the complexity and variability of real-world data, prompting a shift towards advanced machine learning models. This paper presents a comprehensive analysis of deep learning algorithms, including YOLOv5, 3D CNNs, LSTM, Deep SVDD, Vision Transformers, Temporal Transformers, and Autoencoders, applied to three benchmark datasets: CIFAR-10, MVTec AD, and UCSD Anomaly Detection. We compare these algorithms based on accuracy, precision, recall, and F1-score, providing insight into their strengths and weaknesses. The results suggest that Vision Transformers and CNN-LSTM hybrids offer superior performance across spatial and temporal anomaly detection tasks.

AONet: Attention network with optional activation for unsupervised video anomaly detection

AbstractAnomaly detection in video surveillance is crucial but challenging due to the rarity of irregular events and ambiguity of defining anomalies. We propose a method called AONet that utilizes a spatiotemporal module to extract spatiotemporal features efficiently, as well as a residual autoencoder equipped with an attention network for effective future frame prediction in video anomaly detection. AONet utilizes a novel activation function called OptAF that combines the strengths of the ReLU, leaky ReLU, and sigmoid functions. Furthermore, the proposed method employs a combination of robust loss functions to address various aspects of prediction errors and enhance training effectiveness. The performance of the proposed method is evaluated on three widely used benchmark datasets. The results indicate that the proposed method outperforms existing state‐of‐the‐art methods and demonstrates comparable performance, achieving area under the curve values of 97.0%, 86.9%, and 73.8% on the UCSD Ped2, CUHK Avenue, and ShanghaiTech Campus datasets, respectively. Additionally, the high speed of the proposed method enables its application to real‐time tasks.

A Comprehensive Comparative Analysis of Deep Learning Architectures for Suspicious Activity Detection in Video Surveillance

Abstract: In the landscape of modern security infrastructure, video surveillance has evolved into a ubiquitous and indispensable tool for ensuring public safety and safeguarding critical assets. This research paper delves into an extensive examination of various deep learning architectures for the purpose of detecting suspicious activities in video surveillance. The investigation encompasses Convolutional Long Short-Term Memory (ConvLSTM), Convolutional Neural Network (CNN) combined with Long Short-Term Memory (LSTM), ConvLSTM, Bidirectional Long Short-Term Memory (BiLSTM), and diverse combinations thereof. Each model is rigorously trained and tested on a carefully curated dataset designed to encapsulate a spectrum of normal and suspicious activities like shooting and fighting. The study aspires to identify the most efficacious model for enhancing the accuracy of suspicious activity detection in complex and dynamic environments. Metrics such as accuracy, precision, recall, and F1 score will be rigorously assessed to ascertain the model’s comparative performances

Balancing Accuracy and Training Time in Federated Learning for Violence Detection in Surveillance Videos: A Study of Neural Network Architectures

Balancing Accuracy and Training Time in Federated Learning for Violence Detection in Surveillance Videos: A Study of Neural Network Architectures

Safeguards-related event detection in surveillance video using semi-supervised learning approach

We develop a deep learning model employing a semi-supervised learning approach, which can detect automatically safeguards-related events in nuclear facility from surveillance video. Our model is designed after a comprehensive analysis of the trends in artificial intelligence-based models to identify abnormal events in video. Our model incorporates a reconstruction module and a prediction module independently. The reconstruction module is trained to generate video frames within a sliding window, while the prediction module is trained to predict future motion feature based on the motion features within the video frames in a sliding window. Each module utilizes an autoencoder with a memory module positioned between an encoder and an decoder of the autoencoder. We evaluate the model's performance using a benchmark dataset and a self-produced dataset obtained from facility related to pyroprocessing. Our model's performanace is comparable to or superior to that of the prevous models from the benchmark dataset analysis, and all the abnormal events can be detected without false positive error from the self-produced dataset analysis.

Crime Activity Detection in Surveillance Videos Based on Developed Deep Learning Approach

In modern communities, lots of offenders are prone to recidivism, hence, there is a requirement to inhibit such criminals, especially from impending socioeconomically disadvantaged and high-crime areas that experience elevated levels of criminal activity, involving drug-related offenses, violence, theft, and other forms of anti-social behavior. Consequently, surveillance cameras have been installed in relevant institutions, and further personnel have been provided to monitor videos using various surveillance apparatus. However, relying solely on monitoring with the naked eye and manual video processing falls short of accurately evaluating the footage acquired via such cameras. To handle the issues of conventional systems, there is a need for a system that is able to classify acquired images while supporting surveillance personnel actively. Therefore, in this paper, a deep-learning approach is developed to build a crime detection system. This developed approach includes various layers necessary to perform feature extraction and classification processes and make the system capable of efficiently and accurately detecting crime activities from surveillance video frames. Besides the proposed crime activity detection system, two deep-learning approaches (EfficientNet-B7, and MobileNet-V2) are trained and assessed on the popular UCF Crime and DCSASS datasets. Generally, the proposed detection system encompasses dataset preparation and pre-processing, splitting the pre-processed crime activity image dataset, and implementing the proposed deep learning approach and other pre-trained approaches.

Anomaly detection in surveillance videos using Transformer with margin learning

Anomaly detection in surveillance videos using Transformer with margin learning

Enhancing Video Anomaly Detection Using a Transformer Spatiotemporal Attention Unsupervised Framework for Large Datasets

This work introduces an unsupervised framework for video anomaly detection, leveraging a hybrid deep learning model that combines a vision transformer (ViT) with a convolutional spatiotemporal relationship (STR) attention block. The proposed model addresses the challenges of anomaly detection in video surveillance by capturing both local and global relationships within video frames, a task that traditional convolutional neural networks (CNNs) often struggle with due to their localized field of view. We have utilized a pre-trained ViT as an encoder for feature extraction, which is then processed by the STR attention block to enhance the detection of spatiotemporal relationships among objects in videos. The novelty of this work is utilizing the ViT with the STR attention to detect video anomalies effectively in large and heterogeneous datasets, an important thing given the diverse environments and scenarios encountered in real-world surveillance. The framework was evaluated on three benchmark datasets, i.e., the UCSD-Ped2, CHUCK Avenue, and ShanghaiTech. This demonstrates the model’s superior performance in detecting anomalies compared to state-of-the-art methods, showcasing its potential to significantly enhance automated video surveillance systems by achieving area under the receiver operating characteristic curve (AUC ROC) values of 95.6, 86.8, and 82.1. To show the effectiveness of the proposed framework in detecting anomalies in extra-large datasets, we trained the model on a subset of the huge contemporary CHAD dataset that contains over 1 million frames, achieving AUC ROC values of 71.8 and 64.2 for CHAD-Cam 1 and CHAD-Cam 2, respectively, which outperforms the state-of-the-art techniques.

Enhanced Abnormal Event Detection in Surveillance Videos Through Optimized Regression Algorithms

Enhanced Abnormal Event Detection in Surveillance Videos Through Optimized Regression Algorithms

Sequential Learning of Flame Objects Sorted by Size for Early Fire Detection in Surveillance Videos

Sequential Learning of Flame Objects Sorted by Size for Early Fire Detection in Surveillance Videos

Real-Time Anomaly Detection in Video Surveillance: A Mathematical Modeling and Nonlinear Analysis Perspective with MobileNet and Bi-LSTM

Real-Time Anomaly Detection in Video Surveillance: A Mathematical Modeling and Nonlinear Analysis Perspective with MobileNet and Bi-LSTM

Anomaly Detection for Video Surveillance

Anomaly detection in video surveillance is critical for identifying abnormal behaviours or events that deviate from typical patterns. This abstract explores various techniques used to detect anomalies in surveillance videos, including traditional methods like motion detection and background subtraction, as well as advanced approaches such as deep learning and anomaly scoring algorithms. By leveraging these techniques, surveillance systems can automatically detect suspicious activities such as trespassing, theft, or violence, enabling timely intervention and improved security measures. Highlighting the challenges in anomaly detection, such as dealing with complex scenes, occlusions and discusses ongoing research efforts to enhance the accuracy and efficiency of anomaly detection systems in real-world surveillance scenarios. Key Words: anomaly, suspicious, pattern, motion, detection, security measure

Normality learning reinforcement for anomaly detection in surveillance videos

Video Anomaly Detection (VAD) is a key technology that enables automatic anomaly detection in surveillance video systems. Due to the considerable dimensions and fine-grained spatial details of video sequences, VAD tasks necessitate the collaboration of learning both spatial and temporal contextualized information. Due to the restricted understanding of spatial details, previous unsupervised VAD works have yet to fully tap into the potential of prediction-based methods to fully capture the spatial–temporal correlations. Inspired by the lateral geniculate nucleus in visual processing, we propose a Normality Learning Reinforcement framework of VAD (NLR-VAD) that enables learning more sophisticated and efficient spatial–temporal interactions. Specifically, NLR-VAD introduces a Normality Learning Reinforcement Unit (NLRU) rooted in diffusion models. NLRU refines the irrelevant information and amplifies the knowledge retrieval of spatial normality, particularly in high-resolution details. NLRU collaborates with Memory-Augmented Auto-Encoder (MAAE) to enhance spatial–temporal normality through fine-grained extractions. Moreover, a cross-NLRU fusion unit is proposed for information integration as a bridge between NLRU and MAAE. Quantitative results on three benchmarks show that NLR-VAD performs competitively compared with the previous methods. Extended analysis and visualization fully demonstrate the effectiveness of the proposed units.

Real-Time Violence Detection in Surveillance Videos Using Deep Learning Approach

Abstract: In order to tackle the ongoing problem of keeping fruits fresh, this research presents an automated freshness detection. The ever-growing presence of surveillance cameras has undoubtedly enhanced public safety. However, manually reviewing vast quantities of video footage for violence detection remains a tedious and error-prone task. This paper proposes a deep learningbased approach for anomaly detection in surveillance videos, with a specific focus on identifying violent activities. Traditional surveillance methods often lack the sophistication to distinguish between normal behaviour and potentially threatening actions. Deep learning offers a significant advantage by automating the identification of deviations from established behavioural patterns in video data. This automation enables real-time analysis of footage, potentially signifying the occurrence of violent incidents and allowing for a swifter response from security personnel. This research delves into the application of deep learning models for violence detection in surveillance footage. We begin by discussing the limitations inherent in conventional methods, such as motion detection and manual review, which struggle to capture the nuances of human behaviour. Subsequently, we explore the advantages that deep learning-based approaches bring to the table, including their ability to learn complex patterns from large datasets and identify subtle anomalies that might escape human observation.

Dynamic Object Detection in Surveillance Videos using Temporal Convolutional Networks and Federated Learning in Edge Computing Environments

This research addresses the importance of advancing dynamic object detection in surveillance videos by introducing a novel framework that integrates Temporal Convolutional Networks (TCNs) and Federated Learning (FL) within edge computing environments. This research is motivated by the critical need for real-time threat response, enhanced security measures, and privacy preservation in dynamic surveillance scenarios. Leveraging TCNs, the system captures temporal dependencies, providing a comprehensive understanding of object movements. FL ensures decentralized model training, mitigating privacy concerns associated with centralized approaches. Current challenges in real-time processing, privacy preservation, and adaptability to dynamic environments are addressed through innovative solutions. Model optimization techniques optimize TCN efficiency, ensuring real-time processing. Advanced privacy-preserving mechanisms secure FL, addressing privacy concerns. Transfer learning and data augmentation enhance adaptability to dynamic scenarios. The proposed system not only addresses existing challenges but also contributes to the evolution of surveillance technology. Comprehensive metrics, including accuracy, sensitivity, specificity, and real-time processing metrics, provide a thorough evaluation of the system's performance. This research introduces an approach to dynamic object detection, combining TCN and FL in edge computing environments. Results show accuracy exceeding 97%, sensitivity and specificity at 97% and 98%, and F1 score reaching 96%.

GDJN: Global density joined network for abnormal event detection in surveillance videos

Detecting abnormal events in surveillance involves identifying unexpected behavior through video analysis. This involves recognizing patterns or deviations from normal behavior and taking actions to mitigate potential risks. However, the distribution of data can change over time, leading to concept drift, which can make it challenging to accurately detect abnormal events. To address this issue, a new approach using a global density network (GDN) has been proposed. The GDN allows for more efficient identification of object distributions in surveillance videos, leading to improved accuracy in abnormal event detection. The proposed method combines features extracted by a backbone network with a global density joined network (GDJN), which refines density features using dilated convolutional networks. A multistage long short-term memory (LSTM) network is then used to classify abnormal events. The experimental results are conducted on two datasets, UMN and UCSD Ped2. The achieved F1 scores were 93.42 and 94.46 respectively, with corresponding AUC values of 93.5 and 94.8.

A Light Weight Depthwise Separable Layer Optimized CNN Architecture for Object-Based Forgery Detection in Surveillance Videos

Abstract The present era is at the peak of technological advancement in image and video processing techniques, with user-friendly accessible tools/techniques. This immersive technology development makes video forensics enormously challenging. Specifically, the passive approaches to object-based forgeries in videos are crucial for legal and judicial matters. Hence, to ensure the integrity of the videos, a scientific, statistical and passive investigation of videos is required to maintain the spatial and temporal information content. This paper aims to develop a passive approach for digging out the forgery traces by applying the motion residue windowing technique for object removal forgery in surveillance videos. The novel max averaging windowing techniques improve visual imprints of the object removal forgery in the videos from the existing methods in the literature. A deep learning approach is the next step for achieving forgery detection in surveillance videos. The proposed lightweight depth-separable layer-optimized CNN has fast execution speed, optimized in terms of parameters without compromising the desired accuracy. This network is trained at a frame level with 98.60% testing accuracy, followed by a pipeline architecture of the proposed model for detection of forgery at video level with 99.01% accuracy. The suggested model works better than current models regarding post-processing operations, compression rates, forged video detection accuracy, precision, recall and F1 score.