Detection In Surveillance Videos Research Articles

Anomaly Detection for Video Surveillance

Anomaly detection in video surveillance is critical for identifying abnormal behaviours or events that deviate from typical patterns. This abstract explores various techniques used to detect anomalies in surveillance videos, including traditional methods like motion detection and background subtraction, as well as advanced approaches such as deep learning and anomaly scoring algorithms. By leveraging these techniques, surveillance systems can automatically detect suspicious activities such as trespassing, theft, or violence, enabling timely intervention and improved security measures. Highlighting the challenges in anomaly detection, such as dealing with complex scenes, occlusions and discusses ongoing research efforts to enhance the accuracy and efficiency of anomaly detection systems in real-world surveillance scenarios. Key Words: anomaly, suspicious, pattern, motion, detection, security measure

Real-Time Violence Detection in Surveillance Videos Using Deep Learning Approach

Abstract: In order to tackle the ongoing problem of keeping fruits fresh, this research presents an automated freshness detection. The ever-growing presence of surveillance cameras has undoubtedly enhanced public safety. However, manually reviewing vast quantities of video footage for violence detection remains a tedious and error-prone task. This paper proposes a deep learningbased approach for anomaly detection in surveillance videos, with a specific focus on identifying violent activities. Traditional surveillance methods often lack the sophistication to distinguish between normal behaviour and potentially threatening actions. Deep learning offers a significant advantage by automating the identification of deviations from established behavioural patterns in video data. This automation enables real-time analysis of footage, potentially signifying the occurrence of violent incidents and allowing for a swifter response from security personnel. This research delves into the application of deep learning models for violence detection in surveillance footage. We begin by discussing the limitations inherent in conventional methods, such as motion detection and manual review, which struggle to capture the nuances of human behaviour. Subsequently, we explore the advantages that deep learning-based approaches bring to the table, including their ability to learn complex patterns from large datasets and identify subtle anomalies that might escape human observation.

Dynamic Object Detection in Surveillance Videos using Temporal Convolutional Networks and Federated Learning in Edge Computing Environments

This research addresses the importance of advancing dynamic object detection in surveillance videos by introducing a novel framework that integrates Temporal Convolutional Networks (TCNs) and Federated Learning (FL) within edge computing environments. This research is motivated by the critical need for real-time threat response, enhanced security measures, and privacy preservation in dynamic surveillance scenarios. Leveraging TCNs, the system captures temporal dependencies, providing a comprehensive understanding of object movements. FL ensures decentralized model training, mitigating privacy concerns associated with centralized approaches. Current challenges in real-time processing, privacy preservation, and adaptability to dynamic environments are addressed through innovative solutions. Model optimization techniques optimize TCN efficiency, ensuring real-time processing. Advanced privacy-preserving mechanisms secure FL, addressing privacy concerns. Transfer learning and data augmentation enhance adaptability to dynamic scenarios. The proposed system not only addresses existing challenges but also contributes to the evolution of surveillance technology. Comprehensive metrics, including accuracy, sensitivity, specificity, and real-time processing metrics, provide a thorough evaluation of the system's performance. This research introduces an approach to dynamic object detection, combining TCN and FL in edge computing environments. Results show accuracy exceeding 97%, sensitivity and specificity at 97% and 98%, and F1 score reaching 96%.

GDJN: Global density joined network for abnormal event detection in surveillance videos

Detecting abnormal events in surveillance involves identifying unexpected behavior through video analysis. This involves recognizing patterns or deviations from normal behavior and taking actions to mitigate potential risks. However, the distribution of data can change over time, leading to concept drift, which can make it challenging to accurately detect abnormal events. To address this issue, a new approach using a global density network (GDN) has been proposed. The GDN allows for more efficient identification of object distributions in surveillance videos, leading to improved accuracy in abnormal event detection. The proposed method combines features extracted by a backbone network with a global density joined network (GDJN), which refines density features using dilated convolutional networks. A multistage long short-term memory (LSTM) network is then used to classify abnormal events. The experimental results are conducted on two datasets, UMN and UCSD Ped2. The achieved F1 scores were 93.42 and 94.46 respectively, with corresponding AUC values of 93.5 and 94.8.

A Light Weight Depthwise Separable Layer Optimized CNN Architecture for Object-Based Forgery Detection in Surveillance Videos

Abstract The present era is at the peak of technological advancement in image and video processing techniques, with user-friendly accessible tools/techniques. This immersive technology development makes video forensics enormously challenging. Specifically, the passive approaches to object-based forgeries in videos are crucial for legal and judicial matters. Hence, to ensure the integrity of the videos, a scientific, statistical and passive investigation of videos is required to maintain the spatial and temporal information content. This paper aims to develop a passive approach for digging out the forgery traces by applying the motion residue windowing technique for object removal forgery in surveillance videos. The novel max averaging windowing techniques improve visual imprints of the object removal forgery in the videos from the existing methods in the literature. A deep learning approach is the next step for achieving forgery detection in surveillance videos. The proposed lightweight depth-separable layer-optimized CNN has fast execution speed, optimized in terms of parameters without compromising the desired accuracy. This network is trained at a frame level with 98.60% testing accuracy, followed by a pipeline architecture of the proposed model for detection of forgery at video level with 99.01% accuracy. The suggested model works better than current models regarding post-processing operations, compression rates, forged video detection accuracy, precision, recall and F1 score.

Exploring deep learning-based content-based video retrieval with Hierarchical Navigable Small World index and ResNet-50 features for anomaly detection

Surveillance video can capture a variety of realistic scenarios for providing security in both private and public spaces. The prediction of anomalies promptly is essential. However, existing methods are time-consuming due to marking conflicting clips in training videos. Therefore, this paper proposes a CBVR (Content-Based Video Retrieval) framework for detecting anomalies in surveillance videos. The proposed method leverages deep learning techniques and the HNSW (Hierarchical Navigable Small World) index to achieve efficient and accurate video retrieval. To preprocess the video data, frames are resized and normalized, and the dataset is augmented with data transformations such as rotations, translations, and flips to improve model generalization. The best results are offered while evaluating with the UCSD Ped2 dataset which produces the MAP score of 94.25%, and recall, precision, and F1-scores of 96.15%, 98.03%, and 92.63% respectively. The results proved that maximizing the number of layers in the HNSW index enhances the MAP score, suggesting that deeper layers are more effective at retrieving relevant results. Overall, the proposed CBVR framework provides an efficient and accurate method for anomaly detection in surveillance videos. It has potential applications in various domains, including video summarization, recommendation, and surveillance, among others.

DSLSTM: a deep convolutional encoder–decoder architecture for abnormality detection in video surveillance

DSLSTM: a deep convolutional encoder–decoder architecture for abnormality detection in video surveillance

A data stream-based approach for anomaly detection in surveillance videos

A data stream-based approach for anomaly detection in surveillance videos

Self-Supervised Masked Convolutional Transformer Block for Anomaly Detection.

Anomaly detection has recently gained increasing attention in the field of computer vision, likely due to its broad set of applications ranging from product fault detection on industrial production lines and impending event detection in video surveillance to finding lesions in medical scans. Regardless of the domain, anomaly detection is typically framed as a one-class classification task, where the learning is conducted on normal examples only. An entire family of successful anomaly detection methods is based on learning to reconstruct masked normal inputs (e.g. patches, future frames, etc.) and exerting the magnitude of the reconstruction error as an indicator for the abnormality level. Unlike other reconstruction-based methods, we present a novel self-supervised masked convolutional transformer block (SSMCTB) that comprises the reconstruction-based functionality at a core architectural level. The proposed self-supervised block is extremely flexible, enabling information masking at any layer of a neural network and being compatible with a wide range of neural architectures. In this work, we extend our previous self-supervised predictive convolutional attentive block (SSPCAB) with a 3D masked convolutional layer, a transformer for channel-wise attention, as well as a novel self-supervised objective based on Huber loss. Furthermore, we show that our block is applicable to a wider variety of tasks, adding anomaly detection in medical images and thermal videos to the previously considered tasks based on RGB images and surveillance videos. We exhibit the generality and flexibility of SSMCTB by integrating it into multiple state-of-the-art neural models for anomaly detection, bringing forth empirical results that confirm considerable performance improvements on five benchmarks: MVTec AD, BRATS, Avenue, ShanghaiTech, and Thermal Rare Event.

Anchor boxes adaptive optimization algorithm for maritime object detection in video surveillance

With the development of the marine economy, video surveillance has become an important technical guarantee in the fields of marine engineering, marine public safety, marine supervision, and maritime traffic safety. In video surveillance, maritime object detection (MOD) is one of the most important core technologies. Affected by the size of maritime objects, distance, day and night weather, and changes in sea conditions, MOD faces challenges such as false detection, missed detection, slow detection speed, and low accuracy. However, the existing object detection algorithms usually adopt predefined anchor boxes to search and locate for objects of interest, making it difficult to adapt to maritime objects’ complex features, including the varying scale and large aspect ratio difference. Therefore, this paper proposes a maritime object detection algorithm based on the improved convolutional neural network (CNN). Firstly, a differential-evolutionary-based K-means (DK-means) anchor box clustering algorithm is proposed to obtain adaptive anchor boxes to satisfy the maritime object characteristics. Secondly, an adaptive spatial feature fusion (ASFF) module is added in the neck network to enhance multi-scale feature fusion. Finally, focal loss and efficient intersection over union (IoU) loss are adopted to replace the original loss function to improve the network convergence speed. The experimental results on the Singapore maritime dataset show that our proposed algorithm improves the average precision by 7.1%, achieving 72.7%, with a detection speed of 113 frames per second, compared with You Only Look Once v5 small (YOLOv5s). Moreover, compared to other counterparts, it can achieve a better speed–accuracy balance, which is superior and feasible for the complex maritime environment.

Semi-supervised anomaly detection in video surveillance by inpainting

Semi-supervised anomaly detection in video surveillance by inpainting

Stochastic video normality network for abnormal event detection in surveillance videos

Video Anomaly Detection (VAD) aims to automatically identify unexpected spatial–temporal patterns to detect abnormal events in surveillance videos. Existing unsupervised VAD methods either use a single proxy task to represent the prototypical normal patterns, or split the video into static and dynamic parts and learn spatial and temporal normality separately, which cannot model the inherent uncertainty of motion. In this regard, we propose for the first time to learn video normality via stochastic motion representation. Specifically, the proposed Stochastic Video Normality (SVN) network learns the prototypical local appearance patterns via deterministic multi-task learning and global motion patterns in a non-deterministic manner. The stochastic representation module uses recurrent networks to model historical motion as a conditional Gaussian distribution and infers motion explicitly with the learned prior. Besides, a mask auto-encoder is introduced to explore the relationship between appearance and motion, boosting the model to learn spatial–temporal normality. Experimental results show that our proposed SVN network performs comparably to the state-of-the-art methods. Extensive analysis demonstrates the effectiveness of multi-task learning-based appearance representation and stochastic motion representation for unsupervised video anomaly detection.

End-to-end learning for weakly supervised video anomaly detection using Absorbing Markov Chain

We propose a principled deep neural network framework with Absorbing Markov Chain (AMC) for weakly supervised anomaly detection in surveillance videos. Our model consists of both a weakly supervised binary classification network and a Graph Convolutional Network (GCN), which are jointly optimized by backpropagation. Unlike the previous works that employ AMC for label noise filtering in a post-processing step, the proposed framework migrates the component inside the GCN part of our model and realizes an end-to-end learning network. The integration of the AMC module into our deep neural network model enables us to learn the associated parameters automatically, which is helpful to improve the quality of segment-wise label estimation via tightly-coupled processing between the main network and the AMC module. In addition, we introduce a pseudo-labeling strategy based on Gaussian mixture model to fully utilize examples in abnormal videos. Our algorithm achieves outstanding performance compared to the state-of-the-art weakly supervised anomaly detection methods on UCF-Crime and ShanghaiTech datasets.

Dilated spatial–temporal convolutional auto-encoders for human fall detection in surveillance videos

Although methods based on supervised learning have demonstrated remarkable performance on fall detection, these existing fall detection algorithms require a substantial quantity of manually labeled training data. In this paper, we combine dilated convolution and LSTM based on auto-encoder, which can be trained on unlabeled data, further saving time and resources, and a novel fall score is computed based on the high-quality reconstructed frame to detect falls. Extensive experimental results indicate that the proposed method further boosts the performance, achieving recognition rate of 97.1%, sensitivity rate of 93.9% and precision rate of 95.1% on the UR dataset.

An Iterative Learning Scheme with Binary Classifier for Improved Event Detection in Surveillance Video

This paper presents an iterative training framework with a binary classifier to improve the learning capability of a deep learning model for detecting abnormal behaviors in surveillance video. When a deep learning model trained on data from one surveillance video is deployed to monitor another video stream, its abnormal behavior detection performance often decreases significantly. To ensure the desired performance in new environments, the deep learning model needs to be retrained with additional training data from the new video stream. Iterative training requires manual annotation of the additional training data during the fine-tuning process, which is a tedious and error-prone task. To address this issue, this paper proposes a binary classifier to automatically label false positive data without human intervention. The binary classifier is trained on bounding boxes extracted from the detection model to identify which boxes are true positives or false positives. The proposed learning framework incrementally enhances the performance of the deep learning model for detecting abnormal behaviors in a surveillance video stream through repeated iterative learning cycles. Experimental results demonstrate that the accuracy of the detection model increases from 0.35 (mAP = 0.74) to 0.91 (mAP = 0.99) in just a few iterations.

A data-driven approach for road accident detection in surveillance videos

A data-driven approach for road accident detection in surveillance videos

Key-Frame Extraction for Reducing Human Effort in Object Detection Training for Video Surveillance

This paper presents a supervised learning scheme that employs key-frame extraction to enhance the performance of pre-trained deep learning models for object detection in surveillance videos. Developing supervised deep learning models requires a significant amount of annotated video frames as training data, which demands substantial human effort for preparation. Key frames, which encompass frames containing false negative or false positive objects, can introduce diversity into the training data and contribute to model improvements. Our proposed approach focuses on detecting false negatives by leveraging the motion information within video frames that contain the detected object region. Key-frame extraction significantly reduces the human effort involved in video frame extraction. We employ interactive labeling to annotate false negative video frames with accurate bounding boxes and labels. These annotated frames are then integrated with the existing training data to create a comprehensive training dataset for subsequent training cycles. Repeating the training cycles gradually improves the object detection performance of deep learning models to monitor a new environment. Experiment results demonstrate that the proposed learning approach improves the performance of the object detection model in a new operating environment, increasing the mean average precision (mAP@0.5) from 54% to 98%. Manual annotation of key frames is reduced by 81% through the proposed key-frame extraction method.

Abnormal Behavior Detection in Video Surveillance Using Inception-v3 Transfer Learning Approaches

The use of video surveillance systems has increased due to security concerns and their relatively low cost. Researchers are working to create intelligent Closed Circuit Television (CCTV) cameras that can automatically analyze behavior in real-time to detect anomalous behaviors and prevent dangerous accidents. Deep Learning (DL) approaches, particularly Convolutional Neural Networks (CNNs), have shown outstanding results in video analysis and anomaly detection. This research paper focused on using Inception-v3 transfer learning approaches to improve the accuracy and efficiency of abnormal behavior detection in video surveillance. The Inception-v3 network is used to classify keyframes of a video as normal or abnormal behaviors by utilizing both pre-training and fine-tuning transfer learning approaches to extract features from the input data and develop a new classifier. The UCF-Crime dataset is used to train and evaluate the proposed models. The performance of both models was evaluated using accuracy, recall, precision, and F1 score. The fine-tuned model achieved 88.0%, 89.24%, 85.83%, and 87.50% for these measures, respectively. In contrast, the pre-trained model obtained 86.2%, 86.43%, 84.62%, and 85.52%, respectively. These results demonstrate that transfer learning using Inception-v3 architecture can effectively classify normal and abnormal behaviors in videos, and fine-tuning the weights of the layers can further improve the model's performance. Index Terms— Abnormal behavior detection, Video surveillance, Deep learning, Transfer learning, InceptionV3.

A novel hybrid feature extraction and ensemble C3D classification for anomaly detection in surveillance videos

Anomaly detection in several deep learning frameworks are recently presented on real-time video databases as a challenging task. However, these frameworks have high false positive rate (FPR) and error rate due to various backgrounds, motion appearance and semantic high-level and low-level features for anomaly detection through action classification. Also, extraction of features and classification are the major problems in traditional convolution neural network (CNN) on real-time video databases. The proposed work is a novel action classification framework which is designed and implemented on large video databases with high true positive rate (TPR) and error rate. In this framework, Kalman based incremental principal component analysis (IPCA) feature extraction method; C3D and non-linear support vector machine (SVM) classifier are used to improve the action prediction (anomaly detection) on the large real-time video databases. The proposed frame work shown new results of high computation performance than the traditional deep learning frameworks for action classification.

Boosting Variational Inference With Margin Learning for Few-Shot Scene-Adaptive Anomaly Detection

Anomaly detection in surveillance videos aims to identify frames where abnormal events happen. Existing approaches assume that the training and testing videos are from the same scene, exhibiting poor generalization performance when encountering an unseen scene. In this paper, we propose a Variational Anomaly Detection Network (VADNet), which is characterized by its high scene-adaptation - it can identify abnormal events in a new scene only via referring to a few normal samples without fine-tuning. Our model embodies two major innovations. First, a novel Variational Normal Inference (VNI) module is proposed to formulate image reconstruction in a conditional variational auto-encoder (CVAE) framework, which learns a probabilistic decision model instead of a traditional deterministic one. Secondly, a Margin Learning Embedding (MLE) module is leveraged to boost the variational inference and aid in distinguishing normal events. We theoretically demonstrate that minimizing the triplet loss in MLE module facilitates maximizing the evidence lower bound (ELBO) of CVAE, which promotes the convergence of VNI. By incorporating variational inference with margin learning, VADNet becomes much more generative that is able to handle the uncertainty caused by the changed scene and limited reference data. Extensive experiments on several datasets demonstrate that the proposed VADNet can adapt to a new scene effectively without fine-tuning and achieve remarkable performance, which outperforms other methods significantly and establishes new state-of-the-art in the case of few-shot scene-adaptive anomaly detection. We believe our method is closer to real-world application due to its strong generalization ability. All codes are released in https://github.com/huangxx156/VADNet.