Video Detection Research Articles

Two–Stage Detection and Localization of Inter–Frame Tampering in Surveillance Videos Using Texture and Optical Flow

Surveillance cameras provide security and protection through real-time monitoring or through the investigation of recorded videos. The authenticity of surveillance videos cannot be taken for granted, but tampering detection is challenging. Existing techniques face significant limitations, including restricted applicability, poor generalizability, and high computational complexity. This paper presents a robust detection system to meet the challenges of frame duplication (FD) and frame insertion (FI) detection in surveillance videos. The system leverages the alterations in texture patterns and optical flow between consecutive frames and works in two stages; first, suspicious tampered videos are detected using motion residual–based local binary patterns (MR–LBPs) and SVM; second, by eliminating false positives, the precise tampering location is determined using the consistency in the aggregation of optical flow and the variance in MR–LBPs. The system is extensively evaluated on a large COMSATS Structured Video Tampering Evaluation Dataset (CSVTED) comprising challenging videos with varying quality of tampering and complexity levels and cross–validated on benchmark public domain datasets. The system exhibits outstanding performance, achieving 99.5% accuracy in detecting and pinpointing tampered regions. It ensures the generalization and wide applicability of the system while maintaining computational efficiency.

Enhanced object detection in videos using bio vision

Video surveillance systems enhance security and monitor various environments. They act as a deterrent to potential threats and provide vital surveillance to maintain safety. A key component of these systems is object detection, which identifies objects of interest and generates critical data for tracking and analysis. However, detecting objects in low-light conditions is particularly challenging due to reduced visibility, low contrast, and shadows, which can obscure objects or cause them to blend into the background. To address these challenges, this research introduces an enhanced object detection model that combines bio-inspired vision techniques with traditional object detection methods. The YOLO model is augmented with bio-vision principles, and specialized CNN models are developed to improve detection accuracy, particularly in low-light scenarios.

Real-Time Smoke Detection in Surveillance Videos Using an Enhanced RT-DETR Framework with Triplet Attention and HS-FPN

This study addresses the urgent need for an efficient and accurate smoke detection system to enhance safety measures in fire monitoring, industrial safety, and urban surveillance. Given the complexity of detecting smoke in diverse environments and under real-time constraints, our research aims to solve challenges related to low-resolution imagery, limited computational resources, and environmental variability. This study introduces a novel smoke detection system that utilizes the real-time detection Transformer (RT-DETR) architecture to enhance the speed and precision of video analysis. Our system integrates advanced modules, including triplet attention, ADown, and a high-level screening-feature fusion pyramid network (HS-FPN), to address challenges related to low-resolution imagery, real-time processing constraints, and environmental variability. The triplet attention mechanism is essential for detecting subtle smoke features, often overlooked due to their nuanced nature. The ADown module significantly reduces computational complexity, enabling real-time operation on devices with limited resources. Furthermore, the HS-FPN enhances the system’s robustness by amalgamating multi-scale features for reliable detection across various smoke types and sizes. Evaluation using a diverse dataset showcased notable improvements in average precision (AP50) and frames per second (FPS) metrics compared to existing state-of-the-art networks. Ablation studies validated the contributions of each component in achieving an optimal balance between accuracy and operational efficiency. The RT-DETR-based smoke detection system not only meets real-time requirements for applications like fire monitoring, industrial safety, and urban surveillance but also establishes a new performance benchmark in this field.

Optimizing Football Formation Analysis via LSTM-Based Event Detection

The process of manually annotating sports footage is a demanding one. In American football alone, coaches spend thousands of hours reviewing and analyzing videos each season. We aim to automate this process by developing a system that generates comprehensive statistical reports from full-length football game videos. Having previously demonstrated the proof of concept for our system, here, we present optimizations to our preprocessing techniques along with an inventive method for multi-person event detection in sports videos. Employing a long short-term memory (LSTM)-based architecture to detect the snap in American football, we achieve an outstanding LSI (Levenshtein similarity index) of 0.9445, suggesting a normalized difference of less than 0.06 between predictions and ground truth labels. We also illustrate the utility of snap detection as a means of identifying the offensive players’ assuming of formation. Our results exhibit not only the success of our unique approach and underlying optimizations but also the potential for continued robustness as we pursue the development of our remaining system components.

Hybrid Deep-Learning Model for Deepfake Detection in Video using Transfer Learning Approach

Hybrid Deep-Learning Model for Deepfake Detection in Video using Transfer Learning Approach

Toward comprehensive short utterances manipulations detection in videos

AbstractIn a landscape increasingly populated by convincing yet deceptive multimedia content generated through generative adversarial networks, there exists a significant challenge for both human interpretation and machine learning algorithms. This study introduces a shallow learning technique specifically tailored for analyzing visual and auditory components in videos, targeting the lower face region. Our method is optimized for ultra-short video segments (200-600 ms) and employs wavelet scattering transforms for audio and discrete cosine transforms for video. Unlike many approaches, our method excels at these short durations and scales efficiently to longer segments. Experimental results demonstrate high accuracy, achieving 96.83% for 600 ms audio segments and 99.87% for whole video sequences on the FakeAVCeleb and DeepfakeTIMIT datasets. This approach is computationally efficient, making it suitable for real-world applications with constrained resources. The paper also explores the unique challenges of detecting deepfakes in ultra-short sequences and proposes a targeted evaluation strategy for these conditions.

Temporally enhanced abnormal behavior detection based on multi-channel coupling

Abnormal behavior detection in surveillance videos based on deep learning has becomea hot topic in current research. However, due to the complexity and variability of crowd movement and external environment, detecting abnormal behavior faces great challenges. Current abnormal behavior recognition models have limitations in feature extraction and pay insufficient attention to dynamic temporal features. To address these problems, a spatiotemporal enhancement anomaly detection method based on multi-channel coupling is proposed. Based on the SlowFast network, a multi-channel coupled temporal enhancement module and a spatial enhancement module are introduced respectively to extract more discriminative static and dynamic feature information. A large number of experiments are carried out on three benchmark datasets (Violent Flow, Hockey Fight, and Real-life Violence Situations). The results show that the prediction accuracy of the proposed abnormal behavior recognition method reaches 95.3%、97.3%and respectively 94%, thus verifying the effectiveness of the method.

A Novel DC GCN with Attention Mechanism for Accurate Near Duplicate Video Data Cleaning

There has been a steady emergence of nearly identical recordings in the last several decades, thanks to the exponential development of video data. The use of regular videos has been impacted by data quality difficulties produced by near-duplicate movies, which are becoming increasingly noticeable. While there has been progress in the field of near-duplicate video detection, there is still no automated merging method for video data characterised by high-dimensional features. As a result, it is challenging to automatically clean near-duplicate videos in advance video dataset data quality. Research on removing near-duplicate video data is still in its early stages. The precision of near-duplicate video data cleaning is severely compromised by the delicate issues of video data organization besides initial clustering centres in the current research, which arise when the previous distribution is unknown. In tackle these problems, we offer a new kind of Graph Convolutional Neural Network (GCN) that uses dense influences and a categorization attention mechanism. Deeply connected graph convolutional networks (DC-GCNs) learn about faraway nodes by making GCNs deeper. By using dense connections, the DC-GCN is able to multiplex the small-scale features of shallow layers and generate features at diverse scales. Finally, an attention mechanism is incorporated to aid in feature combination and importance determination. Sparrow Search Optimisation Algorithm (SSA) is used to pick the parameters of the given model in the most optimal way. In the end, experiments are carried out using a coal mining video dataset and a widely known dataset called CC_WEB_VIDEO. The simulation findings show that the suggested strategy performs better than certain previous studies.

Abnormal detection in nuclear security videos based on label-specific autoencoders and reconstruction errors comparison

Abnormal detection in surveillance cameras never fails to be a crucial and challenging task across many fields, aimed at protecting the property safety of citizens and society from abnormal events. Currently, the most common solution for abnormal detection proves to be the combination of autoencoder models and unsupervised training scheme. Due to the scarcity of abnormal samples, all training samples are typically treated as normal samples when constructing the autoencoder calculation models, which generate reconstructed video frames with the same dimension as input data. In this approach, the identification of testing samples is determined by calculating the reconstruction errors between raw and reconstructed video frames. However, such calculation frameworks turn out to be unpractical for nuclear security, as the threshold value of reconstruction errors is overly sensitive, often leading to extreme and unacceptable results. In this manuscript, a novel framework for abnormal detection in nuclear security is proposed. Unlike common unsupervised training scheme, this approach constructs two label-specific autoencoders, while one is trained exclusively with normal samples and the other one is only fed with abnormal samples. Therefore, there is no need to predefine the threshold value for reconstruction errors, as it is replaced by comparing the two label-specific reconstruction errors. Besides, two training schemes for the label-specific autoencoders are proposed and analyzed. Based on a self-collected dataset for nuclear security, the results show that the proposed framework is both practical and appliable, with both training schemes achieving an acceptable accuracy of 0.8182.

Balancing Accuracy and Training Time in Federated Learning for Violence Detection in Surveillance Videos: A Study of Neural Network Architectures

Balancing Accuracy and Training Time in Federated Learning for Violence Detection in Surveillance Videos: A Study of Neural Network Architectures

Extremely compact video representation for efficient near-duplicates detection

With the influx of online video uploads, a robust method for detecting video copies under various distortions is essential for copyright protection and search efficiency. We propose a compact video representation that uses a Siamese Neural Network for near-duplicate detection. This approach achieves up to 0.998 recall and 0.853 precision, even with distorted 56x56px miniatures. Consequently, we derive extremely compact video descriptors, facilitating video fragment retrieval in large datasets. Our method pre-selects frames from a video by identifying local maximums from the interframe differences curve, preserving characteristic sequence patterns even after extreme compression. Utilizing a simpler convolution-based model in the Siamese Neural Network, we improve results by up to 8% over VGG-16, with a 1.7-fold reduction in inference time. With compact descriptors of 1.875 kB, our models outperform others by more than two times. Additionally, we introduce a new dataset with dynamically changing scenes, enhancing its suitability for near-duplicate video detection.

Safeguards-related event detection in surveillance video using semi-supervised learning approach

We develop a deep learning model employing a semi-supervised learning approach, which can detect automatically safeguards-related events in nuclear facility from surveillance video. Our model is designed after a comprehensive analysis of the trends in artificial intelligence-based models to identify abnormal events in video. Our model incorporates a reconstruction module and a prediction module independently. The reconstruction module is trained to generate video frames within a sliding window, while the prediction module is trained to predict future motion feature based on the motion features within the video frames in a sliding window. Each module utilizes an autoencoder with a memory module positioned between an encoder and an decoder of the autoencoder. We evaluate the model's performance using a benchmark dataset and a self-produced dataset obtained from facility related to pyroprocessing. Our model's performanace is comparable to or superior to that of the prevous models from the benchmark dataset analysis, and all the abnormal events can be detected without false positive error from the self-produced dataset analysis.

Computer vision that can ‘see’ in the dark

Insufficient lighting environment has raised challenges for night shift workers’ safety monitoring. Thus, we have developed a computer vision-based algorithm recognizing 11 actions based on action recognition in dark (ARID) dataset. A hybrid model of integrating convolutional neural network (CNN) into YOLOv7 has been proposed. YOLOv7 is an algorithm designed for real-time object detection in image or video, for fast and accurate detection in applications such as autonomous vehicles and surveillance systems. In this work, video in dark environment has first been enhanced using CNN algorithm before feeding into YOLOv7 network for activity recognition. Adaptive gamma intensity correction (GIC) has been integrated to further improving the overall result. The proposed model has been evaluated over different enhancement modes. The proposed model is able to handle dark video frames with 74.95% Top-1 accuracy with fast processing speed of 93.99 ms/frame on a 4 GB RTX 3050 graphical processing unit (GPU) and 17.59 ms/frame on 16 GB Tesla T4 GPU. The base size of the proposed model is tiny, only 74.8 MB, but with 36.54 M of total parameters indicating that it has more capacity to learn more meaningful information with limited hardware resources.

Transformer-Based Spatio-Temporal Unsupervised Traffic Anomaly Detection in Aerial Videos

Transformer-Based Spatio-Temporal Unsupervised Traffic Anomaly Detection in Aerial Videos

Crime Activity Detection in Surveillance Videos Based on Developed Deep Learning Approach

In modern communities, lots of offenders are prone to recidivism, hence, there is a requirement to inhibit such criminals, especially from impending socioeconomically disadvantaged and high-crime areas that experience elevated levels of criminal activity, involving drug-related offenses, violence, theft, and other forms of anti-social behavior. Consequently, surveillance cameras have been installed in relevant institutions, and further personnel have been provided to monitor videos using various surveillance apparatus. However, relying solely on monitoring with the naked eye and manual video processing falls short of accurately evaluating the footage acquired via such cameras. To handle the issues of conventional systems, there is a need for a system that is able to classify acquired images while supporting surveillance personnel actively. Therefore, in this paper, a deep-learning approach is developed to build a crime detection system. This developed approach includes various layers necessary to perform feature extraction and classification processes and make the system capable of efficiently and accurately detecting crime activities from surveillance video frames. Besides the proposed crime activity detection system, two deep-learning approaches (EfficientNet-B7, and MobileNet-V2) are trained and assessed on the popular UCF Crime and DCSASS datasets. Generally, the proposed detection system encompasses dataset preparation and pre-processing, splitting the pre-processed crime activity image dataset, and implementing the proposed deep learning approach and other pre-trained approaches.

Local Region Frequency Guided Dynamic Inconsistency Network for Deepfake Video Detection

Local Region Frequency Guided Dynamic Inconsistency Network for Deepfake Video Detection

Video Analytic for Human Management and Security and FPGA Accelerated High Concurrency

This paper explores the use of video analytics by leveraging accelerated FPGA technology in combination with high-performance computing, specifically utilizing two Xilinx Alveo U50Lv cards and one U55C card. While many applications exist for motion analysis and detection in videos, the use of FPGAs in this context remains relatively scarce. FPGAs offer significant advantages in terms of energy efficiency and throughput. We present results demonstrating the parallelism capabilities in terms of the number of threads within a single Docker container that shares stack memory, as well as across multiple Docker containers. When operating within a single Docker process, the application shares the same memory space and resources, making it ideal for tasks that require efficient communication or data sharing. In contrast, running in multiple containers isolates processes, each with its own environment, and can significantly increase the number of threads. Our findings show that the combination of these techniques offers optimal performance for video analytics.

Fake video detection among secondary school students: The impact of sociocultural, media literacy and media use factors

The harmful political, economic and health effects of fake news on social media are well known. The present study examines the impact of two socio-cultural variables (political orientation and parents' educational attainment), one media literacy variable and two media use variables (news control, platform usage frequency and media addiction) on fake videos detection. The ability to detect fake videos among Hungarian secondary school students (N = 507) was assessed using a 16-item video test. The results of the online survey are partly consistent and partly contradictory to the literature. There is no gender difference in the ability to detect fake videos in the age group studied, and media literacy and media use do not influence the ability to detect fake videos. However, a more conservative worldview and higher parental education are associated with better detection of fake videos. The paper concludes with recommendations.

Transformer and Adaptive Threshold Sliding Window for Improving Violence Detection in Videos.

This paper presents a comprehensive approach to detect violent events in videos by combining CrimeNet, a Vision Transformer (ViT) model with structured neural learning and adversarial regularization, with an adaptive threshold sliding window model based on the Transformer architecture. CrimeNet demonstrates exceptional performance on all datasets (XD-Violence, UCF-Crime, NTU-CCTV Fights, UBI-Fights, Real Life Violence Situations, MediEval, RWF-2000, Hockey Fights, Violent Flows, Surveillance Camera Fights, and Movies Fight), achieving high AUC ROC and AUC PR values (up to 99% and 100%, respectively). However, the generalization of CrimeNet to cross-dataset experiments posed some problems, resulting in a 20-30% decrease in performance, for instance, training in UCF-Crime and testing in XD-Violence resulted in 70.20% in AUC ROC. The sliding window model with adaptive thresholding effectively solves these problems by automatically adjusting the violence detection threshold, resulting in a substantial improvement in detection accuracy. By applying the sliding window model as post-processing to CrimeNet results, we were able to improve detection accuracy by 10% to 15% in cross-dataset experiments. Future lines of research include improving generalization, addressing data imbalance, exploring multimodal representations, testing in real-world applications, and extending the approach to complex human interactions.

Combating deepfakes: a comprehensive multilayer deepfake video detection framework

AbstractDeepfakes represent a class of synthetic media crafted with the aid of advanced deep learning techniques that exhibit an unparalleled degree of authenticity. The rapid advancement in Artificial Intelligence (AI) has contributed to an increase in the prevalence of deepfakes on the internet, consequently amplifying the spread of misinformation among the public. Consequently, the detection of deepfakes has become a pressing concern. In this context, we put forth a comprehensive framework for deepfake video detection, which is built upon three distinct layers. The first layer, termed as the RGB features extraction layer, is designed to identify potential signs of forgery within the spatial domain of analogous video frames. The second layer, known as the GAN features extraction layer, focuses on the extraction of forgery fingerprints in the high-frequency region. This layer is specifically engineered to detect the fingerprints left by the Generative Adversarial Network (GAN) process in fake videos and the traces of the imaging process in genuine videos. The third and final layer, referred to as the facial region intra-frame inconsistency feature extraction layer, is dedicated to uncovering the anomalies associated with the manipulation process. This is achieved by extracting features from both the inner and outer regions of the manipulated portion of a frame. The extensive experimental evaluations have underscored the superior performance of proposed approach in comparison to existing state-of-the-art methods.