Video Detection Research Articles

Revisiting vision-based violence detection in videos: A critical analysis

An ever-increasing installation of surveillance cameras at different places for ensuring public safety, security and asset protection has triggered the need for intelligent video surveillance to monitor the people and their behavior. Violence detection is a prominent application of intelligent surveillance as it plays a vital role in public safety, behavior monitoring, and law enforcement. It deals with identifying whether the violent event or behavior occurred in video sequence or not. Various researchers have developed different techniques and features for the detection of violence in recent years. The purpose of this paper is to provide an expository study of various state-of-the-art approaches for detecting violence in videos. It is evident from the ongoing effort in this field and with the advancement in computer vision technology, previous approaches get surpassed and new methods and features always keep on developing. Therefore, in order to complement ongoing research, it is also imperative to conduct a comprehensive analysis of different works from time to time. In this survey, each work has been critically analyzed, along with its pros and cons. The violence detection techniques have been divided under three categories: handcrafted features based, deep learning and hybrid violence detection approaches that have been extensively reviewed in various sub-categories. The major challenges faced by researchers and the steps involved in the process of violence detection are also discussed. Moreover, unlike typical practices of comparing the performance of models within same datasets, we examined a few models for cross-dataset assessment and revealed their limited generalization. This emphasizes the crucial need for robust model generalization to ensure efficacy across varied real-world scenarios. Finally, the paper carried out a discussion of various research gaps in the current approaches and the possible solutions to be taken to resolve them, laying a solid foundation for future work in this area.

Tiny drone object detection in videos guided by the bio-inspired magnocellular computation model

Detecting drones in infrared videos is highly desired in many realistic scenarios, e.g., unauthorized drone monitoring around airports. Nevertheless, automated drone detection is rather challenging when the targets appear as tiny objects (≤10×10 pixels) against complex backgrounds. Conventional object detection algorithms, which mainly use static visual features, can hardly distinguish tiny objects from undesired artefacts in complex backgrounds. To alleviate this problem, we learn from the early biological visual pathway (including the parvocellular and magnocellular pathways), which process static and motion information simultaneously. Therefore, we propose a magnocellular inspired method for video tiny-object detection (Magno-VTOD) that integrates both static and motion visual information. The Magno-VTOD firstly employs a retinal magnocellular computation model to extract the motion strength of moving objects. The motion responses are then used to enhance the areas of the flying tiny drones effectively and efficiently, thereby facilitating the subsequent target detection procedure. We implement the video tiny-object detection method based on the widely adopted deep neural networks guided by the magnocellular computation model. Experimental results obtained on the large-scale Anti-UAV dataset (304451 video frames) validate that the proposed Magno-VTOD method significantly outperforms the competing state-of-the-art object detection methods on the tiny drone detection task. Particularly, the AP value is increased by 15.4% for tiny object detection, and by 17.1%/13.7% against wood/mountain backgrounds.

Enhanced Video-Level Anomaly Feature Detection for Nuclear Power Plant Component Inspections Using the Latency Mechanism

The conditions of various nuclear power plant facilities are regularly examined through manual inspections. Remote visual inspection is commonly applied and requires engineers to watch lengthy inspection footage and seek anomaly features therein. This is a labor-intensive process as anomaly features of interest usually only appear in very short segments of the original whole video. Therefore, an automated anomaly detection system is preferred to lessen the intensive labor cost in the inspection process. The detection process could also benefit from useful information that could potentially contribute to addressing reasoning traceability. With a well-prepared training data set of the anomaly feature, a convolutional neural network (CNN) can be developed to automatically detect anomaly indications in the inspection video. However, false-positive detections may occur and can be difficult to remove without seeking manual verification. To overcome this problem, we present a new automated video-level anomaly detection framework that utilizes the latency mechanism to effectively lessen false-positive occurrences, and therefore, increase detection accuracy. In this framework, a CNN-based anomaly classifier first performs initial scanning of the anomaly type of interest in every region of the sampled frames. Then our latency mechanism is applied to refine the initial scanning results by flagging up a region as an “anomaly” indication only when “anomaly” is detected by CNN in the current frame and also in a sequence of previous consecutive frames of the same region. We present a case study of crack feature detection in superheater inspection videos to illustrate the performance of the proposed framework. The results show that the latency mechanism can effectively remove the original false-positive detections seen in the initial scanning. In order to provide a primary exploration of suggesting possible formats for addressing reasoning traceability, knowledge graphs of the reasoning process in the video-level detection framework are built to provide a better understanding of why a specific section of the video is flagged as anomaly contents by the video-level detection framework.

An Approach to Deepfake Video Detection Based on ACO-PSO Features and Deep Learning

The rapid advancement of deepfake technology presents significant challenges in detecting highly convincing fake videos, posing risks such as misinformation, identity theft, and privacy violations. In response, this paper proposes an innovative approach to deepfake video detection by integrating features derived from ant colony optimization–particle swarm optimization (ACO-PSO) and deep learning techniques. The proposed methodology leverages ACO-PSO features and deep learning models to enhance detection accuracy and robustness. Features from ACO-PSO are extracted from the spatial and temporal characteristics of video frames, capturing subtle patterns indicative of deepfake manipulation. These features are then used to train a deep learning classifier to automatically distinguish between authentic and deepfake videos. Extensive experiments using comparative datasets demonstrate the superiority of the proposed method in terms of detection accuracy, robustness to manipulation techniques, and generalization to unseen data. The computational efficiency of the approach is also analyzed, highlighting its practical feasibility for real-time applications. The findings revealed that the proposed method achieved an accuracy of 98.91% and an F1 score of 99.12%, indicating remarkable success in deepfake detection. The integration of ACO-PSO features and deep learning enables comprehensive analysis, bolstering precision and resilience in detecting deepfake content. This approach addresses the challenges involved in facial forgery detection and contributes to safeguarding digital media integrity amid misinformation and manipulation.

An improved mobile reinforcement learning for wrong actions detection in aerobics training videos

An improved mobile reinforcement learning for wrong actions detection in aerobics training videos

Novel Deep Learning Domain Adaptation Approach for Object Detection Using Semi-Self Building Dataset and Modified YOLOv4

Moving object detection is a vital research area that plays an essential role in intelligent transportation systems (ITSs) and various applications in computer vision. Recently, researchers have utilized convolutional neural networks (CNNs) to develop new techniques in object detection and recognition. However, with the increasing number of machine learning strategies used for object detection, there has been a growing need for large datasets with accurate ground truth used for the training, usually demanding their manual labeling. Moreover, most of these deep strategies are supervised and only applicable for specific scenes with large computational resources needed. Alternatively, other object detection techniques such as classical background subtraction need low computational resources and can be used with general scenes. In this paper, we propose a new a reliable semi-automatic method that combines a modified version of the detection-based CNN You Only Look Once V4 (YOLOv4) technique and background subtraction technique to perform an unsupervised object detection for surveillance videos. In this proposed strategy, background subtraction-based low-rank decomposition is applied firstly to extract the moving objects. Then, a clustering method is adopted to refine the background subtraction (BS) result. Finally, the refined results are used to fine-tune the modified YOLO v4 before using it in the detection and classification of objects. The main contribution of this work is a new detection framework that overcomes manual labeling and creates an automatic labeler that can replace manual labeling using motion information to supply labeled training data (background and foreground) directly from the detection video. Extensive experiments using real-world object monitoring benchmarks indicate that the suggested framework obtains a considerable increase in mAP compared to state-of-the-art results on both the CDnet 2014 and UA-DETRAC datasets.

Deepfake Video Detection Using Facial Feature Points and Ch-Transformer

With the development of Metaverse technology, the avatar in Metaverse has faced serious security and privacy concerns. Analyzing facial features to distinguish between genuine and manipulated facial videos holds significant research importance for ensuring the authenticity of characters in the virtual world and for mitigating discrimination, as well as preventing malicious use of facial data. To address this issue, the Facial Feature Points and Ch-Transformer (FFP-ChT) deepfake video detection model is designed based on the clues of different facial feature points distribution in real and fake videos and different displacement distances of real and fake facial feature points between frames. The face video input is first detected by the BlazeFace model, and the face detection results are fed into the FaceMesh model to extract 468 facial feature points. Then the Lucas-Kanade (LK) optical flow method is used to track the points of the face, the face calibration algorithm is introduced to re-calibrate the facial feature points, and the jitter displacement is calculated by tracking the facial feature points between frames. Finally, the Class-head (Ch) is designed in the transformer, and the facial feature points and facial feature point displacement are jointly classified through the Ch-Transformer model. In this way, the designed Ch-Transformer classifier is able to accurately and effectively identify deepfake videos. Experiments on open datasets clearly demonstrate the effectiveness and generalization capabilities of our approach.

Violence detection in compressed video

Violence detection in compressed video

Sequential Learning of Flame Objects Sorted by Size for Early Fire Detection in Surveillance Videos

Sequential Learning of Flame Objects Sorted by Size for Early Fire Detection in Surveillance Videos

Graph neural network based human detection in videos during occlusion environments

One of the most difficult perceptual problems for many applications is accurately recognizing the human object in a variety of circumstances. This can be difficult due to obstructions, weather, complex backdrops, cast shadows, and occlusions. Occlusion is a challenging open problem where a detector can only perceive a portion of the target human because of obstacles in the surrounding. In this research, an experimental investigation was conducted using the multi object tracking (MOT17) datasets to construct a graph neural network-based solution for the detection of humans in videos while considering the possibility of occlusion. Graph neural network (GNN) is used for the construction of neural solver model for detecting human object in occlusion scenario. The results obtained shows that this proposed method offers a considerable improvement in efficiency in comparison to the ways that have been used in the past. The values obtained for the standard performance metrics are higher than the state-of-the-art methods.

Bi-Source Reconstruction-Based Classification Network for Face Forgery Video Detection

Bi-Source Reconstruction-Based Classification Network for Face Forgery Video Detection

Violence detection in crowd videos using nuanced facial expression analysis

Video analysis for violence detection is crucial, especially when dealing with crowd data, where the potential for severe mob attacks in sensitive areas is high. This paper proposes a solution utilizing Convolutional Restricted Boltzmann Machine (CRBM) for video analysis, integrating the strengths of Convolutional Neural Network (CNN) and Restricted Boltzmann Machine (RBM). By focusing on image patches rather than entire frames, the method addresses the challenge of object detection in crowded scenes. The CRBM combines deep-level image analysis from CNN with unsupervised feature extraction in RBM, facilitated by image convolution using Gabor filters in the hidden layer. Dropout regularization mitigates overfitting, enhancing model generality. Extracted features are inputted into an SVM classifier for face detection and a custom VGG16 model for emotion identification. Event probability is then determined through logistic regression based on facial expressions. Despite existing approaches for smart crowd behaviour identification, there remains a tradeoff between accuracy and processing time. Our proposed solution addresses this by employing proper frame preprocessing techniques for feature extraction. Validation using quantitative and qualitative metrics confirms the effectiveness of the approach.

Deepfake Video Detection Using Res-Next CNN and LSTM

Abstract: Deepfake videos are becoming more and more common in the digital age, which has led to major worries about how they can undermine the trustworthiness of visual media. With deep learning algorithms' rising processing power, producing lifelike human-synthesized films or deepfakes has never been easier. Disinformation and political unrest can be caused by these videos. A novel deep learning-based method has been created to distinguish between actual and AI-generated fraudulent videos to address this problem. The suggested technique uses Attention-based networks (Res-Next CNN), a kind of deep learning architecture that can selectively focus on significant features in a video, to fine-tune the transformer module to search for new sets of feature space to detect false images.

Video Detection Method Based on Temporal and Spatial Foundations for Accurate Verification of Authenticity

With the rapid development of deepfake technology, it is finding applications in virtual movie production and entertainment. However, its potential for malicious use, such as generating false information, fake news, or synthetic pornography, poses significant threats to national and social security. Various research disciplines are actively engaged in developing deepfake video detection technologies to mitigate the risks associated with malicious deepfake content. Therefore, the importance of deepfake video detection technology cannot be overemphasized. This study addresses the challenge posed by images in nonexistent datasets by analyzing deepfake video detection methods. Using temporal and spatial detection techniques and employing 68 facial landmarks for alignment and feature extraction, this research integrates the attention-guided data augmentation (AGDA) strategy to enhance generalization capabilities. The detection performance is evaluated on four datasets: UADFV, FaceForensics++, Celeb-DF, and DFDC, with superior results compared to alternative approaches. To evaluate the study’s ability to accurately discriminate authenticity, detection experiments are conducted on both genuine and deepfake videos synthesized using the DeepFaceLab and FakeApp frameworks. The experimental results show better performance in detecting deepfake videos than other methods compared.

Deepfake video detection: challenges and opportunities

Deepfake videos are a growing social issue. These videos are manipulated by artificial intelligence (AI) techniques (especially deep learning), an emerging societal issue. Malicious individuals misuse deepfake technologies to spread false information, such as fake images, videos, and audio. The development of convincing fake content threatens politics, security, and privacy. The majority of deepfake video detection methods are data-driven. This survey paper aims to thoroughly analyse deepfake video generation and detection. The paper’s main contribution is the classification of the many challenges encountered while detecting deepfake videos. The paper discusses data challenges such as unbalanced datasets and inadequate labelled training data. Training challenges include the need for many computational resources. It also addresses reliability challenges, including overconfidence in detection methods and emerging manipulation approaches. The research emphasises the dominance of deep learning-based methods in detecting deepfakes despite their computational efficiency and generalisation limitations. However, it also acknowledges the drawbacks of these approaches, such as their limited computing efficiency and generalisation. The research also critically evaluates deepfake datasets, emphasising the necessity for good-quality datasets to improve detection methods. The study also indicates major research gaps, guiding future deepfake detection research. This entails developing robust models for real-time detection.

Enhancing Safety and Workover Efficiency Through Real-Time Scale-Adaptive Video Detection of Critical Objects at the Wellsite

Summary The wellsite serves as the fundamental unit in the development of oil and gas fields, functioning as a hub for the production activities, with workover operations being a critical means to ensure production continuity. In addition, it plays a crucial role in environmental protection, preventing oil and gas leakage and pollution. Various pieces of mechanical equipment deployed at the wellsite are essential for tasks such as oil and gas extraction and well repair operations, holding a pivotal position in oil- and gasfield development. Consequently, intelligent wellsite implementation necessitates a primary focus on monitoring mechanical equipment, with video emerging as a vital form of multisource information at the wellsite. While existing research on wellsite video monitoring predominantly addresses system and data transmission issues, it falls short in addressing the challenges of real-time assessment and early warning in intelligent wellsite operations. This study introduces a method for identifying critical targets at the wellsite based on a scale-adaptive network. The model employs a multiscale fusion network to extract different image features and semantic features at various scales, facilitating their fusion. The processing of wellsite video images occurs in multiple stages, outputting predicted box locations and category information, enabling the localization and recognition of critical objects at the wellsite. Unlike traditional deep convolutional object detection methods, this model incorporates a parameter-free attention mechanism, enhancing the accurate feature learning of small targets during the extraction process and addressing the issue of multiscale imbalance. The experimental results validate the robust performance of the method, surpassing the latest one-stage object detection models and mainstream loss function methods. Comparative experiments demonstrate a 9.22% improvement in mean average precision (mAP) compared with YOLOv8, establishing the proposed model as a top performer in loss function optimization experiments. Furthermore, we propose a video security detection model whose results, combined with the recognition model, are applicable for video detection in wellsite scenarios. The model exhibits strong integration capabilities for scene area division and behavior regulation monitoring. In addition, the model provides valuable insights for analyzing equipment operating status, aligning with the practical needs of oil fields.

Objective and automatic assessment approach for diagnosing attention-deficit/hyperactivity disorder based on skeleton detection and classification analysis in outpatient videos

BackgroundAttention-deficit/hyperactivity disorder (ADHD) is diagnosed in accordance with Diagnostic and Statistical Manual of Mental Disorders, Fifth Edition criteria by using subjective observations and information provided by parents and teachers. However, subjective analysis often leads to overdiagnosis or underdiagnosis. There are two types of motor abnormalities in patients with ADHD. First, hyperactivity with fidgeting and restlessness is the major diagnostic criterium for ADHD. Second, developmental coordination disorder characterized by deficits in the acquisition and execution of coordinated motor skills is not the major criterium for ADHD. In this study, a machine learning-based approach was proposed to evaluate and classify 96 patients into ADHD (48 patients, 26 males and 22 females, with mean age: 7y6m) and non-ADHD (48 patients, 26 males and 22 females, with mean age: 7y8m) objectively and automatically by quantifying their movements and evaluating the restlessness scales.MethodsThis approach is mainly based on movement quantization through analysis of variance in patients’ skeletons detected in outpatient videos. The patients’ skeleton sequence in the video was detected using OpenPose and then characterized using 11 values of feature descriptors. A classification analysis based on six machine learning classifiers was performed to evaluate and compare the discriminating power of different feature combinations.ResultsThe results revealed that compared with the non-ADHD group, the ADHD group had significantly larger means in all cases of single feature descriptors. The single feature descriptor “thigh angle”, with the values of 157.89 ± 32.81 and 15.37 ± 6.62 in ADHD and non-ADHD groups (p < 0.0001), achieved the best result (optimal cutoff, 42.39; accuracy, 91.03%; sensitivity, 90.25%; specificity, 91.86%; and AUC, 94.00%).ConclusionsThe proposed approach can be used to evaluate and classify patients into ADHD and non-ADHD objectively and automatically and can assist physicians in diagnosing ADHD.

Real-time sports injury monitoring system based on the deep learning algorithm

In response to the low real-time performance and accuracy of traditional sports injury monitoring, this article conducts research on a real-time injury monitoring system using the SVM model as an example. Video detection is performed to capture human movements, followed by human joint detection. Polynomial fitting analysis is used to extract joint motion patterns, and the average of training data is calculated as a reference point. The raw data is then normalized to adjust position and direction, and dimensionality reduction is achieved through singular value decomposition to enhance processing efficiency and model training speed. A support vector machine classifier is used to classify and identify the processed data. The experimental section monitors sports injuries and investigates the accuracy of the system’s monitoring. Compared to mainstream models such as Random Forest and Naive Bayes, the SVM utilized demonstrates good performance in accuracy, sensitivity, and specificity, reaching 94.2%, 92.5%, and 96.0% respectively.

A Dynamic Detection Method for Railway Slope Falling Rocks Based on the Gaussian Mixture Model Segmentation Algorithm

Rockfall intrusion detection is crucial for the safety management of railway operations, and video detection methods help reduce deployment costs and improve detection efficiency. Mainstream neural network-based video detection methods have rapidly evolved in recent years but struggle to adapt to complex scenarios such as existing railway slope constructions due to weak generalization ability, low accuracy, and limited information acquisition. Therefore, this paper introduces a dynamic neural network detection model and establishes a dataset for rockfall intrusions in existing railway slope scenarios. The model initially relies on the YOLOv5 neural network, adopting an activation function suitable for the target scenario and addressing overfitting to achieve precise target recognition. Based on the neural network, the model dynamically detects rolling rockfalls by integrating a background subtraction algorithm based on the Gaussian Mixture Model and captures target dimensions using monocular vision technology, thus broadening the dimensions of detection information. Trials conducted on a railway in Shandong, China, demonstrate that the model accurately identifies moving rockfalls along the railway slopes and acquires the dimensions of moving rockfalls, successfully filtering out low-risk targets in the scene.

An ensemble based approach for violence detection in videos using deep transfer learning

An ensemble based approach for violence detection in videos using deep transfer learning