Video Detection System Research Articles

Toward trustworthy human suspicious activity detection from surveillance videos using deep learning

AbstractIn today’s world, suspicious or unusual activities express threats and danger to others. For the prevention of various security issues, an automatic video detection system is very important. It is difficult to consecutively monitor camera videos recorded in public places to detect any abnormal event, so an automated video detection system is needed. The study objective is to create an intelligent and trustworthy system that will take a video stream as input and detect what kind of suspicious activity is happening in that video to reduce the time that consumes watching the video. In this work, we use three models Convolutional Neural Network (CNN), GRU, and ConvLSTM model. These models are trained on the same dataset of 6 suspicious activities of humans that are: Running, Punching, Falling, Snatching, Kicking, and Shooting. The dataset consists of various videos related to each activity. Different deep learning techniques are applied in the proposed work: preprocessing, data annotation model training, and classification. The frames are extracted from the source video, and then features are calculated through the model in Inception v3, a Convolutional Neural Network variant. On the same dataset, the CNN model attains 91.55% accuracy, the ConvLSTM model attains 88.73% accuracy, and the GRU model attains 84.01% accuracy. The performance of the proposed models is evaluated using a confusion matrix, F1-Score, precision, and recall. The proposed model proved better than other models in terms of performance and accuracy. The findings of this study prove helpful unusual events by examining a person's abnormal behavior.

Estimation of Arrivals on Green at Signalized Intersections Using Stop-Bar Video Detection

Across the world, traffic congestion is increasing with alarming rapidity. Traffic signal control effectiveness, in coordinated networks, is often investigated in relation to the type of vehicle arrivals at the signalized intersections. Recently, several transportation agencies have switched from traditional loop detectors to video detection. When video cameras are accompanied by computer vision, one can extract more information about traffic “dynamics” than by using traditional inductive loop detectors. Collecting arrival times of multiple vehicles after the first arrival at the stop-bar detector might be challenging when using inductive loop detectors (since after the first arrival, detector status is always occupied). However, emerging video detection systems allow tracking of each vehicle’s entrance time in the detection zone, departure time from the detection zone, and the type of vehicle. This information can be used to estimate vehicular arrival and departure times, which then can be fed into machine learning algorithms to estimate arrivals on green (AOG). However, such research ideas have not been documented so far. Thus, this paper presents an estimation model for AOG, which was developed using multigene genetic programming. A robust experimental dataset was collected from a highly calibrated and validated microsimulation model of an 11-intersection corridor in Chattanooga, TN. The results of the model’s performance analysis showed the high accuracy of the training-, testing-, and validation datasets. The practical benefit of this model is that it can be applied to estimate arrival types at intersections where only stop-bar video detection exists.

Real-Time Forest Fire Detection by Ensemble Lightweight YOLOX-L and Defogging Method.

Forest fires can destroy forest and inflict great damage to the ecosystem. Fortunately, forest fire detection with video has achieved remarkable results in enabling timely and accurate fire warnings. However, the traditional forest fire detection method relies heavily on artificially designed features; CNN-based methods require a large number of parameters. In addition, forest fire detection is easily disturbed by fog. To solve these issues, a lightweight YOLOX-L and defogging algorithm-based forest fire detection method, GXLD, is proposed. GXLD uses the dark channel prior to defog the image to obtain a fog-free image. After the lightweight improvement of YOLOX-L by GhostNet, depth separable convolution, and SENet, we obtain the YOLOX-L-Light and use it to detect the forest fire in the fog-free image. To evaluate the performance of YOLOX-L-Light and GXLD, mean average precision (mAP) was used to evaluate the detection accuracy, and network parameters were used to evaluate the lightweight effect. Experiments on our forest fire dataset show that the number of the parameters of YOLOX-L-Light decreased by 92.6%, and the mAP increased by 1.96%. The mAP of GXLD is 87.47%, which is 2.46% higher than that of YOLOX-L; and the average fps of GXLD is 26.33 when the input image size is 1280 × 720. Even in a foggy environment, the GXLD can detect a forest fire in real time with a high accuracy, target confidence, and target integrity. This research proposes a lightweight forest fire detection method (GXLD) with fog removal. Therefore, GXLD can detect a forest fire with a high accuracy in real time. The proposed GXLD has the advantages of defogging, a high target confidence, and a high target integrity, which makes it more suitable for the development of a modern forest fire video detection system.

An Energy Efficient Task Scheduling for a Cloud-edge System

With the widespread popularity of edge computing, the energy consumption of edge devices has attracted lots of research interests. This paper studies the cloud-edge system under the solar energy supply scenario, and optimizes the system parameter configuration according to the energy supply of photovoltaic power generation, to reduce the system energy consumption and prolong the system running time. Specifically, we built an edge video detection system based on Raspberry Pi, and deeply analyzed the energy consumption model of the system. Based on this, the problem of dynamically adjusting the video detection operating parameters is modeled as a long-term optimization problem. In order to solve this problem effectively, the long-term optimization problem is transformed to an instantaneous optimization problem, so that the problem can be solved efficiently in real time, and the rationality of this transformation is proved. Finally, we propose a low-energy-consumption scheduling algorithm to optimize the task scheduling and energy consumption of the system. Through simulation experiments, the performance of the proposed algorithm is evaluated. The results show that the scheduling algorithm can reduce the energy consumption of the system by an average of 18%.

A Novel Smart Deepfake Video Detection System

Rapid advancements in deep learning-based technologies have developed several synthetic video and audio generation methods producing incredibly hyper-realistic deepfakes. These deepfakes can be employed to impersonate the identity of a source person in videos by swapping the source's face with the target one. Deepfakes can also be used to clone the voice of a target person utilizing audio samples. Such deepfakes may pose a threat to societies if they are utilized maliciously. Consequently, distinguishing either one or both deepfake visual video frames and cloned voices from genuine ones has become an urgent issue. This work presents a novel smart deepfake video detection system. The video frames and audio are extracted from given videos. Two feature extraction methods are proposed, one for each modality; visual video frames, and audio. The first method is an upgraded XceptionNet model, which is utilized for extracting spatial features from video frames. It produces feature representation for visual video frames. The second one is a modified InceptionResNetV2 model based on the Constant-Q Transform (CQT) method. It is employed to extract deep time-frequency features from the audio modality. It produces feature representation for the audio. The corresponding extracted features of both modalities are fused at a mid-layer to produce a bimodal information-based feature representation for the whole video. These three representation levels are independently fed into the Gated Recurrent Unit (GRU) based attention mechanism helping to learn and extract deep and important temporal information per level. Then, the system checks whether the forgery is only applied to video frames, audio, or both, and produces the final decision about video authenticity. The newly suggested method has been evaluated on the FakeAVCeleb multimodal videos dataset. The experimental results analysis assures the superiority of the new method over the current–state-of-the-art methods.

Human Violence Detection Using Deep Learning Techniques

The world’s average annual fatality rate from human violence is 7.9 per 10,000 people. Most of this human violence takes place in an isolated area or of sudden. The information delay here is a major impediment in stopping these acts. To thrive on this issue, the detection technique is used in this study. Detecting moving objects from CCTV is one of the most effective computer vision algorithms. CCTV cameras are now in every streets which are extremely helpful in solving cases. Some techniques of deep learning are used as computer vision to predict and detect the action, properties from video. In real-time police reach violent destinations and start checking CCTV cameras, and investigate to proceed further. This study is deliberately designed to detect violent acts from CCTV cameras. The Inception – v3 and Yolo – v5 models detect the violent act, the number of persons involved, and also the weapons used in the situation. The study consists of these deep learning models, which are used to form a video detection system. This model can be used in real-time as an application programming interface (API) or software. The study results showed the proposed model achieves an accuracy of 74%.

Toward Efficient and Adaptive Design of Video Detection System with Deep Neural Networks

In the past decade, Deep Neural Networks (DNNs), e.g., Convolutional Neural Networks, achieved human-level performance in vision tasks such as object classification and detection. However, DNNs are known to be computationally expensive and thus hard to be deployed in real-time and edge applications. Many previous works have focused on DNN model compression to obtain smaller parameter sizes and consequently, less computational cost. Such methods, however, often introduce noticeable accuracy degradation. In this work, we optimize a state-of-the-art DNN-based video detection framework—Deep Feature Flow (DFF) from the cloud end using three proposed ideas. First, we propose Asynchronous DFF (ADFF) to asynchronously execute the neural networks. Second, we propose a Video-based Dynamic Scheduling (VDS) method that decides the detection frequency based on the magnitude of movement between video frames. Last, we propose Spatial Sparsity Inference, which only performs the inference on part of the video frame and thus reduces the computation cost. According to our experimental results, ADFF can reduce the bottleneck latency from 89 to 19 ms. VDS increases the detection accuracy by 0.6% mAP without increasing computation cost. And SSI further saves 0.2 ms with a 0.6% mAP degradation of detection accuracy.

A New Method for Surface-to-Air Video Detection and Tracking of Airborne Vehicles

Detection process of airborne targets may be thought simple because of the incompatible nature of aircraft, choppers, UAVs, and drones regarding clear sky background. When changes in the background are considered, brightness variation of the sky complicates the process. Changes in the shapes and types of clouds add another challenge to the process. Tracking process directly depends on the detection process and type of the data stream. The practical systems used for video detection and tracking of airborne targets are manual, and manual structures have some drawbacks compared to automatic structures. For video surveillance, guidance, regional security, and defense applications in dense environments, automatic detection and tracking process may be an obligation rather than preference. In this study, an automatic detection and tracking algorithm for video streams of airborne targets is proposed. A land-based moving camera captures the video data, and not only the flying objects but probably also the camera are in motion. Although the detection and tracking of moving objects via moving sensors is a relatively arduous task, this is the prevalent case in real-life scenarios. Video detection and tracking systems have one or more moving video sensors, while one or more flying air vehicles are in operation area. The proposed algorithm includes an image processing stage for detection and a tracking stage for initiation and continuation. An assessment study has been conducted for the actual video data and found that the proposed method yields successful results for detection, track formation, and continuation processes.

Natural language processing based advanced method of unnecessary video detection

<span>In this study we have described the process of identifying unnecessary video using an advanced combined method of natural language processing and machine learning. The system also includes a framework that contains analytics databases and which helps to find statistical accuracy and can detect, accept or reject unnecessary and unethical video content. In our video detection system, we extract text data from video content in two steps, first from video to MPEG-1 audio layer 3 (MP3) and then from MP3 to WAV format. We have used the text part of natural language processing to analyze and prepare the data set. We use both Naive Bayes and logistic regression classification algorithms in this detection system to determine the best accuracy for our system. In our research, our video MP4 data has converted to plain text data using the python advance library function. This brief study discusses the identification of unauthorized, unsocial, unnecessary, unfinished, and malicious videos when using oral video record data. By analyzing our data sets through this advanced model, we can decide which videos should be accepted or rejected for the further actions.</span>

Implementation of an Intelligent Video Detection System using Deep Learning in the Manufacturing Process of Tungsten Hexafluoride

Implementation of an Intelligent Video Detection System using Deep Learning in the Manufacturing Process of Tungsten Hexafluoride

Computer vision-based approach for smart traffic condition assessment at the railroad grade crossing

Slow-moving or stopped trains at highway-railroad grade crossings, especially in the populated metropolitan areas, not only cause significant traffic delays to commuters, but also prevent first responders from timely responding to emergencies. In this study, the researchers introduce an automated video analysis, detection and tracking system to evaluate the traffic conditions, analyze blocked vehicle behaviors at grade crossings, and predict the decongestion time under a simplified scenario. A novel YOLOv3-SPP+ model has been developed to improve the detection performance with dividing the image from finer to coarser levels and enhance local features. The SORT module has been integrated to the model for a simple yet efficient manner to track vehicles at the railroad grade crossing. Two field datasets at the Columbia, SC, with train blockage video records have been tested. The model training performance has been evaluated by mAP @0.5, F1 score, and total loss. Based on the training results, our model outperforms other YOLO series models. The field tracking performance has been assessed by the ratio between prediction and ground truth. The mean value of accuracy of our test cases is 92.37%, indicating a reliable tracking performance. In addition, the present results indicate the traffic during and after the crossing blockage does follow a pattern, and there is a general trend of the behavior of the vehicles waiting or taking an alternative route. A good linear correlation between the decongestion time and the number of blocked vehicles has been observed at the monitored grade crossing at the City of Columbia, SC.

Urban dual mode video detection system based on fisheye and PTZ cameras

This work presents an artificial vision-based monitoring system for urban environments. It comprises a fisheye camera monitoring the scenes 180ox360o hemisphere and a Pan-Tilt-Zoom camera capturing narrower regions of interest in high-resolution. The ONVIF protocol standard is used to interface both IP-cameras, allowing for the integration of camera control (image acquisition and movement) and geometric calculations on a single device. The events of interest (motion of vehicles and pedestrians) are assumed to happen on the ground plane. This assumption is required to solve the back-projection, the function that maps coordinates in the highly distorted images of the fisheye camera to the ground plane. A calibration strategy estimates the poses of the cameras without placing restrictions on their orientations or relative distance. It optimizes the back-projection error in the ground plane instead of the re-projection error in the image. Finally, a simple pointing and zoom adjustment strategy controls the Pan-Tilt-Zoom camera. The system is tested in controlled laboratory conditions and shows accurate outdoor performance for pedestrian observation.

Internet of Things Based Indoor Smart Surveillance and Monitoring System using Arduino and Raspberry Pi

Security now constitutes a big concern for a day & is rising fast every day. Save your house, records, money & other valuable things like jewellery is vital to all. The methodologies used by offenders have now strengthened with the advancement of very recent technology. Therefore, the appropriate monitoring strategies for global transformation are very critical to develop. Video detection and tracking system are the latest and powerful technologies used against burglary and stealing. However, some individuals can not bear the cost of building and running such devices. A new and efficient system has been suggested for motion detection rather than applying different complex algorithms. When the PIR sensor senses the action, it records the image with the camera & sends it to the user. The cost of construction can be that, and the energy-efficient program can be used with this approach.

Effective and Efficient Content Redundancy Detection of Web Videos

Currently, an unprecedentedly vast amount of videos are hosted on the Internet and shared by users across the world. Within these videos, a considerable portion is duplicate or near-duplicate. Consequently, building an effective yet efficient content-based redundancy detection system is of importance, as this research would be beneficial to a variety of applications. Despite the progress in this field, designing a practical detection system for web videos continues to be difficult, because of the contradictions between the accuracy and speed requirements. In this paper, we propose a novel near-duplicate video detection system, CompoundEyes, whose design philosophy deviates from the conventional feature-centered paradigm. Instead, the focus of our system has been shifted from the design of an advanced feature representation to the design of system architecture. This design methodology not only ensures a decent detection accuracy by the collaboration of the classifiers but also substantially accelerates the detection speed due to the low dimensionality of the feature representations and the exploitation of the parallelism among the components. Experiments have been conducted to demonstrate that the CompoundEyes is both accurate and fast.

An infrared video detection and categorization system based on machine learning

An infrared video detection and categorization system based on machine learning

Using Adversarial Learning and Biterm Topic Model for an Effective Fake News Video Detection System on Heterogeneous Topics and Short Texts

Fake news videos are being actively produced and uploaded on YouTube to attract public attention. In this paper, we propose a topic-agnostic fake news video detection model based on adversarial learning and topic modeling. The proposed model estimates the topic distribution of a video using its title/description and comments by topic modeling and tries to identify the differences in stance by the topic distribution difference between title/description and comments. Then, it constructs an adversarial neural network to extract topic-agnostic features effectively. The proposed model can effectively detect topic changes for stance analysis and easily shifts among various topics. In this study, it achieves a 2.68%p greater F1-score than previous models used for fake news video detection.

A Recognition Method for Rice Plant Diseases and Pests Video Detection Based on Deep Convolutional Neural Network.

Increasing grain production is essential to those areas where food is scarce. Increasing grain production by controlling crop diseases and pests in time should be effective. To construct video detection system for plant diseases and pests, and to build a real-time crop diseases and pests video detection system in the future, a deep learning-based video detection architecture with a custom backbone was proposed for detecting plant diseases and pests in videos. We first transformed the video into still frame, then sent the frame to the still-image detector for detection, and finally synthesized the frames into video. In the still-image detector, we used faster-RCNN as the framework. We used image-training models to detect relatively blurry videos. Additionally, a set of video-based evaluation metrics based on a machine learning classifier was proposed, which reflected the quality of video detection effectively in the experiments. Experiments showed that our system with the custom backbone was more suitable for detection of the untrained rice videos than VGG16, ResNet-50, ResNet-101 backbone system and YOLOv3 with our experimental environment.

Effect of Video Detection System Layout under Covering and Differentiating Route Flow Principles

AbstractVideo detection system (VDS) sensors play an important role in monitoring traffic flows. Because of limited numbers of camera installations, there is a dilemma in positioning camera sensors...

Key Algorithms of Video Target Detection and Recognition in Intelligent Transportation Systems

With the popularization of video detection and recognition systems and the advancement of video image processing technology, the application research of intelligent transportation systems based on computer vision technology has received more and more attention. It comprehensively utilizes image processing, pattern recognition, artificial intelligence and other technologies. It also involves processing and analyzing the video image sequence collected by the detection system, intelligently understanding the video content and making processing, and dealing with various problems such as accident information judgment, pedestrian and vehicle classification, traffic flow parameter detection, and moving target tracking. It promotes intelligent transportation systems to be more intelligent and practical, and provides comprehensive, real-time traffic status information for traffic management and control. Therefore, the research on the method of traffic information detection based on computer vision has important theoretical and practical significance. The detection and recognition of video targets is an important research direction in the field of intelligent transportation and computer vision. However, due to the background complexity, illumination changes, target occlusion and other factors in the detection and recognition environment, the application still faces many difficulties, and the robustness and accuracy of detection and recognition need to be further improved. In this paper, several key problems in video object detection and recognition are studied, including accurate segmentation of target and background, shadow in complex scenes; accurate classification of extracted foreground targets; and target recognition in complex background. In response to these problems, this paper proposes a corresponding solution.

Delay Optimization Using Genetic Algorithm at the Road Intersection

Metropolitan road traffic mechanisms in developing countries are a critical problem due to fast motorization. The optimization of traffic control is one method to decrease this problem. In this study, a genetic algorithm was implemented to minimize delay at an intersection by finding red and green cycle intervals at an intersection. The objective function minimizes the delay at an intersection and increases progressive flows of traffic on roads. The study was done on real data collected from three t- intersections in the city of Hardwar, India. Traffic data for traffic flows, queue sizes, and traffic speed are collected using video detection systems in the study area. The digital images from the camera were analyzed in real time. The results show that the traffic control performance is improved up to 85% over existing algorithms proposed by the same author.