Automated Video Surveillance System Research Articles

Video Anomaly Detection in Crime Analysis using Deep learning Architecture- A survey

The importance of developing automated video surveillance systems for public safety and security, particularly in crime analysis, has witnessed significant growth in recent years. This survey delves into the current landscape of automated video surveillance systems, emphasizing advancements in crime analysis and exploring existing methodologies and technologies. The study underscores the significance of employing deep learning models in video analysis. Furthermore, the study suggests a deep learning architecture to address the challenges of the existing methods. The goal of the suggested approach is to help security and law enforcement organizations quickly react to any dangers by precisely identifying unusual occurrences or actions in video sequences. The DenseNet-121 architecture is used for efficient spatial and temporal data acquisition from the video frames. This architecture is characterized by a dense connection structure in which all levels get feature mappings from all layers before them. The characteristics of DenseNet-121 can help in the accurate identification of anomalies in video streams and differentiate between normal and abnormal actions. In addition, the study also delves into the topic of using a cell structure with varied sizes to effectively split video sequences. This allows for flexible analysis and can accommodate different sorts of abnormalities. Anomaly detection accuracy can be further improved by adding size, motion, and location information to prediction and measurement models. This study serves as a foundation for the future research that aims to develop a more robust and efficient automated video surveillance solutions.

A deep-learning framework running on edge devices for handgun and knife detection from indoor video-surveillance cameras

The early detection of handguns and knives from surveillance videos is crucial to enhance people’s safety. Despite the increasing development of Deep Learning (DL) methods for general object detection, weapon detection from surveillance videos still presents open challenges. Among these, the most significant are: (i) the very small size of the weapons with respect to the camera field of view and (ii) the need of a real-time feedback, even when using low-cost edge devices for computation. Complex and recently-developed DL architectures could mitigate the former challenge but do not satisfy the latter one. To tackle such limitation, the proposed work addresses the weapon-detection task from an edge perspective. A double-step DL approach was developed and evaluated against other state-of-the-art methods on a custom indoor surveillance dataset. The approach is based on a first Convolutional Neural Network (CNN) for people detection which guides a second CNN to identify handguns and knives. To evaluate the performance in a real-world indoor environment, the approach was deployed on a NVIDIA Jetson Nano edge device which was connected to an IP camera. The system achieved near real-time performance without relying on expensive hardware. The results in terms of both COCO Average Precision (AP = 79.30) and Frames per Second (FPS = 5.10) on the low-power NVIDIA Jetson Nano pointed out the goodness of the proposed approach compared with the others, encouraging the spread of automated video surveillance systems affordable to everyone.

Intelligent Video Surveillance Mechanisms for Abnormal Activity Recognition in Real-Time: A Systematic Literature Review

Video surveillance plays a crucial role in securing indoor and outdoor locations in today's unreliable world, particularly in real-time applications for behaviour detection, comprehension, and labelling activities as normal or suspicious. For example, in the development of automated video surveillance systems, smart video reconnaissance systems based on picture recognition and activity recognition that detect violent behaviours is basic to forestalling wrongdoings and giving public security. According to the literature, Artificial Intelligent, Machine Learning, and deep portrayal-based approaches have been effectively utilized in image recognition and human activity observation tasks. In this literature review, a 3D convolution neural network based on deep learning is used as the proposed methodology. Thus, this article completed a Systematic Literature Review (SLR) in light of intelligent video surveillance to real-time identify abnormal activities from 2016 to 2021. In this current study, 50 research papers were considered and based on the screen filtering, the most suitable 16 papers were filtered based on intelligent video surveillance and real-time abnormal activities. Furthermore, this study identifies potential areas for improvement in intelligent video surveillance systems that can enhance public safety and security, underscoring the importance of ongoing research in this field.

Video surveillance using deep transfer learning and deep domain adaptation: Towards better generalization

Recently, developing automated video surveillance systems (VSSs) has become crucial to ensure the security and safety of the population, especially during events involving large crowds, such as sporting events. While artificial intelligence (AI) smooths the path of computers to think like humans, machine learning (ML) and deep learning (DL) pave the way more, even by adding training and learning components. DL algorithms require data labeling and high-performance computers to effectively analyze and understand surveillance data recorded from fixed or mobile cameras installed in indoor or outdoor environments. However, they might not perform as expected, take much time in training, or not have enough input data to generalize well. To that end, deep transfer learning (DTL) and deep domain adaptation (DDA) have recently been proposed as promising solutions to alleviate these issues. Typically, they can (i) ease the training process, (ii) improve the generalizability of ML and DL models, and (iii) overcome data scarcity problems by transferring knowledge from one domain to another or from one task to another. Although the increasing number of articles proposed to develop DTL- and DDA-based VSSs, a thorough review that summarizes and criticizes the state-of-the-art is still missing. To that end, this paper introduces, to the best of the authors’ knowledge, the first overview of existing DTL- and DDA-based video surveillance to (i) shed light on their benefits, (ii) discuss their challenges, and (iii) highlight their future perspectives.

Deep Multi-Scale Features Fusion for Effective Violence Detection and Control Charts Visualization

The study of automated video surveillance systems study using computer vision techniques is a hot research topic and has been deployed in many real-world CCTV environments. The main focus of the current systems is higher accuracy, while the assistance of surveillance experts in effective data analysis and instant decision making using efficient computer vision algorithms need researchers' attentions. In this research, to the best of our knowledge, we are the first to introduce a process control technique: control charts for surveillance video data analysis. The control charts concept is merged with a novel deep learning-based violence detection framework. Different from the existing methods, the proposed technique considers the importance of spatial information, as well as temporal representations of the input video data, to detect human violence. The spatial information are fused with the temporal dimension of the deep learning model using a multi-scale strategy to ensure that the temporal information are properly assisted by the spatial representations at multi-levels. The proposed frameworks' results are kept in the history-maintaining module of the control charts to validate the level of risks involved in the live input surveillance video. The detailed experimental results over the existing datasets and the real-world video data demonstrate that the proposed approach is a prominent solution towards automated surveillance with the pre- and post-analyses of violent events.

Multi-modal cyber security based object detection by classification using deep learning and background suppression techniques

Foreground object recognition in video is critical in many computer vision applications and automated video surveillance systems. Object detection and tracking are critical steps in navigation, object recognition and surveillance schemes. Object detection is process of separating foreground and background items in photographs. In this paper, we proposes a framework for achieving these tasks with the enhanced cyber security control facility. This proposed algorithmperformed in 2 stages: multi-object detection utilizing the Cyber secure Probabilistic Gaussian Mixture Model (Cy_SPGMM) and background suppression and another stage is multiple moving objects tracking utilizing Kernel convoluted moving window with Kalman filter (KCMW_KF). It can, however, deal with a variety of video sequences in the MOT 20 dataset. The experimental findings reveal that the proposed algorithm detects and tracks foreground objects in complex and dynamic scenarios with high accuracy, robustness, and efficiency. This method also produces smoothened images without noise.

Crime Activity Detection Using Machine Learning

Abstract: This study focuses mostly on criminal activity identification. To solve this issue, we employ the machine learning methodology. A criminal offense is characterized as an act or omission that violates the law and is penalized. Crimes seldom involve a particular place because they can occur anywhere, from small towns to major cities. Using an automated video surveillance system rather than human operators is one strategy to combat this issue. A system like this allows for simultaneous monitoring of numerous screens without sacrificing focus. Intelligent video surveillance is only one of the many fields where understanding human behavior in the actual world has applications.

Moving Object Tracking Using Multiple Cameras in Network

In real time object tracking is an important task in various surveillance application. Nowadays, surveillance system very common in offices, collage campus, shopping malls etc. In this paper, an automated video surveillance system presented. Althrough tracking and counting system are commercially available today; there is need for further reaserach to address the challenges of real world scenarios. There is lot of surveillance cameras installed around us but there are no means to monitor all of them continuously. It is necessary to develop a computer vision based technologies that automatically process those images in order to detect situations or unusual behavior. In order to solve the problem of poor real time tracking performance using convolutional neural network. This paper proposes a fast and accurate real time video detection and tracking algorithm. Keyword: - Video surveillance system, Moving object detection, Tracking, Background subtraction. I. Introduction Real time human body detection and tracking in indoor environment deals with the concept of higher level application of image processing and computer vision. The following key components of identify to path of real time tracking are extracting a feature, background image subtraction. If two similar color objects come to close the tracker of one object can jump to associated object. The problem is called hijacking problem. Drifting problem occur when an object abruptly changes its direction to reverse. In that case it’s become very difficult to track the object because the motion model does not work. An object tracking can be understood as the problem of finding the path (i.e. Trajectory).

Deep learning model based on cascaded autoencoders and one‐class learning for detection and localization of anomalies from surveillance videos

Due to the need for increased security measures for monitoring and safeguarding the activities, video anomaly detection is considered as one of the significant research aspects in the domain of computer vision. Assigning human personnel to continuously check the surveillance videos for finding suspicious activities such as violence, robbery, wrong U-turns, to mention a few, is a laborious and error-prone task. It gives rise to the need for devising automated video surveillance systems ensuring security. Motivated by the same, this paper addresses the problem of detection and localization of anomalies from surveillance videos using pipelined deep autoencoders and one-class learning. Specifically, we used a convolutional autoencoder and a sequence-to-sequence long short-term memory autoencoder in a pipelined fashion for spatial and temporal learning of the videos, respectively. The authors followed the principle of one-class classification for training the model on normal data and testing it on anomalous testing data. The authors achieved a reasonably significant performance in terms of an equal error rate and the time required for anomaly detection and localization comparable to standard benchmarked approaches, thus, qualifies to work in a near-real-time manner for anomaly detection and localization.

Robust baggage detection and classification based on local tri-directional pattern

In recent decades, the automatic video surveillance system has gained significant importance in computer vision community. The crucial objective of surveillance is monitoring and security in public places. In the traditional Local Binary Pattern, the feature description is somehow inaccurate, and the feature size is large enough. Therefore, to overcome these shortcomings, our research proposed a detection algorithm for a human with or without carrying baggage. The Local tri-directional pattern descriptor is exhibited to extract features of different human body parts including head, trunk, and limbs. Then with the help of support vector machine, extracted features are trained and evaluated. Experimental results on INRIA and MSMT17 V1 datasets show that LtriDP outperforms several state-of-the-art feature descriptors and validate its effectiveness.

Live Spoofing Detection for Automatic Human Activity Recognition Applications.

Human Activity Recognition (HAR) has become increasingly crucial in several applications, ranging from motion-driven virtual games to automated video surveillance systems. In these applications, sensors such as smart phone cameras, web cameras or CCTV cameras are used for detecting and tracking physical activities of users. Inevitably, spoof detection in HAR is essential to prevent anomalies and false alarms. To this end, we propose a deep learning based approach that can be used to detect spoofing in various fields such as border control, institutional security and public safety by surveillance cameras. Specifically, in this work, we address the problem of detecting spoofing occurring from video replay attacks, which is more common in such applications. We present a new database containing several videos of users juggling a football, captured under different lighting conditions and using different display and capture devices. We train our models using this database and the proposed system is capable of running in parallel with the HAR algorithms in real-time. Our experimental results show that our approach precisely detects video replay spoofing attacks and generalizes well, even to other applications such as spoof detection in face biometric authentication. Results show that our approach is effective even under resizing and compression artifacts that are common in HAR applications using remote server connections.

Multi-Person Tracking and Crowd Behavior Detection via Particles Gradient Motion Descriptor and Improved Entropy Classifier.

To prevent disasters and to control and supervise crowds, automated video surveillance has become indispensable. In today’s complex and crowded environments, manual surveillance and monitoring systems are inefficient, labor intensive, and unwieldy. Automated video surveillance systems offer promising solutions, but challenges remain. One of the major challenges is the extraction of true foregrounds of pixels representing humans only. Furthermore, to accurately understand and interpret crowd behavior, human crowd behavior (HCB) systems require robust feature extraction methods, along with powerful and reliable decision-making classifiers. In this paper, we describe our approach to these issues by presenting a novel Particles Force Model for multi-person tracking, a vigorous fusion of global and local descriptors, along with a robust improved entropy classifier for detecting and interpreting crowd behavior. In the proposed model, necessary preprocessing steps are followed by the application of a first distance algorithm for the removal of background clutter; true-foreground elements are then extracted via a Particles Force Model. The detected human forms are then counted by labeling and performing cluster estimation, using a K-nearest neighbors search algorithm. After that, the location of all the human silhouettes is fixed and, using the Jaccard similarity index and normalized cross-correlation as a cost function, multi-person tracking is performed. For HCB detection, we introduced human crowd contour extraction as a global feature and a particles gradient motion (PGD) descriptor, along with geometrical and speeded up robust features (SURF) for local features. After features were extracted, we applied bat optimization for optimal features, which also works as a pre-classifier. Finally, we introduced a robust improved entropy classifier for decision making and automated crowd behavior detection in smart surveillance systems. We evaluated the performance of our proposed system on a publicly available benchmark PETS2009 and UMN dataset. Experimental results show that our system performed better compared to existing well-known state-of-the-art methods by achieving higher accuracy rates. The proposed system can be deployed to great benefit in numerous public places, such as airports, shopping malls, city centers, and train stations to control, supervise, and protect crowds.

Spiking neural network based on joint entropy of optical flow features for human action recognition

In the recent past, human action recognition is inviting increased attention in the automated video surveillance systems. An efficient human action classification technique in an unconstrained environment is proposed in this paper. A novel descriptor relying on joint entropy of difference in magnitude and orientation of the optical flow feature is developed in order to model human actions. Initially, flow feature is computed using Pyramid–Warping–Cost volume Network (PWCNet), considering every two consecutive frames. Then, the feature descriptor is formed based on the joint entropy of difference in flow magnitude and orientation collected from the regular grid of each frame in the action sequence. Finally, in order to incorporate long-term temporal dependency, a spiking neural network is embedded to aggregate the information across the frames. Different optimization techniques and different types of hidden nodes are utilized in the spiking neural network to analyze the performance of the proposed work. Extensive experiments on the benchmark datasets for human action recognition show the efficacy of the proposed method.

Mathematical model of an automated video surveillance system for performing special functions.

The mathematical model of the automated video surveillance system is offered. Mathematical models of separate units of the aircraft are considered: quadrocopter dynamics, RC transmitter, RC receiver, HD camera. It is shown that such mathematical models are the most effective for performing key tasks.

Deep learning-based video surveillance system managed by low cost hardware and panoramic cameras

The design of automated video surveillance systems often involves the detection of agents which exhibit anomalous or dangerous behavior in the scene under analysis. Models aimed to enhance the video pattern recognition abilities of the system are commonly integrated in order to increase its performance. Deep learning neural networks are found among the most popular models employed for this purpose. Nevertheless, the large computational demands of deep networks mean that exhaustive scans of the full video frame make the system perform rather poorly in terms of execution speed when implemented on low cost devices, due to the excessive computational load generated by the examination of multiple image windows. This work presents a video surveillance system aimed to detect moving objects with abnormal behavior for a panoramic 360∘ surveillance camera. The block of the video frame to be analyzed is determined on the basis of a probabilistic mixture distribution comprised by two mixture components. The first component is a uniform distribution, which is in charge of a blind window selection, while the second component is a mixture of kernel distributions. The kernel distributions generate windows within the video frame in the vicinity of the areas where anomalies were previously found. This contributes to obtain candidate windows for analysis which are close to the most relevant regions of the video frame, according to the past recorded activity. A Raspberry Pi microcontroller based board is employed to implement the system. This enables the design and implementation of a system with a low cost, which is nevertheless capable of performing the video analysis with a high video frame processing rate.

Intelligent optimization for automated video surveillance at the edge: A cross-layer approach

The interest in large scale automated video surveillance systems and the interest in using cloud in supporting such systems has increased dramatically. Unfortunately, building such a large system requires huge resources (for processing and storage) and very high network bandwidth. This paper, proposes a framework for resources efficient intelligent automated surveillance framework that utilizes edge servers. In this framework, multiple video sources capture and send videos to a an edge server which performs an intelligent computer vision based cross-layer optimization of the video sources hardware resources and the network bandwidth. The network can be a wireless network and the sources can be mobile. The proposed solution changes the application rates and the required link layer parameters (the transmission opportunities in our case study) of the sending video sources according to the dynamic network conditions to maximize the overall accuracy of the computer vision algorithm(s) of interest in the system. The proposed framework utilizes an enhanced effective airtime estimation algorithm utilizing a Proportional Integral Differential (PID) controller that measure the available useful bandwidth in the network. Furthermore, we propose a bandwidth pruning mechanism to reach any desired tradeoff between the computer vision algorithm accuracy and the energy consumption of the video sources. We evaluate and present the effectiveness of the proposed framework, the effective airtime estimation algorithm, and the proposed bandwidth pruning mechanism, through extensive experiments using OPNET.

CARRIED BAGGAGE DETECTION AND CLASSIFICATION USING MULTI-TREND BINARY CODE DESCRIPTOR AND SUPPORT VECTOR MACHINE

Automatic video surveillance systems have gained significant importance due to an increase in crime rate over the last two decades. Automatic baggage detection through surveillance camera can help in security and monitoring in public places. A detection algorithm for humans (with or without carrying baggage) is proposed in this paper. Detection in the proposed method can be achieved by employing spatial information of the baggage of various texture patterns with locus to the human body carrying it. To extract the features of body parts (such as head, trunk and limbs), the descriptor is exhibited and trained by the support vector machine classifier. The proposed approach has been widely assessed by using publically available datasets. The experimental results have shown that the proposed approach is viable for baggage detection and classification as compared to the other available approaches.

RETRACTED: Adaptive motion estimation and sequential outline separation based moving object detection in video surveillance system

Moving object detection is an important task in the basic research field of automated video surveillance systems in computer vision and video processing. The application of moving object tracking is critical for military, surveillance systems and operational robot applications, and getting more critical day by day. Closer view gap, ghosting, and sudden lighting changes have been the primary issues in moving object detection in existing methods. To think about the above issues, this work proposes two methodologies like consolidating background subtraction and improved sequential outline separation strategies for the recognition of various moving objects from indoor and outdoor genuine video dataset. This kind of framework can be realized in the general public places such as shopping malls, air terminals and railway stations or the safety of any private premises is the primary concern. This work precisely distinguishes the moving objects with shifting object size and number in various complex situations. The simulation work is done with MATLAB software, to measure the detection error and processing time of the proposed strategy. The proposed sequential outline separation method starts with background subtraction and foreground detection for motion and object discovery and it is a procedure of separating the territory of enthusiasm from developed background. Simulation results and error rate investigation demonstrate that our proposed strategies identify the moving targets productively. As Compared to other conventional systems, our proposed adaptive motion estimation and sequential outline separation method performs better by achieving an accuracy of 97.45%, a sensitivity of 94.2%, and a specificity of 97.72%.

Hybrid imaging-AI approach for handling critical situations in a fast-changing environment: preliminary study

The purpose of this study is to explore the possibility of using selected imaging technologies in automated video surveillance systems. The main goal of this project is to handle events that may lead to security risks, injuries, etc in various environments without relaying on more conventional sensors such as infrared photocells. For this purpose it is necessary to perform a thorough analysis of the events to be interpreted as situations of interest. It is also important to consider the hardware requirements and restrictions for developing such system. The project requires defining a hardware as well as software platform(s) and their integration into an automated tool. This paper describes the implementation of the famous Microsoft Kinect 2.0 depth sensor (well known in gaming and recreational applications) for shape/skeleton detection, and its integration into an artificial intelligence based platform utilizing selected machine learning methods. The author reveals the system implementation details, and then demonstrates its shape detection capabilities while in operation.

A Robust Watermarking Method for an Authentication of Video Surveillance Applications

The problem of authenticating the video content is the major role of any automated video surveillance systems (AVS). This research work delivers an approach to authenticate the surveillance video which can be used by attorneys to prove their clients using Digital Watermarking. The digital watermark gives an assurance that the provided video surveillance frame is not tampered. In this paper, we proposed a robust video watermarking approach for an authentication of video surveillance applications. The digital watermark is embedded into the Discrete Wavelet domain of the identified key frames using holoentropy. Here, the pixel map will be optimallygenerated for the watermarking and the image embedding using the proposed classifier using Moth – flame - Rider optimization based Neural Network (MF-ROA-based NN) will generate the optimal map prediction based on the fitness measure, such as wavelet coefficient, energy, entropy, loop coefficient, and standard deviation, respectively. This optimal map is used for both embedding and extraction. The experimental results proves that the proposed systems can authenticate the surveillance video frames against various attacks when compared to the existing systems.