Smart Video Surveillance System Research Articles

Tracking illegal activities using video surveillance systems: a review of the current state of research

The current state of research on the use of the neural networks under martial law to identify offenders committing illegal acts, prevent acts of terrorism, combat sabotage groups in cities, track weapons and control traffic is considered. The methods of detecting illegal actions, weapons, face recognition and traffic violations using video surveillance cameras are analysed. It is proposed to introduce the studied methods into the work of “smart” video surveillance systems in Ukrainian settlements.
 The most effective means of reducing the number of offences is the inevitability of legal liability for offences, so many efforts in law enforcement are aimed at preventing offences. Along with public order policing by patrol police, video surveillance is an effective way to prevent illegal activities in society. Increasing the coverage area of cameras and their number helps to ensure public safety in the area where they are used. However, an increase in the number of cameras creates another problem which is the large amount of video data that needs to be processed. To solve the problem of video data processing, various methods are used, the most modern of which is the use of artificial intelligence to filter a large amount of data from video cameras and the application of various video processing algorithms. The ability to simultaneously process video data from many CCTV cameras without human intervention not only contributes to public safety, but also improves the work of patrol police. The introduction of smart video surveillance systems allows monitoring the situation in public places around the clock, even if there is no police presence in the area.
 In the reviewed studies of video surveillance systems, neural networks, in particular MobileNet V2, YOLO, mYOLOv4-tiny, are used to track illegal actions, criminals and weapons, which are trained on large amounts of video and photo data. It has been found that although neural networks used to require a lot of computing power, they can now be used in IoT systems and smartphones, and this contributes to the fact that more video surveillance devices can be used to monitor the situation.

Enhancing Object Detection in Smart Video Surveillance: A Survey of Occlusion-Handling Approaches

Smart video surveillance systems (SVSs) have garnered significant attention for their autonomous monitoring capabilities, encompassing automated detection, tracking, analysis, and decision making within complex environments, with minimal human intervention. In this context, object detection is a fundamental task in SVS. However, many current approaches often overlook occlusion by nearby objects, posing challenges to real-world SVS applications. To address this crucial issue, this paper presents a comprehensive comparative analysis of occlusion-handling techniques tailored for object detection. The review outlines the pretext tasks common to both domains and explores various architectural solutions to combat occlusion. Unlike prior studies that primarily focus on a single dataset, our analysis spans multiple benchmark datasets, providing a thorough assessment of various object detection methods. By extending the evaluation to datasets beyond the KITTI benchmark, this study offers a more holistic understanding of each approach’s strengths and limitations. Additionally, we delve into persistent challenges in existing occlusion-handling approaches and emphasize the need for innovative strategies and future research directions to drive substantial progress in this field.

Ensemble-Based Knowledge Distillation for Video Anomaly Detection

Video anomaly detection has become a vital task for smart video surveillance systems because of its significant potential to minimize the video data to be analyzed by choosing unusual and critical patterns in the scenes. In this paper, we introduce three novel ensemble and knowledge distillation-based adaptive training methods to handle robust detection of different abnormal patterns in video scenes. Our approach leverages the adaptation process by providing information transfer from multiple teacher models with different network structures and further alleviates the catastrophic forgetting issue. The proposed ensemble knowledge distillation methods are implemented on two state-of-the-art anomaly detection models. We extensively evaluate our methods on two public video anomaly datasets and present a detailed analysis of our results. Finally, we show that not only does our best version model achieve comparable performance with a frame-level AUC of 75.82 to other state-of-the-art models on UCF-Crime as the target dataset, but more importantly our approaches prevent catastrophic forgetting and dramatically improve our model’s performance.

Understanding Policy and Technical Aspects of AI-enabled Smart Video Surveillance to Address Public Safety

Recent advancements in artificial intelligence (AI) have seen the emergence of smart video surveillance (SVS) in many practical applications, particularly for building safer and more secure communities in our urban environments. Cognitive tasks, such as identifying objects, recognizing actions, and detecting anomalous behaviors, can produce data capable of providing valuable insights to the community through statistical and analytical tools. However, artificially intelligent surveillance systems design requires special considerations for ethical challenges and concerns. The use and storage of personally identifiable information (PII) commonly pose an increased risk to personal privacy. To address these issues, this paper identifies the privacy concerns and requirements needed to address when designing AI-enabled smart video surveillance. Further, we propose the first end-to-end AI-enabled privacy-preserving smart video surveillance system that holistically combines computer vision analytics, statistical data analytics, cloud-native services, and end-user applications. Finally, we propose quantitative and qualitative metrics to evaluate intelligent video surveillance systems. The system shows the 17.8 frame-per-second (FPS) processing in extreme video scenes. However, considering privacy in designing such a system results in preferring the pose-based algorithm to the pixel-based one. This choice resulted in dropping accuracy in both action and anomaly detection tasks. The results drop from 97.48% to 73.72% in anomaly detection and 96% to 83.07% in the action detection task. On average, the latency of the end-to-end system is 36.1 seconds.

Development of SmartTrack with Authentication for Tracking Objects Business Promotion

Abstract: One of the most important and difficult duties of a smart video surveillance system is object tracking. The major purpose of this is to pinpoint the precise location of a specific item and to highlight that location with a buzzer and light. We considered secrecy when designing. We must protect the private documents. Therefore, we need to offer users permission before granting access to the location, buzzer, and light. This system was specifically created with authorization and confidentiality in mind. The various object tracking techniques that the researchers used are thoroughly reviewed in this publication. The purpose of this essay isto review numerous ways to object tracking and to examine, study, and evaluate them

Advancements in Real-Time Face Recognition Algorithms for Enhanced Smart Video Surveillance

Smart video surveillance systems are essential for guaranteeing security in a variety of settings. The capacity to recognize and follow people of interest inside video feeds is made possible by real-time face recognition, which is a key component of these systems. The refinement of a real-time face recognition system designed for video surveillance applications is the focus of this research. We proposed a technique that can effectively recognize faces in a variety of lighting situations, poses, and occlusions by utilizing innovative combination of Haar Cascade and Convolutional Neural Networks. To balance computational difficulties and increase scalability, the merging of edge computing and cloud-based solutions are also investigated. The performance of the proposed system is evaluated by vigilant evaluation, demonstrating its applicability in real-world circumstances. Our research aims to advance video surveillance technology with an emphasis on the synergy of real-time face recognition for more secure and effective surveillance systems.

Smart Security Camera using Machine Learning

Numerous applications have been found for video surveillance. Smart video surveillance systems can improve situational awareness at various scales of time and space. It describes a remote control and surveillance architecture based on mobile devices. To record camera photos and detect intrusion using an image comparison technique, this project uses the Opency library. The streamed video is sent from the server to the remote administrator over the phone after the comparison is completed and an intrusion is discovered. Afterward, admin can take the necessary action. Automatic video analytics are used in smart surveillance to increase the efficiency of security measures. By automatically identifying the individual's deviant conduct, this system complements the existing video surveillance systems while also introducing intelligent analysis of single person activity to improve home security. By sending an SMS, the user is informed and the pertinent data is recorded. The user is able to watch the specific video. This system keeps the home secure, which lowers the frequency of burglary cases and improves social stability.

Perception of Risks and Usefulness of Smart Video Surveillance Systems

The number of video cameras in public places increases due to different reasons such as detecting dangers (e.g., thefts, robberies, terrorist attacks) and security breaches in crowds. The application of video surveillance systems is sometimes evaluated ambivalently; therefore, the presented study focuses on factors influencing the acceptance of a privacy-friendly, smart video surveillance system. Overall, 216 persons aged between 18 and 81 years participated in an online survey. In terms of the perceived usefulness, there are significant interactions of public spaces × gender and public spaces × time of day. In addition, the assessment of different privacy levels of a video surveillance system differ significantly in terms of perceived risk. Interestingly, men rate the risk concerning their own privacy significantly higher than women do. Participants rate the presented system as fairly useful and slightly risky for their own privacy. The findings of the presented exploratory study provide insight into how people perceive smart video surveillance. These findings have the potential to support the conditions of the use of smart video surveillance systems and to address the possibly affected individuals.

Smart Deep Learning Based Human Behaviour Classification for Video Surveillance

Real-time video surveillance system is commonly employed to aid security professionals in preventing crimes. The use of deep learning (DL) technologies has transformed real-time video surveillance into smart video surveillance systems that automate human behavior classification. The recognition of events in the surveillance videos is considered a hot research topic in the field of computer science and it is gaining significant attention. Human action recognition (HAR) is treated as a crucial issue in several applications areas and smart video surveillance to improve the security level. The advancements of the DL models help to accomplish improved recognition performance. In this view, this paper presents a smart deep-based human behavior classification (SDL-HBC) model for real-time video surveillance. The proposed SDL-HBC model majorly aims to employ an adaptive median filtering (AMF) based pre-processing to reduce the noise content. Also, the capsule network (CapsNet) model is utilized for the extraction of feature vectors and the hyperparameter tuning of the CapsNet model takes place utilizing the Adam optimizer. Finally, the differential evolution (DE) with stacked autoencoder (SAE) model is applied for the classification of human activities in the intelligent video surveillance system. The performance validation of the SDL-HBC technique takes place using two benchmark datasets such as the KTH dataset. The experimental outcomes reported the enhanced recognition performance of the SDL-HBC technique over the recent state of art approaches with maximum accuracy of 0.9922.

Algorithm to Merge Images to Increase Browsing Speed of Smart Video Surveillance System

Algorithm to Merge Images to Increase Browsing Speed of Smart Video Surveillance System

Abnormal Human Behavior Detection in Videos: A Review

Modeling human behavior patterns for detecting the abnormal event has become an important domain in recentyears. A lot of efforts have been made for building smart video surveillance systems with the purpose ofscene analysis and making correct semantic inference from the video moving target. Current approaches havetransferred from rule-based to statistical-based methods with the need of efficient recognition of high-levelactivities. This paper presented not only an update expanding previous related researches, but also a study coveredthe behavior representation and the event modeling. Especially, we provided a new perspective for eventmodeling which divided the methods into the following subcategories: modeling normal event, predictionmodel, query model and deep hybrid model. Finally, we exhibited the available datasets and popular evaluationschemes used for abnormal behavior detection in intelligent video surveillance. More researches will promotethe development of abnormal human behavior detection, e.g. deep generative network, weakly-supervised. It isobviously encouraged and dictated by applications of supervising and monitoring in private and public space.The main purpose of this paper is to widely recognize recent available methods and represent the literature ina way of that brings key challenges into notice.

Smart Video Surveillance System Based on Edge Computing.

New processing methods based on artificial intelligence (AI) and deep learning are replacing traditional computer vision algorithms. The more advanced systems can process huge amounts of data in large computing facilities. In contrast, this paper presents a smart video surveillance system executing AI algorithms in low power consumption embedded devices. The computer vision algorithm, typical for surveillance applications, aims to detect, count and track people’s movements in the area. This application requires a distributed smart camera system. The proposed AI application allows detecting people in the surveillance area using a MobileNet-SSD architecture. In addition, using a robust Kalman filter bank, the algorithm can keep track of people in the video also providing people counting information. The detection results are excellent considering the constraints imposed on the process. The selected architecture for the edge node is based on a UpSquared2 device that includes a vision processor unit (VPU) capable of accelerating the AI CNN inference. The results section provides information about the image processing time when multiple video cameras are connected to the same edge node, people detection precision and recall curves, and the energy consumption of the system. The discussion of results shows the usefulness of deploying this smart camera node throughout a distributed surveillance system.

A Novel Deep Learning Based Automated Academic Activities Recognition in Cyber-Physical Systems

Internet of Things (IoT) has rapidly developed in multidisciplinary research topics, particularly in Cyber-Physical infrastructures, such as smart-health care, transportation systems, vehicle management surveillance systems. The smart-video surveillance system has become an essential part of almost all security applications, including academic institutions. University campuses have rich video repositories comprising almost all kinds of academic and non-academic activities. Researchers have introduced many state-of-art activity recognition methods for various application domains with the availability of several activity data sets. Unfortunately, none of these data sets or methods have been developed explicitly for academia and do not cover academic activities. With the advancement of deep learning and IoT, the processing of large-scale video data has become convenient for performing various video analysis tasks. Thus, in this work, an automated deep learning-based academic activities recognition system is presented in smart-cyber infrastructure. We explore a new academic campus domain for research and proposed a novel Convolutional Neural Network (CNN) model for academic activities recognition utilizing a realistic campus dataset. The video database typically contains long, 24-hour video streams recorded by surveillance cameras installed in campus environments. The proposed model’s efficiency is tested through extensive experimentation in terms of accuracy, computation time, and memory requirement. The experimental results reveal that the proposed method attains good results with an accuracy of 98%.

Real-time smart video surveillance to manage safety: A case study of a transport mega-project

There is a tendency for accidents and even fatalities to arise when people enter hazardous work areas during the construction of projects in urban areas. A limited amount of research has been devoted to developing vision-based proximity warning systems that can determine when people enter a hazardous area automatically. Such systems, however, are unable to identify specific hazards and the status of a piece of plant (e.g., excavator) in real-time. In this paper, we address this limitation and develop a real-time smart video surveillance system that can detect people and the status of plant (i.e. moving or stationary) in a hazardous area. The application of this approach is demonstrated during the construction of a mega-project, the Wuhan Rail Transit System in China. We reveal that our combination of computer vision and deep learning can accurately recognize people in a hazardous work area in real-time during the construction of transport projects. Our developed systems can provide instant feedback concerning unsafe behavior and thus enable appropriate actions to be put in place to prevent their re-occurrence.

Multi-view face recognition using deep neural networks

F Face recognition has been widely used in modern intelligent systems, such as smart video surveillance, online payment, and intelligent access control system. Existing face recognition algorithms are prone to be attacked by various face presentation attacks (face-PAs), such as printed paper, video replay, and silicone masks. To optimally handle the aforementioned problems, we formulate a novel deep architecture to increase the accuracy of multi-view human face recognition. In particular, in the first place, a novel deep neural network is built for deeply encoding the face regions, where a novel face alignment algorithm is employed to localize the key points inside faces. Subsequently, we utilize the well-known PCA for reducing the dimensionality of the deep features and simultaneously, removing the redundant and contaminated visual features. Thereafter, we propose a joint Bayesian framework in order to evaluate the similarity of feature vectors and highly competitive face classification accuracy can be achieved. Comprehensive experiments were conducted on our compiled CAS-PEAL dataset and achieved a 98.52% face recognition performance. Moreover, our proposed face recognition system can robustly handle various face recognition attack under various contexts.

Recognising of repetitive and stereotyped movements for children with autism spectrum disorder

People with autism often engage in stereotyped and repetitive motor movements. Hence, our aim is to put out a smart video surveillance system that facilitates the diagnosis of autism for doctors. In this respect, we propose an automatic stereotypical motor movement detection system in real time. Firstly, we use the Kinect sensor to monitor the child's with autism movements. Secondly, we propose a data integration process to make the provided data from Kinect sensor more comprehensive and specific. Thirdly, we perform the gesture detection by using the well know machine learning algorithms such as decision tree, artificial neural network and nearest neighbour. The obtained result is very promising and shows that the data integration step enhances the gesture recognition. The experimental results show that our system can achieve above 99.8% recognition rate. Also, we evaluate the efficiency of different machine learning algorithms in recognition tasks.

An application layer forward error correction method of wireless network broadcasting communication for smart video surveillance system

Aiming at the wireless channel in which the state of video surveillance communication network changes dramatically, to ensure the video impact under a restricted channel bandwidth condition, an application layer forward error correction system for wireless video communication is proposed. This scheme is an improved forward error correction algorithm. The most appropriate redundancy amount is implemented for each FEC block by constantly altering the redundancy in FEC blocks. Firstly, the system describes and implements the optimum video transmission. Then, the RCPC code and Raptor code are used as forward error correction codes in physical layer and application layer respectively, in this way, the optimal system solution can be acquired. In the forward error correction technology, a feedback mechanism is then developed between the receiver and the sender, which will acknowledge that the receiver can extract the packet loss rate from the received speech signal in the current network state in real time, and then the packet loss rate will be transferred back to the sender by the feedback mechanism. The sender can modify the forward packet loss rate according to the packet loss rate. The error-correcting schema can help the next speech signal have the most appropriate forward error-correcting redundancy, reducing bandwidth waste, significantly improving the adaptability and efficiency of forward error-correcting technology in the wireless network. The simulation results show that the average video quality of the proposed scheme is 0.5–4 dB higher than that of the equal error protection scheme and the extended window unequal error protection scheme.

Automatic detection of stereotyped movements in autistic children using the Kinect sensor

Autism Spectrum Disorders (ASD) is a developmental disorder that affects communications, social skills or behaviours that can occur in some people. Children or adults with ASD often have repetitive motor movements or unusual behaviours. The objective of this work is to automatically detect stereotypical motor movements in real time using Kinect sensor. The approach is based on the $P Point-Cloud Recogniser to identify multi-stroke gestures as point clouds. This paper presents new methodology to automatically detect five stereotypical motor movements: body rocking, hand flapping, fingers flapping, hand on the face and hands behind back. With many ASD-children, our proposed system gives us satisfactory results. This can help to implement a smart video surveillance system and then helps clinicians in the diagnosing ASD.

Automatic detection of stereotyped movements in autistic children using the Kinect sensor

Autism Spectrum Disorders (ASD) is a developmental disorder that affects communications, social skills or behaviours that can occur in some people. Children or adults with ASD often have repetitive motor movements or unusual behaviours. The objective of this work is to automatically detect stereotypical motor movements in real time using Kinect sensor. The approach is based on the $P Point-Cloud Recogniser to identify multi-stroke gestures as point clouds. This paper presents new methodology to automatically detect five stereotypical motor movements: body rocking, hand flapping, fingers flapping, hand on the face and hands behind back. With many ASD-children, our proposed system gives us satisfactory results. This can help to implement a smart video surveillance system and then helps clinicians in the diagnosing ASD.

Breaking the Limits in Urban Video Monitoring: Massive Crowd Sourced Surveillance over Vehicles

Contemporary urban environments are in prompt need of the means for intelligent decision- making, where a crucial role belongs to smart video surveillance systems. While existing deployments of stationary monitoring cameras already deliver notable societal benefits, the proposed concept of massive video surveillance over connected vehicles that we contribute in this article may further augment these important capabilities. We therefore introduce the envisioned system concept, discuss its implementation, outline the high-level architecture, and identify major data flows, while also offering insights into the corresponding design and deployment aspects. Our conducted case study confirms the potential of the described crowd sourced vehicular system to effectively complement and eventually surpass even the best of today's static video surveillance setups. We expect that our proposal will become of value and integrate seamlessly into the future Internetof- Things landscape, thus enabling a plethora of advanced urban applications.