Video Surveillance System Research Articles

Multi-Object Detection using Enhanced YOLOv2 and LuNet Algorithms in Surveillance Videos

Multiple object tracking (MOT) in videos benefits multiple applications, including robot navigation, video surveillance, video analytics, and intelligent transportation systems. Although significant progress has been made since early studies, visual tracking of many objects remains challenging because of frequent occlusions in measurements, environmental noise, changeable number of objects, and appearance similarity across objects. The proposed work focused on three significant processes, feature extraction, object detection, and classification, to identify moving objects before sharing information. This work proposes a multi-object video detection method using LuNet and deep reinforcement learning. The enhanced “you only look once” version 2 (YOLOv2) initially detects numerous objects. In this work, a base network of the YOLOv2 changed by lowering the metrics and substituting it with LuNet. In the enhanced model, the LuNet network is used for feature extraction to extract the most expected characteristics from the image. Furthermore, the proposed model is compact because of the underlying network's LuNet architecture. To demonstrate the proposed technique's performance, this method compares it to numerous state-of-the-art algorithms on the MOT20 vehicle benchmark dataset. The proposed method achieves a higher % classification accuracy of 94% for moving object settings. The experiments demonstrate that the proposed method outperforms existing models in terms of performance and accuracy.

Knowledge Development Trajectories of Intelligent Video Surveillance Domain: An Academic Study Based on Citation and Main Path Analysis.

Smart city is an area where the Internet of things is used effectively with sensors. The data used by smart city can be collected through the cameras, sensors etc. Intelligent video surveillance (IVS) systems integrate multiple networked cameras for automatic surveillance purposes. Such systems can analyze and monitor video data and perform automatic functions required by users. This study performed main path analysis (MPA) to explore the development trends of IVS research. First, relevant articles were retrieved from the Web of Science database. Next, MPA was performed to analyze development trends in relevant research, and g-index and h-index values were analyzed to identify influential journals. Cluster analysis was then performed to group similar articles, and Wordle was used to display the key words of each group in word clouds. These key words served as the basis for naming their corresponding groups. Data mining and statistical analysis yielded six major IVS research topics, namely video cameras, background modeling, closed-circuit television, multiple cameras, person reidentification, and privacy, security, and protection. These topics can boost the future innovation and development of IVS technology and contribute to smart transportation, smart city, and other applications. According to the study results, predictions were made regarding developments in IVS research to provide recommendations for future research.

Tracking illegal activities using video surveillance systems: a review of the current state of research

The current state of research on the use of the neural networks under martial law to identify offenders committing illegal acts, prevent acts of terrorism, combat sabotage groups in cities, track weapons and control traffic is considered. The methods of detecting illegal actions, weapons, face recognition and traffic violations using video surveillance cameras are analysed. It is proposed to introduce the studied methods into the work of “smart” video surveillance systems in Ukrainian settlements.
 The most effective means of reducing the number of offences is the inevitability of legal liability for offences, so many efforts in law enforcement are aimed at preventing offences. Along with public order policing by patrol police, video surveillance is an effective way to prevent illegal activities in society. Increasing the coverage area of cameras and their number helps to ensure public safety in the area where they are used. However, an increase in the number of cameras creates another problem which is the large amount of video data that needs to be processed. To solve the problem of video data processing, various methods are used, the most modern of which is the use of artificial intelligence to filter a large amount of data from video cameras and the application of various video processing algorithms. The ability to simultaneously process video data from many CCTV cameras without human intervention not only contributes to public safety, but also improves the work of patrol police. The introduction of smart video surveillance systems allows monitoring the situation in public places around the clock, even if there is no police presence in the area.
 In the reviewed studies of video surveillance systems, neural networks, in particular MobileNet V2, YOLO, mYOLOv4-tiny, are used to track illegal actions, criminals and weapons, which are trained on large amounts of video and photo data. It has been found that although neural networks used to require a lot of computing power, they can now be used in IoT systems and smartphones, and this contributes to the fact that more video surveillance devices can be used to monitor the situation.

Deep Reinforcement Learning-Empowered Cost-Effective Federated Video Surveillance Management Framework.

Video surveillance systems are integral to bolstering safety and security across multiple settings. With the advent of deep learning (DL), a specialization within machine learning (ML), these systems have been significantly augmented to facilitate DL-based video surveillance services with notable precision. Nevertheless, DL-based video surveillance services, which necessitate the tracking of object movement and motion tracking (e.g., to identify unusual object behaviors), can demand a significant portion of computational and memory resources. This includes utilizing GPU computing power for model inference and allocating GPU memory for model loading. To tackle the computational demands inherent in DL-based video surveillance, this study introduces a novel video surveillance management system designed to optimize operational efficiency. At its core, the system is built on a two-tiered edge computing architecture (i.e., client and server through socket transmission). In this architecture, the primary edge (i.e., client side) handles the initial processing tasks, such as object detection, and is connected via a Universal Serial Bus (USB) cable to the Closed-Circuit Television (CCTV) camera, directly at the source of the video feed. This immediate processing reduces the latency of data transfer by detecting objects in real time. Meanwhile, the secondary edge (i.e., server side) plays a vital role by hosting a dynamically controlling threshold module targeted at releasing DL-based models, reducing needless GPU usage. This module is a novel addition that dynamically adjusts the threshold time value required to release DL models. By dynamically optimizing this threshold, the system can effectively manage GPU usage, ensuring resources are allocated efficiently. Moreover, we utilize federated learning (FL) to streamline the training of a Long Short-Term Memory (LSTM) network for predicting imminent object appearances by amalgamating data from diverse camera sources while ensuring data privacy and optimized resource allocation. Furthermore, in contrast to the static threshold values or moving average techniques used in previous approaches for the controlling threshold module, we employ a Deep Q-Network (DQN) methodology to manage threshold values dynamically. This approach efficiently balances the trade-off between GPU memory conservation and the reloading latency of the DL model, which is enabled by incorporating LSTM-derived predictions as inputs to determine the optimal timing for releasing the DL model. The results highlight the potential of our approach to significantly improve the efficiency and effective usage of computational resources in video surveillance systems, opening the door to enhanced security in various domains.

Security Threat Object Recognition System

Security Threat object recognition has emerged as a very advanced topic in the field of video surveillance community. Due to need of system has been proposed for video monitoring or surveillance system for public and private places to protect againt thept and recognise the suspicious object which is harmful for the society and the people. Because of these systems' complexity, researchers frequently handle the various stages of analysis—such as foreground segmentation, stationary object recognition, and abandonment validation—independently. The impact of each level of improvement on the overall performance of the system has not been investigated, despite the fact that each stage has seen gains. This is because the advancements are rarely applied across the entire pipeline. This study presents a thorough assessment of the state-of-the-art methods for each stage and formalizes the framework used by systems for abandoned object detection. Additionally, we develop a multi-configuration system that enables the selection of many options for every step, with the goal of identifying the combination that yields the best results. The scientific community has internet access to this multi-configuration. Keywords: Object detection, Surveillance, Threat Detection, Computer Vision, Deep Learning, YOLO, Image analysis, CNN, R-CNN, SSD

Quantile-Based Maximum Likelihood Training for Outlier Detection

Discriminative learning effectively predicts true object class for image classification. However, it often results in false positives for outliers, posing critical concerns in applications like autonomous driving and video surveillance systems. Previous attempts to address this challenge involved training image classifiers through contrastive learning using actual outlier data or synthesizing outliers for self-supervised learning. Furthermore, unsupervised generative modeling of inliers in pixel space has shown limited success for outlier detection. In this work, we introduce a quantile-based maximum likelihood objective for learning the inlier distribution to improve the outlier separation during inference. Our approach fits a normalizing flow to pre-trained discriminative features and detects the outliers according to the evaluated log-likelihood. The experimental evaluation demonstrates the effectiveness of our method as it surpasses the performance of the state-of-the-art unsupervised methods for outlier detection. The results are also competitive compared with a recent self-supervised approach for outlier detection. Our work allows to reduce dependency on well-sampled negative training data, which is especially important for domains like medical diagnostics or remote sensing.

Deep Learning-Based Intelligent Video Surveillance System for Real-Time Motion Detection

The proliferation of surveillance systems demands advanced solutions to effectively monitor and analyze video feeds for security and safety purposes. In this paper, we propose a Deep Learning-Based Intelligent Video Surveillance System (DIVSS) designed to enhance traditional Closed-Circuit Television (CCTV) setups. DIVSS integrates state-of-the-art deep learning techniques for real-time motion detection and object recognition, enabling proactive threat detection and automated response mechanisms. The system utilizes convolutional neural networks (CNNs) for robust feature extraction and classification, enabling accurate identification of objects of interest amidst complex backgrounds and varying[1] lighting conditions. Furthermore, DIVSS incorporates region of interest (ROI) monitoring to focus attention on specific areas within the surveillance footage, optimizing computational resources and improving response times. We evaluate the performance of DIVSS through comprehensive experiments conducted on diverse surveillance scenarios, demonstrating its effectiveness in enhancing security measures and providing actionable[3] insights for surveillance operators. The proposed system represents a significant advancement in video surveillance technology, offering unparalleled capabilities for proactive threat detection and real-time response in dynamic environments. Keywords— Deep learning has transformed video surveillance, enabling real-time motion detection and object recognition using Convolutional Neural Networks (CNNs). This technology provides proactive threat detection, intelligent systems for ROI monitoring, and enhances overall security measures.

Development of Smart Video Surveillance Systems: Technical and Security Challenges in Urban Environments

Video surveillance schemes have become a critical tool for safeguarding urban and industrial services, thanks to the rapid advancement in object recognition, tracking, and human activity analysis technologies. The achievement of these systems relies on background analysis, motion detection, and innovative solutions for real-time data processing and storage. These schemes have transitioned from analog setups to IP-based systems, offering more flexibility and remote control capabilities. However, they face important cybersecurity challenges, such as data breaches, bandwidth consumption, and high processing demands. This research explores the technological advancements in video surveillance systems, focusing on the security challenges and analytics in urban environments. Furthermore, it discourses strategies for optimizing system efficiency through modern compression algorithms and real-time human activity analysis.

HF-YOLO: Advanced Pedestrian Detection Model with Feature Fusion and Imbalance Resolution

Pedestrian detection is crucial for various applications, including intelligent transportation and video surveillance systems. Although recent research has advanced pedestrian detection models like the YOLO series, they still face limitations in handling diverse pedestrian scales, leading to performance challenges. To address these issues, we propose HF-YOLO, an advanced pedestrian detection model. HF-YOLO tackles the complexities of pedestrian detection in complex scenes by addressing scale variations and occlusions among pedestrians. In the feature fusion stage, our algorithm leverages both shallow localization information and deep semantic information. This involves fusing P2 layer features and adding a high-resolution detection layer, significantly improving the detection of small-scale pedestrians and occluded instances. To enhance feature representation, HF-YOLO incorporates the HardSwish activation function, introducing more non-linear factors and strengthening the model’s ability to represent complex and discriminative features. Additionally, to address regression imbalance, a balance factor is introduced to the CIoU loss function. This modification effectively resolves the imbalance problem and enhances pedestrian localization accuracy. Experimental results demonstrate the effectiveness of our proposed algorithm. HF-YOLO achieves notable improvements, including a 3.52% increase in average precision, a 1.35% boost in accuracy, and a 4.83% enhancement in recall. Moreover, the algorithm maintains real-time performance with a detection time of 8.5ms, meeting the stringent requirements of real-time applications.

A Fast and Robust Safety Helmet Network Based on a Mutilscale Swin Transformer

Visual inspection of the workplace and timely reminders of unsafe behaviors (e.g, not wearing a helmet) are particularly significant for avoiding injuries to workers on the construction site. Video surveillance systems generate large amounts of non-structure image data on site for this purpose; however, they require real-time recognition automation solutions based on computer vision. Although various deep-learning-based models have recently provided new ideas for identifying helmets in traffic monitoring, few solutions suitable for industry applications have been discussed due to the complex scenarios of construction sites. In this paper, a fast and robust network based on a mutilscale Swin Transformer is proposed for safety helmet detection (FRSHNet) at construction sites, which contains the following contributions. Firstly, MAE-NAS with the variant of MobileNetV3’s MobBlock as a basic block is applied to implement feature extraction. Simultaneously, a multiscale Swin Transformer module is utilized to obtain the spatial and contexture relationships in the multiscale features. Subsequently, in order to meet the scheme requirements of real-time helmet detection, efficient RepGFPN are adopted to integrate refined multiscale features to form a pyramid structure. Extensive experiments were conducted on the publicly available Pictor-v3 and SHWD datasets. The experimental results show that FRSHNet consistently provided a favorable performance, outperforming the existing state-of-the-art models.

Improving IP Video Surveillance Systems: The Shift to Digital Networks and Security Challenges

In the numerical era, video surveillance schemes have fast evolved from analog technologies to IP-based systems, enabling widespread deployment in various environments such as streets, train stations, workplaces, factories, and homes. The incorporation of intelligent applications like facial recognition, object tracking, and event detection has expanded the utility of surveillance networks, allowing for the management of thousands of cameras across large geographic areas. This change has been facilitated by advancements in Internet Protocol (IP) technology, shifting surveillance from analog systems using coaxial cables to packet-switching systems in IPv6 and IPv4 networks. However, the extensive adoption of IP-based systems has introduced several challenges, including increased vulnerability to cyber attacks, bandwidth demands, processing power requirements, and network security issues. This research discovers the evolution of video surveillance systems, the technical and security challenges posed by IP-based networks, and potential solutions for improving their efficiency and security.

One level deep convolutional neural network for facial key points detection

Facial landmark detection has a lot of applications in face recognition, face alignment, facial expression recognition, video surveillance and security systems. In the existing literature, there are multiple methods utilizing convolutional neural networks (CNNs) that address this problem in various ways. In many cases, the models use a tree-like structure of CNNs to achieve better results. This paper proposes a combination of three parallel deep convolutional neural networks (DCNNs) to estimate the accurate localization of each keypoint. The first one focuses on the whole face to outperform five points, including the eyes, nose, and mouth corners. The second one focuses on the eyes-nose parts to outperform three points, specifically the eyes and nose. The last one focuses on the nose-mouth parts to outperform three points, namely the nose and mouth corners. Further, we combine all outputs of the three DCNNs and take the average value of each detected key point as the final output. In the first step, we improvthe the parameter efficiency and accuracy of each DCNNs through a set of experiments using the labeled face parts in-the-wild database (LFPW) and the helen facial feature dataset (Helen). Then, we demonstrate that our approach yields more accurate estimations of facial key points than two state-of-the-art methods and commercial software in terms of accuracy.

IMPLEMENTACIÓN DE UN SISTEMA DE COMUNICACIÓN Y VIDEOVIGILANCIA INTEGRADO EN UN UAV PARA MISIONES ESTRATEGICAS

Unmanned Aerial Vehicles (UAVs) have been widely used in the military for monitoring and national defence activities. One of the main advantages of using UAVs is that they reduce loss of life, as they can be controlled remotely or with the help of algorithms, allowing high-risk manoeuvres to be carried out without fatal consequences. One of the objectives of using UAV systems is to reduce the costs of acquiring and operating conventional aircraft. One of the main problems in the development of UAVs the design and dimensioning of communication systems. This paper presents the development of a communication system for strategic missions with the validation of the communication link calculated through the implementation of a video surveillance system integrated in a UAV. The equipment of the communication and video transmission system was based on free commercial hardware and free software that allows the sending of high-resolution images. The UAV operation tests have allowed maintaining the telemetry link and high-quality video transmission at approximately 5 km. Key Words: energy efficiency, historic districts, 3D cities, building typology, categorization tool

Video Anomaly Detection in Crime Analysis using Deep learning Architecture- A survey

The importance of developing automated video surveillance systems for public safety and security, particularly in crime analysis, has witnessed significant growth in recent years. This survey delves into the current landscape of automated video surveillance systems, emphasizing advancements in crime analysis and exploring existing methodologies and technologies. The study underscores the significance of employing deep learning models in video analysis. Furthermore, the study suggests a deep learning architecture to address the challenges of the existing methods. The goal of the suggested approach is to help security and law enforcement organizations quickly react to any dangers by precisely identifying unusual occurrences or actions in video sequences. The DenseNet-121 architecture is used for efficient spatial and temporal data acquisition from the video frames. This architecture is characterized by a dense connection structure in which all levels get feature mappings from all layers before them. The characteristics of DenseNet-121 can help in the accurate identification of anomalies in video streams and differentiate between normal and abnormal actions. In addition, the study also delves into the topic of using a cell structure with varied sizes to effectively split video sequences. This allows for flexible analysis and can accommodate different sorts of abnormalities. Anomaly detection accuracy can be further improved by adding size, motion, and location information to prediction and measurement models. This study serves as a foundation for the future research that aims to develop a more robust and efficient automated video surveillance solutions.

A Design Pattern of IAPVS Platform Based on Distributed Edge Computing

Abstract Traditional video surveillance is difficult to effectively cope with real-time interaction between end side devices of over ten thousand scale and data centers, and these devices are widely dispersed and deployed on a large scale. They cannot meet the needs of diverse scenarios monitoring and smart applications such as security, urban comprehensive management, ports, mines, water conservancy, power, industrial manufacturing and other fields. Based on the analysis of the problems in the popular intelligent video surveillance system, this paper proposes the design mode of an intelligent video surveillance intelligent analysis platform (IAPVS) based on distributed edge computing by improving the existing centralized video cloud computing architecture.This design pattern provides a new overall design solution for the development of platform of intelligent video analysis suitable for diverse scenarios and smart application fields.

Article: AI, Facial Recognition, and Policing: Business Opportunities and Legal Challenges: A UK Analysis with Glimpses of EU Law

It is a well-known fact that the health emergency caused an unprecedented surge in both information technology (IT) and digitalization. Not only did the virus spread, but technological advancement was also taking place: tracking ill people allowed for a semblance of normal social life to continue, despite the pandemic lockdown. One of the main consequences of the pandemic was the implementation of a large-scale video-surveillance system. This article examines, from a legal perspective, the relationship between video-surveillance activity and citizens’ rights, and attempts to balance the two sides of the same coin through a comparative analysis. To this end, reference is made to the UK case law and the controversy resolved by the European Court of Human Rights (ECtHR). Finally, an empirical approach on this matter is provided by the recent guidelines adopted by the European Data Protection Board (EDPB) and the EU Artificial Intelligence Act (AI Act). Artificial Intelligence, AI, Video-Surveillance, Automated Facial Recognition, Technology, Human Rights, Privacy, Policing, Rule of Law

AUTOMATED ACCESS CONTROL SYSTEMS

The paper provides an overview and analysis of the problems in the application of automated access control systems based on NFC. The capabilities of the Arduino platform are investigated and an access control system with an RFID-NFC module is developed on its basis. The use of automated access control systems in the modern world is becoming increasingly relevant. It involves a number of benefits that such systems can bring to businesses and other entities. These systems simplify and speed up the process of identifying a person, save time and increase the efficiency of security services, but still require human control. RFID-NFC technology is up-to-date, so the development and use of this technology for an access control system can solve the problem of unauthorized access to premises and/or controlled areas. Also, a company/enterprise can save money on identification devices, since various documents with an embedded NFC tag can be used as an identifier. For example, it can be a passport of a citizen of Ukraine, a passport of a citizen of Ukraine for travelling abroad, a certificate of registration, bank cards and a smartphone with NFC function, etc. An automated access control system is a set of advanced technologies and software designed to control and restrict access to a specific private territory/controlled area. The main purpose of an automated access control system is to ensure security, control and monitoring of access to a private territory/controlled area. Such systems are used to identify personnel, vehicles, etc. that have access to a restricted area/controlled zone. Access control systems can be integrated with other security systems, such as video surveillance systems, intrusion detection/prevention systems, etc.

Boosting robustness of network intrusion detection systems: A novel two phase defense strategy against untargeted white-box optimization adversarial attack

Machine Learning and Deep Learning based Network Intrusion Detection Systems (NIDS) serve as the backbone to protect computer networks against various cyber security threats. However, their susceptibility to adversarial attacks is the biggest challenge in front of the security administrator. In adversarial machine learning, the attacker crafts adversarial examples by adding small perturbations (tiny noise) in benign and legitimate network traffic data. These adversarial examples can easily deceive machine learning-based NIDS into producing incorrect classification and evading intrusion detection. This necessitates a robust NIDS that can detect and mitigate adversarial attacks immediately. In this research, we have proposed the two-phase defence method against the most powerful optimization-based adversarial attack, Carlini & Wagner (C&W). The two defence phases are the training and testing phase. In the training phase, we have used modified adversarial training using Gaussian Data Augmentation (GDA). In the testing phase, we have applied the Feature Squeezing (FS) method over the generated adversarial list before passing it to the robust NIDS model for final classification. The latest CIC-DDoS-2019 dataset is used to evaluate the proposed two-phase defence method effectively in terms of classification reports and confusion metrics. This research provides a comprehensive perspective of adversarial machine learning, its broad classification, and its defence approaches. The potential of the proposed research is not just limited to computer security. In fact, it can be adopted in other research domains, such as video analytics and surveillance systems.

RANGO: A Novel Deep Learning Approach to Detect Drones Disguising from Video Surveillance Systems

Video surveillance systems provide means to detect the presence of potentially malicious drones in the surroundings of critical infrastructures. In particular, these systems collect images and feed them to a deep-learning classifier able to detect the presence of a drone in the input image. However, current classifiers are not efficient in identifying drones that disguise themselves with the image background, e.g., hiding in front of a tree. Furthermore, video-based detection systems heavily rely on the image’s brightness, where darkness imposes significant challenges in detecting drones. Both these phenomena increase the possibilities for attackers to get close to critical infrastructures without being spotted and hence be able to gather sensitive information or cause physical damages, possibly leading to safety threats. In this article, we propose RANGO, a drone detection arithmetic able to detect drones in challenging images where the target is difficult to distinguish from the background. RANGO is based on a deep learning architecture that exploits a Preconditioning Operation (PREP) that highlights the target by the difference between the target gradient and the background gradient. The idea is to highlight features that will be useful for classification. After PREP, RANGO uses multiple convolution kernels to make the final decision on the presence of the drone. We test RANGO on a drone image dataset composed of multiple already-existing datasets to which we add samples of birds and planes. We then compare RANGO with multiple currently existing approaches to show its superiority. When tested on images with disguising drones, RANGO attains an increase of 6.6% mean Average Precision (mAP) compared to YOLOv5 solution. When tested on the conventional dataset, RANGO improves the mAP by approximately 2.2%, thus confirming its effectiveness also in the general scenario.

Detecting multiple moving objects for unusual crowd activities detection in video surveillance system

Unusual crowd activity detection is a challenging problem in surveillance video applications because feature extraction is difficult process in crowded scenes. The main objective of this research work is to detect unusual crowd activities and to detect unusual splits of moving objects. Various methods have been employed to address these challenges. However, there is still a lack of appropriate handling of this problem due to frames having occlusion, noise, and congestion. This paper proposes a novel clustering approach to detect unusual crowd activities. The proposed method consists of five phases including foreground extraction, foreground enhancement, foreground estimation, clustering crowds, and the Unusual Crowd Activities (UCA) model. The UCA model can find unusual crowd activities and unusual splits of moving objects using the Laplacian Matrix formulation. Two public datasets viz. PETS 2009 and UMN dataset are used for evaluating the proposed methodology. To estimate the effectiveness of the proposed work, several unusual event detection methods are compared with the proposed work results. The experimental results revealed that the proposed method gives better results than the existing methods.