Intelligent Surveillance System Research Articles

PEDESTRIAN ALERT SYSTEM USING DEEP LEARNING

Pedestrian detection is an essential and significant task in any intelligent video surveillance system, as it provides the fundamental information for semantic understanding of the video footages. It has an obvious extension to automotive applications due to the potential for improving safety systems. Many car manufacturers (e.g., Volvo, Ford, GM, Nissan) offer this as an ADAS option in 2017. It is used in many cars and vehicles for automatic driver assistance systems. But when the ADAS system detects pedestrians but if fails to apply brakes on time accidents might happened. To overcome this situation our project provides an audio alert to the human passengers or driver in the vehicle so that they can apply manual brakes. In this project we will require Python libraries such as Pytorch and OpenCV. Pytorch is a widely used Machine Learning library. It is popular for the YOLO (You Only Look Once) algorithm which is built for Object Detection, we are using the YOLO algorithm and customize it to detect objects on a pedestrian dataset. Python has a library pyttsx3, that is capable to convert text-to-speech offline. Itextracts the label from the detected pedestrians in the video and converts the text label into speech. The algorithm detects the pedestrians when they are in a close approach to the vehicles and sends the identifies labels to the pyttsx3 speaker engine. An audio alert is generated by the engine and alerts the passengers or drivers. Keywords - YOLO, Pytorch, Pyttsx3, Object Detection, Python.

Automated Stand-Alone Bot for Intruder Detection

The Automated Standalone Bot for Intruder Detection System represents a cutting-edge approach to bolstering security measures through the deployment of an intelligent surveillance system. By harnessing sophisticated computer vision techniques, this system is capable of vigilantly monitoring its surroundings and swiftly identifying potential security threats with remarkable accuracy. Through continuous real-time analysis of video feeds and the application of advanced image processing algorithms, the bot can promptly detect any unauthorized individuals or suspicious activities within its designated area of coverage. Upon detecting a potential intruder, the system initiates an immediate response protocol, which may include activating alarms, notifying security personnel, or implementing deterrent measures to deter any further intrusion attempts. What sets this system apart is its ability to continually refine its intruder detection capabilities over time, leveraging machine learning algorithms to adapt to evolving threats and minimize false positives. Moreover, the system demonstrates remarkable discernment by distinguishing between human subjects, animals, and other objects, thereby reducing the occurrence of false alarms and enhancing its overall reliability. This comprehensive approach not only fortifies security measures but also instills a greater sense of confidence in the system's ability to safeguard the premises effectively. Key Words: Automated Standalone Bot, Intruder Detection System, Security Measures, Intelligent System, Computer Vision Techniques, Surveillance, Real-time Analysis, Video Feeds, Image Processing Algorithms, Swift Detection, Unauthorized Individuals, Suspicious Activities, Response Protocol, Alarms, Notification, Deterrent Measures, Machine Learning Algorithms, Adaptation, False Positives, Reliability, Human Subjects, Animals, False Alarms, Premises Security

Advancements in Intelligent Video Surveillance for Public Safety: A Comprehensive Review

Abstract: Intelligent video surveillance systems have emerged as invaluable tools for ensuring public safety in a variety of settings, from essential infrastructure to urban environments. These systems, which are driven by state-of-the-art technology like computer vision, machine learning, and artificial intelligence, provide unmatched real-time threat detection, analysis, and response capabilities. This study provides an in-depth analysis of the most recent developments in intelligent video surveillance for public safety, covering key technology, useful uses, enduring difficulties, and potential directions. This paper aims to provide a thorough knowledge of the significant implications and emerging opportunities of intelligent video surveillance in protecting public places from potential dangers through an examination of recent research insights and industry advancements. This article seeks to offer a guide for researchers, practitioners, policymakers, and stakeholders navigating the ever-changing field of intelligent video surveillance for public safety by exploring the evolving trends, emerging technologies, and practical applications.

Real - time Anomaly Target Detection and Recognition in Intelligent Surveillance Systems based on SLAM

Real - time Anomaly Target Detection and Recognition in Intelligent Surveillance Systems based on SLAM

Intelligent Video Surveillance System

Intelligent Video Surveillance Systems (IVSS) have emerged as critical tools for ensuring safety and security in various environments. This paper presents an IVSS capable of real-time detection and analysis of critical events, including falls and vehicle crashes. The proposed system employs state-of-the-art deep learning techniques, specifically the YOLO (You Only Look Once) object detection algorithm, to analyze video streams obtained from diverse sources such as surveil- lance cameras and recorded videos. For fall detection, the system utilizes the aspect ratio of detected persons’ bounding boxes to distinguish between falls and normal activities. Vehicle crashes are identified by detecting cars in each frame and assessing the proximity and intersection of their bounding boxes. The system operates in a continuous loop, processing video frames in real-time and generating alerts when critical events are detected. Experimental results demon- strate the effectiveness and reliability of the proposed IVSS in enhancing safety and security across different environments, including public spaces, workplaces, and transportation systems.

Generalized Video Anomaly Event Detection: Systematic Taxonomy and Comparison of Deep Models

Video Anomaly Detection (VAD) serves as a pivotal technology in the intelligent surveillance systems, enabling the temporal or spatial identification of anomalous events within videos. While existing reviews predominantly concentrate on conventional unsupervised methods, they often overlook the emergence of weakly-supervised and fully-unsupervised approaches. To address this gap, this survey extends the conventional scope of VAD beyond unsupervised methods, encompassing a broader spectrum termed Generalized Video Anomaly Event Detection (GVAED). By skillfully incorporating recent advancements rooted in diverse assumptions and learning frameworks, this survey introduces an intuitive taxonomy that seamlessly navigates through unsupervised, weakly-supervised, supervised and fully-unsupervised VAD methodologies, elucidating the distinctions and interconnections within these research trajectories. In addition, this survey facilitates prospective researchers by assembling a compilation of research resources, including public datasets, available codebases, programming tools, and pertinent literature. Furthermore, this survey quantitatively assesses model performance, delves into research challenges and directions, and outlines potential avenues for future exploration.

Knowledge Development Trajectories of Intelligent Video Surveillance Domain: An Academic Study Based on Citation and Main Path Analysis.

Smart city is an area where the Internet of things is used effectively with sensors. The data used by smart city can be collected through the cameras, sensors etc. Intelligent video surveillance (IVS) systems integrate multiple networked cameras for automatic surveillance purposes. Such systems can analyze and monitor video data and perform automatic functions required by users. This study performed main path analysis (MPA) to explore the development trends of IVS research. First, relevant articles were retrieved from the Web of Science database. Next, MPA was performed to analyze development trends in relevant research, and g-index and h-index values were analyzed to identify influential journals. Cluster analysis was then performed to group similar articles, and Wordle was used to display the key words of each group in word clouds. These key words served as the basis for naming their corresponding groups. Data mining and statistical analysis yielded six major IVS research topics, namely video cameras, background modeling, closed-circuit television, multiple cameras, person reidentification, and privacy, security, and protection. These topics can boost the future innovation and development of IVS technology and contribute to smart transportation, smart city, and other applications. According to the study results, predictions were made regarding developments in IVS research to provide recommendations for future research.

Deep Learning-Based Intelligent Video Surveillance System for Real-Time Motion Detection

The proliferation of surveillance systems demands advanced solutions to effectively monitor and analyze video feeds for security and safety purposes. In this paper, we propose a Deep Learning-Based Intelligent Video Surveillance System (DIVSS) designed to enhance traditional Closed-Circuit Television (CCTV) setups. DIVSS integrates state-of-the-art deep learning techniques for real-time motion detection and object recognition, enabling proactive threat detection and automated response mechanisms. The system utilizes convolutional neural networks (CNNs) for robust feature extraction and classification, enabling accurate identification of objects of interest amidst complex backgrounds and varying[1] lighting conditions. Furthermore, DIVSS incorporates region of interest (ROI) monitoring to focus attention on specific areas within the surveillance footage, optimizing computational resources and improving response times. We evaluate the performance of DIVSS through comprehensive experiments conducted on diverse surveillance scenarios, demonstrating its effectiveness in enhancing security measures and providing actionable[3] insights for surveillance operators. The proposed system represents a significant advancement in video surveillance technology, offering unparalleled capabilities for proactive threat detection and real-time response in dynamic environments. Keywords— Deep learning has transformed video surveillance, enabling real-time motion detection and object recognition using Convolutional Neural Networks (CNNs). This technology provides proactive threat detection, intelligent systems for ROI monitoring, and enhances overall security measures.

HF-YOLO: Advanced Pedestrian Detection Model with Feature Fusion and Imbalance Resolution

Pedestrian detection is crucial for various applications, including intelligent transportation and video surveillance systems. Although recent research has advanced pedestrian detection models like the YOLO series, they still face limitations in handling diverse pedestrian scales, leading to performance challenges. To address these issues, we propose HF-YOLO, an advanced pedestrian detection model. HF-YOLO tackles the complexities of pedestrian detection in complex scenes by addressing scale variations and occlusions among pedestrians. In the feature fusion stage, our algorithm leverages both shallow localization information and deep semantic information. This involves fusing P2 layer features and adding a high-resolution detection layer, significantly improving the detection of small-scale pedestrians and occluded instances. To enhance feature representation, HF-YOLO incorporates the HardSwish activation function, introducing more non-linear factors and strengthening the model’s ability to represent complex and discriminative features. Additionally, to address regression imbalance, a balance factor is introduced to the CIoU loss function. This modification effectively resolves the imbalance problem and enhances pedestrian localization accuracy. Experimental results demonstrate the effectiveness of our proposed algorithm. HF-YOLO achieves notable improvements, including a 3.52% increase in average precision, a 1.35% boost in accuracy, and a 4.83% enhancement in recall. Moreover, the algorithm maintains real-time performance with a detection time of 8.5ms, meeting the stringent requirements of real-time applications.

A Design Pattern of IAPVS Platform Based on Distributed Edge Computing

Abstract Traditional video surveillance is difficult to effectively cope with real-time interaction between end side devices of over ten thousand scale and data centers, and these devices are widely dispersed and deployed on a large scale. They cannot meet the needs of diverse scenarios monitoring and smart applications such as security, urban comprehensive management, ports, mines, water conservancy, power, industrial manufacturing and other fields. Based on the analysis of the problems in the popular intelligent video surveillance system, this paper proposes the design mode of an intelligent video surveillance intelligent analysis platform (IAPVS) based on distributed edge computing by improving the existing centralized video cloud computing architecture.This design pattern provides a new overall design solution for the development of platform of intelligent video analysis suitable for diverse scenarios and smart application fields.

Region-based convolutional neural networks for occluded person re-identification

In a variety of applications, including intelligent surveillance systems, targeted tracking, and assistive human-following robots, the ability to accurately identify individuals even when they are partially obscured is imperative. Such Continuous person tracking is complicated by the close similarity between the appearance of people and target occlusions. This study addresses this significant challenge by proposing a two-step, detection-first approach that uses a region-based convolutional neural network (R-CNN) as the re-identification (re-ID)solution. The model is specifically trained to detect occluded persons at different levels of occlusion before forwarding the image for the re-ID process. Three occluded-specific datasets are selected to evaluate the model's effectiveness in detecting occluded people. There are 379 distinct people in total, and each has five images obstructed from different angles. A sample of the data is taken to simulate various environment settings, and new data points are generated with different degrees of occlusion to assess how well the model performs under varying levels of obstruction. The findings demonstrate that the proposed person re-ID model is reliable in most circumstances, correctly re-identifying at 74% (Rank-1) and 90% (Rank-5). Although there is a decrease in accuracy as the number of distinctive people in the dataset increases, this does not significantly impact the tracking performance in various applications, which are expected to recognize a single person or a small group of individuals. Future works will explore refining similarity matching algorithms by delving into robust image comparison techniques, thereby addressing the challenges presented by occlusions. A critical aspect is to assess the model under diverse lighting conditions and investigate scenarios with multiple individuals in a frame. It is also beneficial to exploit high-resolution datasets, such as DukeMTMC-reID, and integrate finer contextual details, like clothing or carried objects. These collective efforts are essential for optimizing the model’s efficacy in practical applications and advancing person re-ID technologies.

Spatiotemporal Masked Autoencoder with Multi-Memory and Skip Connections for Video Anomaly Detection

Video anomaly detection is a critical component of intelligent video surveillance systems, extensively deployed and researched in industry and academia. However, existing methods have a strong generalization ability for predicting anomaly samples. They cannot utilize high-level semantic and temporal contextual information in videos, resulting in unstable prediction performance. To alleviate this issue, we propose an encoder–decoder model named SMAMS, based on spatiotemporal masked autoencoder and memory modules. First, we represent and mask some of the video events using spatiotemporal cubes. Then, the unmasked patches are inputted into the spatiotemporal masked autoencoder to extract high-level semantic and spatiotemporal features of the video events. Next, we add multiple memory modules to store unmasked video patches of different feature layers. Finally, skip connections are introduced to compensate for crucial information loss caused by the memory modules. Experimental results show that the proposed method outperforms state-of-the-art methods, achieving AUC scores of 99.9%, 94.8%, and 78.9% on the UCSD Ped2, CUHK Avenue, and Shanghai Tech datasets.

A shadow enabled non-invasive probe for multi-feature intelligent liquid surveillance system.

Liquid detection probes used to identify the features of liquids show great promise in a variety of important applications. However, some challenges, such as sample contamination by direct contact with the liquid, the requirement of additional signal emitters, and complex fabrication, hindered the development of liquid detection probes. Here, we developed a non-invasive shadow probe (SP) for a multi-feature intelligent liquid surveillance system (ILSS). The self-powered SP with the working mechanism of the shadow effect can detect the features of liquids by analyzing the variations of liquid shadows such as the area, wavelength, and brightness. The exact resolution (5 different colors, 6 different concentrations, 6 different levels, 100% accuracy) and fast response time (0.2 ms) are shown by the SP under ambient light conditions (even in 0.003 sun). The ILSS, which integrated the SPs with signal processing circuits and applied the artificial intelligence (AI) technique, successfully detects and synoptically learns about liquids simultaneously. The in-real time ILSS reaches a test accuracy of 99.3% for 10 types of liquids with multiple features. This work showcases a promising solution for non-invasive multi-feature liquid detection, displaying great potential for future applications.

Design and Practical Application of a Cost Effective Intelligent Female Surveillance System Using GPS, GSM, and Arduino Technology

This paper suggests a cost effective, portable and intelligent surveillance system for women that they can utilize in risky or emergency situations. In this design, an Arduino Nano microcontroller, a GPS module, and a GSM module are the main parts used. Whenever an emergency arises, the user can instantly send an emergency message to a preset phone number along with their current location. The emergency activation button is in a handy place on the gadget, which is tiny enough to carry in a pocket or purse. This device will improve women's protection and security, particularly in places with high crime rates or poor public safety.

Toward Real Scenery: A Lightweight Tomato Growth Inspection Algorithm for Leaf Disease Detection and Fruit Counting

The deployment of intelligent surveillance systems to monitor tomato plant growth poses substantial challenges due to the dynamic nature of disease patterns and the complexity of environmental conditions such as background and lighting. In this study, an integrated cascade framework that synergizes detectors and trackers was introduced for the simultaneous identification of tomato leaf diseases and fruit counting. We applied an autonomous robot with smartphone camera to collect images for leaf disease and fruits in greenhouses. Further, we improved the deep learning network YOLO-TGI by incorporating Ghost and CBAM modules, which was trained and tested in conjunction with premier lightweight detection models like YOLOX and NanoDet in evaluating leaf health conditions. For the cascading with various base detectors, we integrated state-of-the-art trackers such as Byte-Track, Motpy, and FairMot to enable fruit counting in video streams. Experimental results indicated that the combination of YOLO-TGI and Byte-Track achieved the most robust performance. Particularly, YOLO-TGI-N emerged as the model with the least computational demands, registering the lowest FLOPs at 2.05 G and checkpoint weights at 3.7 M, while still maintaining a mAP of 0.72 for leaf disease detection. Regarding the fruit counting, the combination of YOLO-TGI-S and Byte-Track achieved the best R 2 of 0.93 and the lowest RMSE of 9.17, boasting an inference speed that doubles that of the YOLOX series, and is 2.5 times faster than the NanoDet series. The developed network framework is a potential solution for researchers facilitating the deployment of similar surveillance models for a broad spectrum of fruit and vegetable crops.

Motion Detection Car using WiFi Cam and NodeMCU

Our project combines a WiFi camera and NodeMCU-controlled car to establish an intelligent surveillance system. The WiFi camera captures and streams real-time video to a server while a NodeMCU-controlled car autonomously responds to detected motion. When motion is identified, the system instructs the car to investigate the source, offering a dynamic approach to surveillance. This innovative solution presents cost-effective and flexible applications across diverse environments, including homes, offices, and outdoor spaces. It harnesses the power of WiFi technology for efficient video streaming and integrates motion-based control for enhanced security and responsiveness. The system's adaptability and real-time monitoring capabilities make it a practical choice for a wide range of surveillance needs, ensuring both convenience and peace of mind for users

A visible-infrared clothes-changing dataset for person re-identification in natural scene

Person re-identification (Re-ID) has been widely used in intelligent surveillance systems, aiming at retrieving specific pedestrian images across different cameras. Although existing person Re-ID methods have achieved inspiring success, there are still limitations in practical monitoring system applications. To narrow the gap with the practical application, we introduce the cross-modality person Re-ID problem in the clothes-changing scene. Meanwhile, we construct the first Visible-Infrared Clothes-Changing (NEU-VICC) dataset, which contained 16632 RGB images and 8374 infrared images of 107 pedestrians. The critical challenge of the cross-modality person Re-ID problem in the clothes-changing scene lies in the vast modality discrepancy and the intra-class discrepancy caused by changing clothes. So, we propose a novel Semantic-Constraint Clothes-Changing Augmentation Network (SC3ANet) based on current cross-modality person Re-ID methods to solve this problem. Specifically, we design a semantic-constraint clothes-changing module that guides the model to learn clothes-irrelevant features by randomly changing pedestrians' clothes. In addition, we devise a dual-granularity constraint loss module to mitigate inter-modality and intra-class differences. Experiments on our NEU-VICC dataset show that the SC3ANet achieves the best results. The dataset and code are available at: https://github.com/VDT-2048/NEU-VICC.

Suspicious Activity Classification in Classrooms using Deep Learning

Video processing is attracting the attention of both research and industry. The existence of intelligent surveillance cameras with high processing power has paved the way for designing intelligent visual surveillance systems. Along with analyzing video for information recovery, it is nowadays used to analyze live surveillance video to detect activities. These systems are implemented in real time. The proposed work's goal is to create a method that can examine and discover suspicious behaviors in the lecture room environment. Video analytics offers the most efficient answer because it enables pointing an occasion and retrieves applicable statistics from the video recorded. The method aims to identify suspicious activities like fighting, sleeping, looking elsewhere, eating, etc. that the students might be doing. The proposed method involves breaking a video input into frames and converting it into image data because the model has been trained on images collected from the internet. Several models were tested and experimented with, including efficientnet_b2, spnasnet_100, efficientnet_b3, and mobilenetv3_large_100. Parameters such as the Learning Rate were optimized to find out the best method and create a system with the best results.

CMOT: A cross-modality transformer for RGB-D fusion in person re-identification with online learning capabilities

Person re-identification (reID) is a crucial aspect of intelligent surveillance systems, enabling the recognition of individuals across non-overlapping camera views. As compared to the RGB modality, RGB-D based reID has the potential to achieve robust and high performance by leveraging rich complementary features of both modalities, making it applicable in various occluded scenarios. However, current multimodal reID approaches often rely on late fusion or feature-level fusion techniques to combine multiple modalities, which limits their ability to proficiently exploit complementary visual and depth-related semantic information and capture complex interactions between unimodal features. To address these limitations, this paper introduces a cross-modality online transformer (CMOT) for RGB-D based person reID with online learning capabilities, which effectively utilizes both RGB and depth modalities for the extraction of spatio-temporal features and fuses across modalities. Our CMOT is composed of three main components: (1) a hypothesis generation module based on a person detector and tracker, (2) dual-stream feature extractors via convolutional neural networks (CNNs), (3) and a fusion transformer based on a self-attention-driven self-attentive modality refinement module (SAMR) and a cross-attention-driven cross-attentive modality interaction module (CAMI) to refine and fuse RGB-D complementary features extracted from the dual-stream, RGB and depth stream, CNNs. Additionally, we introduce a bottleneck enhancement feed-forward block to enhance the model’s representation capability within SAMR and CAMI, significantly reducing parameters and computations compared to the traditional feed-forward network. Moreover, we design the triplet loss function with distance measuring ability for incorporating online learning and finally CMOT works as a few-shot network for reID. Experimental results on three RGB-D person re-identification datasets, namely BIWI RGBD-ID, RobotPKU RGBD-ID, and TVPR2, demonstrate the effectiveness and robustness of CMOT.

An improved YOLOv5 for real-time human detection in infrared images

<span>The recent prominence of using infrared technology for human detection has garnered attention, particularly for applications in intelligent video surveillance and self-driving systems. This technology offers advantages in adverse weather conditions and night vision. However, within deep learning, the challenge of variable illumination during human detection persists. This study presents a novel method for identifying individuals in thermal images by enhancing the you only look once (YOLOv5s) algorithm. The approach incorporates the Bi-directional feature pyramid network (BiFPN) and the convolutional block attention module (CBAM) to improve the model’s feature integration and extraction capabilities. Evaluation on established thermal imaging datasets confirms the method’s superiority over state-of-the-art convolutional neural networks (CNN) based techniques, achieving remarkable precision 99.1% and recall 96.9%.</span>