Intelligent Video Surveillance System Research Articles

PEDESTRIAN ALERT SYSTEM USING DEEP LEARNING

Pedestrian detection is an essential and significant task in any intelligent video surveillance system, as it provides the fundamental information for semantic understanding of the video footages. It has an obvious extension to automotive applications due to the potential for improving safety systems. Many car manufacturers (e.g., Volvo, Ford, GM, Nissan) offer this as an ADAS option in 2017. It is used in many cars and vehicles for automatic driver assistance systems. But when the ADAS system detects pedestrians but if fails to apply brakes on time accidents might happened. To overcome this situation our project provides an audio alert to the human passengers or driver in the vehicle so that they can apply manual brakes. In this project we will require Python libraries such as Pytorch and OpenCV. Pytorch is a widely used Machine Learning library. It is popular for the YOLO (You Only Look Once) algorithm which is built for Object Detection, we are using the YOLO algorithm and customize it to detect objects on a pedestrian dataset. Python has a library pyttsx3, that is capable to convert text-to-speech offline. Itextracts the label from the detected pedestrians in the video and converts the text label into speech. The algorithm detects the pedestrians when they are in a close approach to the vehicles and sends the identifies labels to the pyttsx3 speaker engine. An audio alert is generated by the engine and alerts the passengers or drivers. Keywords - YOLO, Pytorch, Pyttsx3, Object Detection, Python.

Advancements in Intelligent Video Surveillance for Public Safety: A Comprehensive Review

Abstract: Intelligent video surveillance systems have emerged as invaluable tools for ensuring public safety in a variety of settings, from essential infrastructure to urban environments. These systems, which are driven by state-of-the-art technology like computer vision, machine learning, and artificial intelligence, provide unmatched real-time threat detection, analysis, and response capabilities. This study provides an in-depth analysis of the most recent developments in intelligent video surveillance for public safety, covering key technology, useful uses, enduring difficulties, and potential directions. This paper aims to provide a thorough knowledge of the significant implications and emerging opportunities of intelligent video surveillance in protecting public places from potential dangers through an examination of recent research insights and industry advancements. This article seeks to offer a guide for researchers, practitioners, policymakers, and stakeholders navigating the ever-changing field of intelligent video surveillance for public safety by exploring the evolving trends, emerging technologies, and practical applications.

Intelligent Video Surveillance System

Intelligent Video Surveillance Systems (IVSS) have emerged as critical tools for ensuring safety and security in various environments. This paper presents an IVSS capable of real-time detection and analysis of critical events, including falls and vehicle crashes. The proposed system employs state-of-the-art deep learning techniques, specifically the YOLO (You Only Look Once) object detection algorithm, to analyze video streams obtained from diverse sources such as surveil- lance cameras and recorded videos. For fall detection, the system utilizes the aspect ratio of detected persons’ bounding boxes to distinguish between falls and normal activities. Vehicle crashes are identified by detecting cars in each frame and assessing the proximity and intersection of their bounding boxes. The system operates in a continuous loop, processing video frames in real-time and generating alerts when critical events are detected. Experimental results demon- strate the effectiveness and reliability of the proposed IVSS in enhancing safety and security across different environments, including public spaces, workplaces, and transportation systems.

Knowledge Development Trajectories of Intelligent Video Surveillance Domain: An Academic Study Based on Citation and Main Path Analysis.

Smart city is an area where the Internet of things is used effectively with sensors. The data used by smart city can be collected through the cameras, sensors etc. Intelligent video surveillance (IVS) systems integrate multiple networked cameras for automatic surveillance purposes. Such systems can analyze and monitor video data and perform automatic functions required by users. This study performed main path analysis (MPA) to explore the development trends of IVS research. First, relevant articles were retrieved from the Web of Science database. Next, MPA was performed to analyze development trends in relevant research, and g-index and h-index values were analyzed to identify influential journals. Cluster analysis was then performed to group similar articles, and Wordle was used to display the key words of each group in word clouds. These key words served as the basis for naming their corresponding groups. Data mining and statistical analysis yielded six major IVS research topics, namely video cameras, background modeling, closed-circuit television, multiple cameras, person reidentification, and privacy, security, and protection. These topics can boost the future innovation and development of IVS technology and contribute to smart transportation, smart city, and other applications. According to the study results, predictions were made regarding developments in IVS research to provide recommendations for future research.

Deep Learning-Based Intelligent Video Surveillance System for Real-Time Motion Detection

The proliferation of surveillance systems demands advanced solutions to effectively monitor and analyze video feeds for security and safety purposes. In this paper, we propose a Deep Learning-Based Intelligent Video Surveillance System (DIVSS) designed to enhance traditional Closed-Circuit Television (CCTV) setups. DIVSS integrates state-of-the-art deep learning techniques for real-time motion detection and object recognition, enabling proactive threat detection and automated response mechanisms. The system utilizes convolutional neural networks (CNNs) for robust feature extraction and classification, enabling accurate identification of objects of interest amidst complex backgrounds and varying[1] lighting conditions. Furthermore, DIVSS incorporates region of interest (ROI) monitoring to focus attention on specific areas within the surveillance footage, optimizing computational resources and improving response times. We evaluate the performance of DIVSS through comprehensive experiments conducted on diverse surveillance scenarios, demonstrating its effectiveness in enhancing security measures and providing actionable[3] insights for surveillance operators. The proposed system represents a significant advancement in video surveillance technology, offering unparalleled capabilities for proactive threat detection and real-time response in dynamic environments. Keywords— Deep learning has transformed video surveillance, enabling real-time motion detection and object recognition using Convolutional Neural Networks (CNNs). This technology provides proactive threat detection, intelligent systems for ROI monitoring, and enhances overall security measures.

HF-YOLO: Advanced Pedestrian Detection Model with Feature Fusion and Imbalance Resolution

Pedestrian detection is crucial for various applications, including intelligent transportation and video surveillance systems. Although recent research has advanced pedestrian detection models like the YOLO series, they still face limitations in handling diverse pedestrian scales, leading to performance challenges. To address these issues, we propose HF-YOLO, an advanced pedestrian detection model. HF-YOLO tackles the complexities of pedestrian detection in complex scenes by addressing scale variations and occlusions among pedestrians. In the feature fusion stage, our algorithm leverages both shallow localization information and deep semantic information. This involves fusing P2 layer features and adding a high-resolution detection layer, significantly improving the detection of small-scale pedestrians and occluded instances. To enhance feature representation, HF-YOLO incorporates the HardSwish activation function, introducing more non-linear factors and strengthening the model’s ability to represent complex and discriminative features. Additionally, to address regression imbalance, a balance factor is introduced to the CIoU loss function. This modification effectively resolves the imbalance problem and enhances pedestrian localization accuracy. Experimental results demonstrate the effectiveness of our proposed algorithm. HF-YOLO achieves notable improvements, including a 3.52% increase in average precision, a 1.35% boost in accuracy, and a 4.83% enhancement in recall. Moreover, the algorithm maintains real-time performance with a detection time of 8.5ms, meeting the stringent requirements of real-time applications.

A Design Pattern of IAPVS Platform Based on Distributed Edge Computing

Abstract Traditional video surveillance is difficult to effectively cope with real-time interaction between end side devices of over ten thousand scale and data centers, and these devices are widely dispersed and deployed on a large scale. They cannot meet the needs of diverse scenarios monitoring and smart applications such as security, urban comprehensive management, ports, mines, water conservancy, power, industrial manufacturing and other fields. Based on the analysis of the problems in the popular intelligent video surveillance system, this paper proposes the design mode of an intelligent video surveillance intelligent analysis platform (IAPVS) based on distributed edge computing by improving the existing centralized video cloud computing architecture.This design pattern provides a new overall design solution for the development of platform of intelligent video analysis suitable for diverse scenarios and smart application fields.

Spatiotemporal Masked Autoencoder with Multi-Memory and Skip Connections for Video Anomaly Detection

Video anomaly detection is a critical component of intelligent video surveillance systems, extensively deployed and researched in industry and academia. However, existing methods have a strong generalization ability for predicting anomaly samples. They cannot utilize high-level semantic and temporal contextual information in videos, resulting in unstable prediction performance. To alleviate this issue, we propose an encoder–decoder model named SMAMS, based on spatiotemporal masked autoencoder and memory modules. First, we represent and mask some of the video events using spatiotemporal cubes. Then, the unmasked patches are inputted into the spatiotemporal masked autoencoder to extract high-level semantic and spatiotemporal features of the video events. Next, we add multiple memory modules to store unmasked video patches of different feature layers. Finally, skip connections are introduced to compensate for crucial information loss caused by the memory modules. Experimental results show that the proposed method outperforms state-of-the-art methods, achieving AUC scores of 99.9%, 94.8%, and 78.9% on the UCSD Ped2, CUHK Avenue, and Shanghai Tech datasets.

An improved YOLOv5 for real-time human detection in infrared images

<span>The recent prominence of using infrared technology for human detection has garnered attention, particularly for applications in intelligent video surveillance and self-driving systems. This technology offers advantages in adverse weather conditions and night vision. However, within deep learning, the challenge of variable illumination during human detection persists. This study presents a novel method for identifying individuals in thermal images by enhancing the you only look once (YOLOv5s) algorithm. The approach incorporates the Bi-directional feature pyramid network (BiFPN) and the convolutional block attention module (CBAM) to improve the model’s feature integration and extraction capabilities. Evaluation on established thermal imaging datasets confirms the method’s superiority over state-of-the-art convolutional neural networks (CNN) based techniques, achieving remarkable precision 99.1% and recall 96.9%.</span>

Research on Video Anomaly Detection Based on Cascaded Memory-augmented Autoencoder

<p>Video anomaly detection (VAD) is crucial in public safety and intelligent video surveillance systems and has been widely researched and applied in academia. This paper proposes a video anomaly detection method based on the Cascaded Memory-augmented Autoencoder (CMAAE). CMAAE stores feature prototypes of standard samples in a memory pool and embed multiple memory-enhancing modules in the encoder-decoder structure. SE attention is introduced into memory modules to improve their performance, and skip connections are used to share attention weights among memory modules, enabling the model to learn more comprehensive feature information and enhance the quality of reconstructed video frames. Multiple loss constraint models are used during training to improve anomaly detection accuracy. CMAAE achieves outstanding performance of 99.2% on the UCSD Ped2 dataset and 89.4% on the CUHK Avenue dataset, demonstrating the effectiveness of our approach.</p> <p> </p>

Ancilia: Scalable Intelligent Video Surveillance for the Artificial Intelligence of Things

With the advancement of vision-based artificial intelligence, the proliferation of the Internet of Things connected cameras, and the increasing societal need for rapid and equitable security, the demand for accurate real-time intelligent surveillance has never been higher. This article presents Ancilia, an end-to-end scalable, intelligent video surveillance system for the Artificial Intelligence of Things. Ancilia brings state-of-theart artificial intelligence to real-world surveillance applications while respecting ethical concerns and performing high-level cognitive tasks in real-time. Ancilia aims to revolutionize the surveillance landscape, to bring more effective, intelligent, and equitable security to the field, resulting in safer and more secure communities without requiring people to compromise their right to privacy.

INTELLIGENT VIDEO SURVEILLANCE SYSTEMS: A SURVEY

Over the years, the need for intelligent video surveillance has increased in other to enhance security and safety in the society. Government, private organization and individual need to make sure its properties are kept safe from intruders and as such intelligent video surveillance plays a key role in ensuring that this is achieved. Intelligent video surveillance is embedded with the capability of providing real time intelligent surveillance and also automatically provide analysis of video and image data without human operation. In the development of such systems, computer vision, machine learning and deep learning plays a vital role in achieving this. Therefore, this paper provides a general survey on intelligent video surveillance system, overview of background concept and discussion on object detections and classification, tracking and deep networks. Also, this paper presents an efficient and faster object detection and classification techniques for intelligent video surveillance.

A conditioned reflex action embedded associative context learning‐based energy efficient paradigm in smart city milieu

AbstractAn intelligent video surveillance system is crucial to enhance public safety, crime prevention, traffic, and crowd management in a smart city milieu. Situational awareness is an essential aspect of these surveillance systems and it is inferred through underlying context aware frameworks. However, these systems may not possess the ability to proactively disseminate the real‐time context among its sensor nodes. Moreover, in the specific conditions of occurrence of related or repeated events, these systems may also perform inefficiently through afresh context processing and disseminate cycles, without learning from the relevant context that has already been occurred and processed by the system. It leads to deteriorated performance, especially delay in reaction, overwhelmed processing, and energy expenditures. Therefore, to counter such issues, this research work proposes an energy efficient situational aware framework deployed in visual sensors network that is incorporated with context associative learning. System observes currently occurring context at each instance of an event. Overtime, context is refined and stored in context database. Such mechanism empowers the system to learn from previous experiences and develop relationship among the subsequent events that is embedded through this associative (adaptive) learning. Eventually, each event is processed through intelligent resource allocation, supported through mechanism of context learning that further illustrates the independent functions of reduced processing and improved (rapid) decision making resulting in evolution of energy efficient computing paradigm. Ultimately, the capability of learned reflex‐action is induced through introspectively evolved context of the system in entirety and against specific condition of recurred situation depicting minimum energy expenditure.

Intelligent Video Surveillance Mechanisms for Abnormal Activity Recognition in Real-Time: A Systematic Literature Review

Video surveillance plays a crucial role in securing indoor and outdoor locations in today's unreliable world, particularly in real-time applications for behaviour detection, comprehension, and labelling activities as normal or suspicious. For example, in the development of automated video surveillance systems, smart video reconnaissance systems based on picture recognition and activity recognition that detect violent behaviours is basic to forestalling wrongdoings and giving public security. According to the literature, Artificial Intelligent, Machine Learning, and deep portrayal-based approaches have been effectively utilized in image recognition and human activity observation tasks. In this literature review, a 3D convolution neural network based on deep learning is used as the proposed methodology. Thus, this article completed a Systematic Literature Review (SLR) in light of intelligent video surveillance to real-time identify abnormal activities from 2016 to 2021. In this current study, 50 research papers were considered and based on the screen filtering, the most suitable 16 papers were filtered based on intelligent video surveillance and real-time abnormal activities. Furthermore, this study identifies potential areas for improvement in intelligent video surveillance systems that can enhance public safety and security, underscoring the importance of ongoing research in this field.

Abnormal Activity Recognition in Private Places Using Deep Learning: A Survey

Abstract: The method of analysing human motion using computer and machine vision technologies is known as "human activity recognition," or HAR. One of the applications of human activity recognition in security systems is anomaly detection. Surveillance cameras have been widely placed as the foundation for video analysis as the demand for security has grown. Identifying aberrant behaviour necessitates considerable human effort, which is one of the major challenges in surveillance video analysis. It is important to set up video recording in order to detect unusual activity automatically. Our intelligent video surveillance system can detect an abnormality in a video using deep learning technologies. Real-time detection of activities is also conceivable, and these video frames will be saved in the system as images for the user to study. The proposed Abnormal Activity Recognition system was designed with the purpose of finding and detecting anomalies in the financial industry, especially in an ATM context, using a live stream. The first part of the research focuses on the use of image deep learning algorithms to recognise different products and detect anomalous behaviour utilizing ATM monitoring systems

Initial validation of an intelligent video surveillance system for automatic detection of dairy cattle lameness.

Lameness is a major welfare challenge facing the dairy industry worldwide. Monitoring herd lameness prevalence, and early detection and therapeutic intervention are important aspects of lameness control in dairy herds. The objective of this study was to evaluate the performance of a commercially available video surveillance system for automatic detection of dairy cattle lameness (CattleEye Ltd). This was achieved by first measuring mobility score agreement between CattleEye and two veterinarians (Assessor 1 and Assessor 2), and second, by investigating the ability of the CattleEye system to detect cows with potentially painful foot lesions. We analysed 6,040 mobility scores collected from three dairy farms. Inter-rate agreement was estimated by calculating percentage agreement (PA), Cohen's kappa (κ) and Gwet's agreement coefficient (AC). Data regarding the presence of foot lesions were also available for a subset of this dataset. The ability of the system to predict the presence of potentially painful foot lesions was tested against that of Assessor 1 by calculating measures of accuracy, using lesion records during the foot trimming sessions as reference. In general, inter-rater agreement between CattleEye and either human assessor was strong and similar to that between the human assessors, with PA and AC being consistently above 80% and 0.80, respectively. Kappa agreement between CattleEye and the human scorers was in line with previous studies (investigating agreement between human assessors) and within the fair to moderate agreement range. The system was more sensitive than Assessor 1 in identifying cows with potentially painful lesions, with 0.52 sensitivity and 0.81 specificity compared to the Assessor's 0.29 and 0.89 respectively. This pilot study showed that the CattleEye system achieved scores comparable to that of two experienced veterinarians and was more sensitive than a trained veterinarian in detecting painful foot lesions.

Predict, Identify and Alert on Suspicious Activity by Multiple Zone

Abstract: An active topic of image processing and computer vision research is the detection of suspicious human behaviour in surveillance footage. In order to stop terrorism, theft,accidents, illegal parking, vandalism, fighting, chain snatching, crime, and other suspicious activities, human activity can be observed through visual surveillance in sensitive and public places like bus stations, railway stations, airports, banks, shopping malls, schools, and colleges. It is highly challenging to continually monitor public spaces, thus an intelligent video surveillance system that can track people's movements in real-time, classify them as routine or odd, and send out an alert is needed. The field of visual surveillance to identify aberrant actions has seen a significant amount of publications in the last ten years. Furthermore There are a few surveys that canbe found in the literature for the recognition of various abnormal activities, but none of them have reviewed various abnormal activities. This study presents the state-of-the-art in the field of recognising suspicious behaviour from surveillance recordings during the past tenyears. We give a quick overview of the concernsand difficulties associated in recognising suspicious human activity. This article examines six aberrant behaviours, including the identification of abandoned objects, theft, falls, traffic accidents, and unlawful parking, as wellas the detection of violence and fire. Generally speaking, we have covered all the processes that have been [1] Foreground object extraction, object identification based on tracking or non-tracking approaches, feature extraction, classification, activity analysis, and recognition are some of the techniques that have been used to identify human activity from surveillance movies in the literature. This paper's goal is to give field researchers a literature assessment of six different suspicious activity identification systems together with its broad framework.[1]

Understanding Policy and Technical Aspects of AI-enabled Smart Video Surveillance to Address Public Safety

Recent advancements in artificial intelligence (AI) have seen the emergence of smart video surveillance (SVS) in many practical applications, particularly for building safer and more secure communities in our urban environments. Cognitive tasks, such as identifying objects, recognizing actions, and detecting anomalous behaviors, can produce data capable of providing valuable insights to the community through statistical and analytical tools. However, artificially intelligent surveillance systems design requires special considerations for ethical challenges and concerns. The use and storage of personally identifiable information (PII) commonly pose an increased risk to personal privacy. To address these issues, this paper identifies the privacy concerns and requirements needed to address when designing AI-enabled smart video surveillance. Further, we propose the first end-to-end AI-enabled privacy-preserving smart video surveillance system that holistically combines computer vision analytics, statistical data analytics, cloud-native services, and end-user applications. Finally, we propose quantitative and qualitative metrics to evaluate intelligent video surveillance systems. The system shows the 17.8 frame-per-second (FPS) processing in extreme video scenes. However, considering privacy in designing such a system results in preferring the pose-based algorithm to the pixel-based one. This choice resulted in dropping accuracy in both action and anomaly detection tasks. The results drop from 97.48% to 73.72% in anomaly detection and 96% to 83.07% in the action detection task. On average, the latency of the end-to-end system is 36.1 seconds.

Abnormal Activity Recognition in Private Places Using Deep Learning

Using computer and machine vision technology, the process of analysing human motion is known as "human activity recognition," or HAR. Anomaly detection in security systems is one of the situations in which human activity recognition is useful. As the demand for security growing, surveillance cameras have been widely installed as the foundation for video analysis. Identifying anomalous behaviour demands strenuous human effort, which is one of the main obstacles in surveillance video analysis. It is necessary to establish video recording in order to automatically catch anomalous activities. Using deep learning methods, our intelligent video surveillance system can identify an anomaly in a video. Real-time detection of the actions is also possible, and these video frames will be afterwards preserved as photographs in the system for the user to examine.The suggested Abnormal Activity Recognition system was created with the goal of identifying and detecting irregularities through a live feed in the banking sector, more specifically in an ATM setting.The initial phase of the study focuses on the application of image deep learning techniques to recognise various items and spot unusual behaviour using ATM monitoring systems.

An Overview of the Vision-Based Human Action Recognition Field

Artificial intelligence’s rapid advancement has enabled various applications, including intelligent video surveillance systems, assisted living, and human–computer interaction. These applications often require one core task: video-based human action recognition. Research in human video-based human action recognition is vast and ongoing, making it difficult to assess the full scope of available methods and current trends. This survey concisely explores the vision-based human action recognition field and defines core concepts, including definitions and explanations of the common challenges and most used datasets. Additionally, we provide in an easy-to-understand manner the literature approaches and their evolution over time, emphasizing intuitive notions. Finally, we explore current research directions and potential future paths. The core goal of this work is to provide future works with a shared understanding of fundamental ideas and clear intuitions about current works and find new research opportunities.