Intelligent Video Surveillance Research Articles

Towards Mine 4.0: A Proposed Multi-Layered Architecture for Real-Time Surveillance and Anomaly Detection in an Open-Pit Phosphate Mine

Given the global importance and limited availability of phosphate, optimizing the use of this critical resource and minimizing its wastage are of paramount importance. In this context, this paper proposes an innovative architecture for the implementation of an intelligent video surveillance system specifically designed for open-pit phosphate mines. The proposed architecture is designed to meet the overall functional requirements of a surveillance system in the challenging environment of open-pit mining, while aligning with the guidelines of the Mine 4.0 revolution. It incorporates advanced technologies that address the critical challenges of latency, data security, and transparency commonly encountered in traditional monitoring systems. By adopting a multi-layered approach that leverages edge, fog, and cloud computing, coupled with blockchain technology and expert collaboration, our architecture offers a comprehensive framework for efficient data processing at every stage, from initial data acquisition to real-time anomaly detection and decision-making.

Person re-identification from UAVs based on Deep hybrid features: Application for intelligent video surveillance

Person re-identification from UAVs based on Deep hybrid features: Application for intelligent video surveillance

Consistency-constrained unsupervised video anomaly detection framework based on Co-teaching

Intelligent video surveillance continues to be a vibrant research domain within the field of computer vision. However, existing representation learning frameworks primarily focus on static information extraction frame by frame such as appearance features, they often overlook the valuable dynamic information like optical flow feature inherent in the video data, which is most essential characteristics of sequence data. To mining dynamic features and bridge this gap, our paper introduces a novel anomaly detection framework that balance dynamic information with static information and construct a relationship between appearance features and corresponding optical flow features, where we sets strong consistency constraints, which reduce the loss between dynamic information and corresponding static information, and we leverages collaborative teaching network to ensure a consistent representation of both static and dynamic information for predict. The proposed framework consists of two sets of encoder–decoder pairs complemented by a memory storage module. Operating in parallel with the dual encoder network is a Co-teaching network, with the shared memory module serving as the cornerstone for collaborative training. The Consistency constrained condition guarantees the strong consistency of dynamic and static information in the learned representations. In our experimental phase, we present compelling results that showcase the superior performance of our algorithm across three publicly available datasets.

Wushu Movement Recognition System Based on DTW Attitude Matching Algorithm

Motion recognition technology is widely used in intelligent video surveillance, human-computer interaction and other fields. With the development of computer vision technology, improving the accuracy and efficiency of motion recognition has become the focus of research. The purpose of this study is to improve the performance of Wushu movement recognition through improved dynamic time warping algorithm and hierarchical model. Firstly, a high-dimensional feature vector is constructed by using the position, velocity and Angle changes of human bone joints. The actions are subdivided by the hierarchical model, and matched and recognized by the max-minimum dynamic time regularization model. Meanwhile, the K-class mean algorithm is combined to optimize the type of tree core, improve the performance of the model, reduce the interference of noise nodes, and effectively classify Wushu actions. Experimental verification was carried out on four public data sets of KTH, Olympic Sports, Hollywood2 and HMDB51. The experimental results showed that the recognition rate of the proposed model in KTH data set was 95.2%, and that in Olympic Sports data set was 91.4%. The Hollywood2 dataset was 66.7%, and the HMDB51 dataset was 61.2%. Comparing the results of different algorithms, the proposed method improved the recognition performance by 10% compared with long short-term memory network and gated cycle unit. Compared with one-dimensional convolutional neural network, the time of the proposed method was 15s longer, but the recognition rate was 1.6% higher. The results showed that the proposed method had significant performance advantages in diverse and complex action recognition tasks. Meanwhile, the results emphasized the factors to be considered in the design of the model, demonstrating its effectiveness in the application of Wushu movement recognition.

Harnessing Deep Learning for Video Based Weapon Detection

This research addresses the escalating global issue of handgun-related crimes by proposing an innovative Intelligent Video Surveillance System (IVSS) that leverages advanced deep learning (DL) techniques for remote firearm detection and timely threat response. The system employs Convolutional Neural Networks (CNN) and the YOLO v3 model, uniquely integrating Transfer Learning (TL) to enhance adaptability and efficacy. Experimental validation using the Internet Movie Firearms Database (IMFDB) demonstrates the system's versatility in detecting various pistols and guns, achieving promising results that surpass existing systems in accuracy and efficiency. Challenges in real-time weapon recognition, such as the absence of a standardized weapon dataset, occlusion, and small object sizes, are acknowledged. Emphasis is placed on the critical need for reliable data acquisition, precise labeling, and preprocessing tailored to different detection algorithms. The implementation encompasses video collection, preprocessing, model loading, algorithm application, segmentation, and classification, alongside a user-friendly webcam interface for real-time detection. Additionally, the system integrates the pyttsx3 library for voice alerts and the Twilio API for voice call alerts to enhance responsiveness. In summary, this study presents a novel CNN-based model combining Transfer Learning with YOLO v3, achieving superior weapon identification and distinguishing between real and fake firearms, representing a significant advancement in intelligent video surveillance and contributing to the reduction of weapon violence.

Learning a Context-Aware Environmental Residual Correlation Filter via Deep Convolution Features for Visual Object Tracking

Visual tracking has become widespread in swarm robots for intelligent video surveillance, navigation, and autonomous vehicles due to the development of machine learning algorithms. Discriminative correlation filter (DCF)-based trackers have gained increasing attention owing to their efficiency. This study proposes “context-aware environmental residual correlation filter tracking via deep convolution features (CAERDCF)” to enhance the performance of the tracker under ambiguous environmental changes. The objective is to address the challenges posed by intensive environment variations that confound DCF-based trackers, resulting in undesirable tracking drift. We present a selective spatial regularizer in the DCF to suppress boundary effects and use the target’s context information to improve tracking performance. Specifically, a regularization term comprehends the environmental residual among video sequences, enhancing the filter’s discrimination and robustness in unpredictable tracking conditions. Additionally, we propose an efficient method for acquiring environmental data using the current observation without additional computation. A multi-feature integration method is also introduced to enhance the target’s presence by combining multiple metrics. We demonstrate the efficiency and feasibility of our proposed CAERDCF approach by comparing it with existing methods using the OTB2015, TempleColor128, UAV123, LASOT, and GOT10K benchmark datasets. Specifically, our method increased the precision score by 12.9% in OTB2015 and 16.1% in TempleColor128 compared to BACF.

Revisiting vision-based violence detection in videos: A critical analysis

An ever-increasing installation of surveillance cameras at different places for ensuring public safety, security and asset protection has triggered the need for intelligent video surveillance to monitor the people and their behavior. Violence detection is a prominent application of intelligent surveillance as it plays a vital role in public safety, behavior monitoring, and law enforcement. It deals with identifying whether the violent event or behavior occurred in video sequence or not. Various researchers have developed different techniques and features for the detection of violence in recent years. The purpose of this paper is to provide an expository study of various state-of-the-art approaches for detecting violence in videos. It is evident from the ongoing effort in this field and with the advancement in computer vision technology, previous approaches get surpassed and new methods and features always keep on developing. Therefore, in order to complement ongoing research, it is also imperative to conduct a comprehensive analysis of different works from time to time. In this survey, each work has been critically analyzed, along with its pros and cons. The violence detection techniques have been divided under three categories: handcrafted features based, deep learning and hybrid violence detection approaches that have been extensively reviewed in various sub-categories. The major challenges faced by researchers and the steps involved in the process of violence detection are also discussed. Moreover, unlike typical practices of comparing the performance of models within same datasets, we examined a few models for cross-dataset assessment and revealed their limited generalization. This emphasizes the crucial need for robust model generalization to ensure efficacy across varied real-world scenarios. Finally, the paper carried out a discussion of various research gaps in the current approaches and the possible solutions to be taken to resolve them, laying a solid foundation for future work in this area.

Leveraging YOLOv5s with optimization‐based effective anomaly detection in pedestrian walkways

AbstractCurrently, video surveillance is generally used to safeguard safety in public places like railway stations, traffic signals, malls, and so on. Video anomaly recognition and localization are the main components of the intelligent video surveillance method. Video anomaly recognition refers to the procedure of spatiotemporal localization of the abnormal design existing in the video. A main task in video surveillance is the classification of anomalies that occur in it like thefts, crimes, and so forth. Also, anomaly recognition in pedestrian walkways has enlarged major attention among the computer vision (CV) groups to improve pedestrian protection. The current developments in Deep Learning (DL) methods have great attention to dissimilar procedures like image classification, object recognition, and so forth. This study designs an Optimal Deep Learning for Effective Anomaly Detection in Pedestrian Walkways (ODL‐EADPW) model. The ODL‐EADPW technique employs a fine‐tuned DL model for the identification of pedestrians and anomalies in the walkways. In the ODL‐EADPW technique, the image pre‐processing is primarily involved in two stages median filtering (MF) based noise removal and adaptive histogram equalization (AHE)‐based contrast enhancement. For anomaly detection in pedestrian walkways, the ODL‐EADPW technique uses the YOLOv5s model with EfficientRep as a backbone network. To enhance the detection results of the ODL‐EADPW technique, a stochastic gradient descent (SGD) optimizer was employed to perfect the hyperparameters of the EfficientRep model. The performance evaluation of the ODL‐EADPW methodology is implemented on the UCSD Anomaly detection dataset. An extensive comparison study stated that the ODL‐EADPW technique gains effectual detection results over other DL models in terms of different measures.

PEDESTRIAN ALERT SYSTEM USING DEEP LEARNING

Pedestrian detection is an essential and significant task in any intelligent video surveillance system, as it provides the fundamental information for semantic understanding of the video footages. It has an obvious extension to automotive applications due to the potential for improving safety systems. Many car manufacturers (e.g., Volvo, Ford, GM, Nissan) offer this as an ADAS option in 2017. It is used in many cars and vehicles for automatic driver assistance systems. But when the ADAS system detects pedestrians but if fails to apply brakes on time accidents might happened. To overcome this situation our project provides an audio alert to the human passengers or driver in the vehicle so that they can apply manual brakes. In this project we will require Python libraries such as Pytorch and OpenCV. Pytorch is a widely used Machine Learning library. It is popular for the YOLO (You Only Look Once) algorithm which is built for Object Detection, we are using the YOLO algorithm and customize it to detect objects on a pedestrian dataset. Python has a library pyttsx3, that is capable to convert text-to-speech offline. Itextracts the label from the detected pedestrians in the video and converts the text label into speech. The algorithm detects the pedestrians when they are in a close approach to the vehicles and sends the identifies labels to the pyttsx3 speaker engine. An audio alert is generated by the engine and alerts the passengers or drivers. Keywords - YOLO, Pytorch, Pyttsx3, Object Detection, Python.

Advancements in Intelligent Video Surveillance for Public Safety: A Comprehensive Review

Abstract: Intelligent video surveillance systems have emerged as invaluable tools for ensuring public safety in a variety of settings, from essential infrastructure to urban environments. These systems, which are driven by state-of-the-art technology like computer vision, machine learning, and artificial intelligence, provide unmatched real-time threat detection, analysis, and response capabilities. This study provides an in-depth analysis of the most recent developments in intelligent video surveillance for public safety, covering key technology, useful uses, enduring difficulties, and potential directions. This paper aims to provide a thorough knowledge of the significant implications and emerging opportunities of intelligent video surveillance in protecting public places from potential dangers through an examination of recent research insights and industry advancements. This article seeks to offer a guide for researchers, practitioners, policymakers, and stakeholders navigating the ever-changing field of intelligent video surveillance for public safety by exploring the evolving trends, emerging technologies, and practical applications.

Intelligent Video Surveillance System

Intelligent Video Surveillance Systems (IVSS) have emerged as critical tools for ensuring safety and security in various environments. This paper presents an IVSS capable of real-time detection and analysis of critical events, including falls and vehicle crashes. The proposed system employs state-of-the-art deep learning techniques, specifically the YOLO (You Only Look Once) object detection algorithm, to analyze video streams obtained from diverse sources such as surveil- lance cameras and recorded videos. For fall detection, the system utilizes the aspect ratio of detected persons’ bounding boxes to distinguish between falls and normal activities. Vehicle crashes are identified by detecting cars in each frame and assessing the proximity and intersection of their bounding boxes. The system operates in a continuous loop, processing video frames in real-time and generating alerts when critical events are detected. Experimental results demon- strate the effectiveness and reliability of the proposed IVSS in enhancing safety and security across different environments, including public spaces, workplaces, and transportation systems.

Knowledge Development Trajectories of Intelligent Video Surveillance Domain: An Academic Study Based on Citation and Main Path Analysis.

Smart city is an area where the Internet of things is used effectively with sensors. The data used by smart city can be collected through the cameras, sensors etc. Intelligent video surveillance (IVS) systems integrate multiple networked cameras for automatic surveillance purposes. Such systems can analyze and monitor video data and perform automatic functions required by users. This study performed main path analysis (MPA) to explore the development trends of IVS research. First, relevant articles were retrieved from the Web of Science database. Next, MPA was performed to analyze development trends in relevant research, and g-index and h-index values were analyzed to identify influential journals. Cluster analysis was then performed to group similar articles, and Wordle was used to display the key words of each group in word clouds. These key words served as the basis for naming their corresponding groups. Data mining and statistical analysis yielded six major IVS research topics, namely video cameras, background modeling, closed-circuit television, multiple cameras, person reidentification, and privacy, security, and protection. These topics can boost the future innovation and development of IVS technology and contribute to smart transportation, smart city, and other applications. According to the study results, predictions were made regarding developments in IVS research to provide recommendations for future research.

Deep Learning-Based Intelligent Video Surveillance System for Real-Time Motion Detection

The proliferation of surveillance systems demands advanced solutions to effectively monitor and analyze video feeds for security and safety purposes. In this paper, we propose a Deep Learning-Based Intelligent Video Surveillance System (DIVSS) designed to enhance traditional Closed-Circuit Television (CCTV) setups. DIVSS integrates state-of-the-art deep learning techniques for real-time motion detection and object recognition, enabling proactive threat detection and automated response mechanisms. The system utilizes convolutional neural networks (CNNs) for robust feature extraction and classification, enabling accurate identification of objects of interest amidst complex backgrounds and varying[1] lighting conditions. Furthermore, DIVSS incorporates region of interest (ROI) monitoring to focus attention on specific areas within the surveillance footage, optimizing computational resources and improving response times. We evaluate the performance of DIVSS through comprehensive experiments conducted on diverse surveillance scenarios, demonstrating its effectiveness in enhancing security measures and providing actionable[3] insights for surveillance operators. The proposed system represents a significant advancement in video surveillance technology, offering unparalleled capabilities for proactive threat detection and real-time response in dynamic environments. Keywords— Deep learning has transformed video surveillance, enabling real-time motion detection and object recognition using Convolutional Neural Networks (CNNs). This technology provides proactive threat detection, intelligent systems for ROI monitoring, and enhances overall security measures.

A person re‐identification method for sports event scenes incorporating textual information mining

AbstractPerson re‐identification represents a pivotal sub‐problem in image retrieval, boasting broad application prospects in fields such as intelligent security and video surveillance. However, most existing person re‐identification methods predominantly focus solely on visual features pertaining to the person targets, thereby disregarding some supporting information closely related to the scene context. In the context of athlete re‐identification during sports event scenes, the athlete bib number is fully considered, an important clue that can provide different athletes' identities, and the traditional visual features of the person and high‐level semantic information of the bib number text are fused. A multi‐source information mutual gain mechanism is designed to improve the accuracy of the person re‐identification task. In the existing only publicly available marathon bib number dataset RBNR, the recognition accuracy of this method is significantly superior to that of the existing person re‐identification method. In addition, this paper constructs and publishes an athlete re‐identification dataset (HNNU‐ReID8000) for mainstream sports events, and the mean average precision (mAP) value of this method reaches 96.1% on this dataset, significantly ahead of existing state‐of‐the‐art person re‐identification methods. The code and the HNNU‐ReID8000 dataset will be released at https://github.com/yanbin‐zhu/zyb_person‐reid.

HF-YOLO: Advanced Pedestrian Detection Model with Feature Fusion and Imbalance Resolution

Pedestrian detection is crucial for various applications, including intelligent transportation and video surveillance systems. Although recent research has advanced pedestrian detection models like the YOLO series, they still face limitations in handling diverse pedestrian scales, leading to performance challenges. To address these issues, we propose HF-YOLO, an advanced pedestrian detection model. HF-YOLO tackles the complexities of pedestrian detection in complex scenes by addressing scale variations and occlusions among pedestrians. In the feature fusion stage, our algorithm leverages both shallow localization information and deep semantic information. This involves fusing P2 layer features and adding a high-resolution detection layer, significantly improving the detection of small-scale pedestrians and occluded instances. To enhance feature representation, HF-YOLO incorporates the HardSwish activation function, introducing more non-linear factors and strengthening the model’s ability to represent complex and discriminative features. Additionally, to address regression imbalance, a balance factor is introduced to the CIoU loss function. This modification effectively resolves the imbalance problem and enhances pedestrian localization accuracy. Experimental results demonstrate the effectiveness of our proposed algorithm. HF-YOLO achieves notable improvements, including a 3.52% increase in average precision, a 1.35% boost in accuracy, and a 4.83% enhancement in recall. Moreover, the algorithm maintains real-time performance with a detection time of 8.5ms, meeting the stringent requirements of real-time applications.

A Design Pattern of IAPVS Platform Based on Distributed Edge Computing

Abstract Traditional video surveillance is difficult to effectively cope with real-time interaction between end side devices of over ten thousand scale and data centers, and these devices are widely dispersed and deployed on a large scale. They cannot meet the needs of diverse scenarios monitoring and smart applications such as security, urban comprehensive management, ports, mines, water conservancy, power, industrial manufacturing and other fields. Based on the analysis of the problems in the popular intelligent video surveillance system, this paper proposes the design mode of an intelligent video surveillance intelligent analysis platform (IAPVS) based on distributed edge computing by improving the existing centralized video cloud computing architecture.This design pattern provides a new overall design solution for the development of platform of intelligent video analysis suitable for diverse scenarios and smart application fields.

A review on current progress of semantic segmentation

Semantic segmentation, as an important task in the field of computer vision, has wide applications in image analysis and scene analysis. These application domains include autonomous driving, medical image analysis, image identification, and intelligent video surveillance. However, it faces many challenges due to the complex image structures and some confusing relationships between objects. This paper aims to provide an overview of key concepts in the field of semantic segmentation, including datasets and annotations, data augmentation, some relevant algorithms and models, and loss functions. By introducing and analyzing these concepts, we can gain a comprehensive understanding of the research progress and future directions in semantic segmentation. This paper also provides research advancements in the field of semantic segmentation. Through the introduction and analysis of these different concepts, we gain a deeper understanding of the current state and challenges of semantic segmentation. With the continuous development of deep learning techniques, we can expect semantic segmentation to have broader applications in the fields of computer vision and artificial intelligence.

Spatiotemporal Masked Autoencoder with Multi-Memory and Skip Connections for Video Anomaly Detection

Video anomaly detection is a critical component of intelligent video surveillance systems, extensively deployed and researched in industry and academia. However, existing methods have a strong generalization ability for predicting anomaly samples. They cannot utilize high-level semantic and temporal contextual information in videos, resulting in unstable prediction performance. To alleviate this issue, we propose an encoder–decoder model named SMAMS, based on spatiotemporal masked autoencoder and memory modules. First, we represent and mask some of the video events using spatiotemporal cubes. Then, the unmasked patches are inputted into the spatiotemporal masked autoencoder to extract high-level semantic and spatiotemporal features of the video events. Next, we add multiple memory modules to store unmasked video patches of different feature layers. Finally, skip connections are introduced to compensate for crucial information loss caused by the memory modules. Experimental results show that the proposed method outperforms state-of-the-art methods, achieving AUC scores of 99.9%, 94.8%, and 78.9% on the UCSD Ped2, CUHK Avenue, and Shanghai Tech datasets.

Temporal Fusion of Time-Distributed VGG-16 and LSTM for Precise Action Recognition in Video Sequences

Autonomous systems play a vital role in creating an ambient environment, but for comprehensive human-robot interaction, the system must interpret and identify specific human behaviours. The demand for recognizing human activity from videos has significantly increased across diverse domains, such as medicine, intelligent video surveillance, human-robot interaction, and ambient-assisted environments. Deep learning has emerged as a pioneering technique for human activity recognition from videos due to its ability to learn features from input videos automatically. This research proposes a novel approach for recognizing activities in videos by leveraging the combined power of the VGG-16 convolutional neural network (CNN) and the Long Short-Term Memory (LSTM). To assess the efficiency of our suggested methodology, experiments are conducted on the widely used benchmark dataset UCF-50. The experimental results demonstrate that our VGG-16 and LSTM-based model outperforms state-of-the-art methods for activity recognition, achieving higher accuracy and robustness across various activity categories.

Suspicious Activity Detection

Abstract: Unpredictable activities is guessing a person's physical location or joint activity based on images or videos. The project will require the use of neural networks to detect human activities from CCTV images. Human behavior is one of the fundamental problems in computer vision and has been studied for over 15 years. This is important because the number of applications that can benefit from the search function is huge. For example, it is used in applications such as human prediction, video analysis, animal analysis and behavior understanding, language recognition, human relations-computer, and characters are less powerful. While low-cost depth sensors have limitations such as being limited to indoor use, their low-noise data and low depth make it difficult to estimate humans from depth images. Therefore, we plan to use neural networks to solve these problems. Analyzing human activity through image analysis is an active area of research in image processing and computer vision. Thanks to visual monitoring, human activities in public places such as bus stops, train stations, airports, banks, shopping malls, schools, parking lots and roads can be monitored in order to prevent attacks, theft, accidents, illegal parking, violence, fights and chain events. purse snatching etc. illegal and other activities. It is very difficult to monitor public places regularly, so there is a need for intelligent video surveillance that can monitor people's activities in real time and divide them into tasks, normal and abnormal activities; and can generate an alarm. Often research is done with images rather than video. Also, none of the published articles attempt to use CNNs to detect suspicious activity.