Visual Surveillance System Research Articles

ADVANCED SHOPLIFTING PREVENTION AND ALERT STSYEM

Detecting human beings accurately in a visual surveillance system is crucial for diverse application areas including abnormal event detection, human gait characterization, congestion analysis, person identification, gender classification and fall detection for elderly people. The first step of the detection process is to detect an object which is in motion. Object detection could be performed using YOLOv7, optical flow and spatio-temporal filtering techniques. Once detected, a moving object could be classified as a human being using shape-based, texture-based or motion-based features. A comprehensive review with comparisons on available techniques for detecting human beings in surveillance videos is presented in this paper. The characteristics of few benchmark datasets as well as the future research directions on human detection have also been discussed. We can use camera for Human Motion Detection. The Camera is used to catch the live images of the area in which it is being implemented, if any object is moving. The captured images are stored for further work. If motion is found in this video, the computer will start recording, buzz an alarm and send SMS to people listed in its database. In this way the system will provide the security against any misdeed. Keywords: YOLOv7, SMS, E-mail, CCTV, SMTP, RFID

Stereo object matching for mobile robot path planning using artificial fish algorithms

<span>The <span>popularity of robots is on the rise, not only in industrial settings but increasingly in daily venues such as airports. Recently, some organizations have carried out experiments utilizing robots specifically created to improve airport hygiene, security, and passengers’ overall satisfaction. Furthermore, the utilization of the artificial fish (AFs) algorithm in path planning for mobile robots yielded exceptional outcomes. The robot can replicate the prey behavior of the AFs algorithm, as evidenced by the prevalence of pos one in the simulation. The robot exhibits another behavior, which is the subsequent behavior. The behavior of the AFs algorithm is influenced by the available food sources. Simultaneously, mobile robots are influenced by the stimulation of their neighboring responses. Afterwards, the three primary classifiers are employed to perform stereo-object matching on different objects. The recognition rate achieved by the AdaBoost classifier is promising, with an accuracy rate of 92.4%. This result shows excellent potential for improving the path planning of mobile robots equipped with visual surveillance systems for their surroundings.</span></span>

Suspicious Activity Classification in Classrooms using Deep Learning

Video processing is attracting the attention of both research and industry. The existence of intelligent surveillance cameras with high processing power has paved the way for designing intelligent visual surveillance systems. Along with analyzing video for information recovery, it is nowadays used to analyze live surveillance video to detect activities. These systems are implemented in real time. The proposed work's goal is to create a method that can examine and discover suspicious behaviors in the lecture room environment. Video analytics offers the most efficient answer because it enables pointing an occasion and retrieves applicable statistics from the video recorded. The method aims to identify suspicious activities like fighting, sleeping, looking elsewhere, eating, etc. that the students might be doing. The proposed method involves breaking a video input into frames and converting it into image data because the model has been trained on images collected from the internet. Several models were tested and experimented with, including efficientnet_b2, spnasnet_100, efficientnet_b3, and mobilenetv3_large_100. Parameters such as the Learning Rate were optimized to find out the best method and create a system with the best results.

Enhancing Multi-Object Detection in Video Content: Exploring Hybrid Techniques and Method Combinations for Improved Classification Accuracy

The classification of objects within video content holds significant importance, particularly in the context of automated visual surveillance systems. Object classification refers to the procedure of categorizing objects into predefined and semantically meaningful groups based on their features. While humans find object classification in videos to be straightforward, machines face complexity and challenges in this task due to various factors like object size, occlusion, scaling, lighting conditions, and more. Consequently, the demand for analyzing video sequences has spurred the development of various techniques for object classification. This paper proposes hybrid techniques for multi object detection. The experimental analysis focused on a vehicles-openimages dataset containing 627 different catagories of vehicles. The results emphasize the profound impact of method combinations on image classification accuracy. Two primary methods, wavelet transformation and Principal Component Analysis (PCA), were employed alongside Convolutional Neural Networks (CNN) and Recurrent Neural Networks (RNN). The evaluation encompassed performance metrics, including accuracy, precision, recall, specificity, and F1 score. In the analysis, Wavelet + RNN" combination consistently achieved the highest accuracy across all performance metrics, including accuracy percentage (96.76%), precision (96.76%), recall (86.32%), F1 score (87.12%), and specificity (87.43%). In addition, the hybrid classifiers were subjected for image classification of different vehicle catagories. In the analysis of different catagories, Wavelet + RNN" emerges as the standout performer, consistently achieving high accuracy percentages across all object categories, ranging from 82.87% for identifying People to 90.12% for recognizing Trucks.

URRNet: A Unified Relational Reasoning Network for Vehicle Re-Identification

With the continuous improvement and optimization of security monitoring networks, vehicle Re-Identification (Re-ID) becomes an emerging key technology in the development of intelligent visual surveillance systems. Due to the influence of viewpoint variation and fine-grained differences, vehicle Re-ID is still a research topic worth investigating. To alleviate above problems, a novel end-to-end framework named Unified Relational Reasoning Network (URRNet) is proposed in this paper, which integrates global features with local features to obtain better recognition accuracy. For the proposed framework, to understand the overall semantics of the image, an algorithm based on the global feature graph-structure learning is designed. The pixel-level feature maps are transformed to the node features of graph in the interactive space by projection, then graph reasoning is performed by using the graph convolutional network to improve the representation of global features. Moreover, an algorithm based on multi-scale local feature relational reasoning is designed. Using keypoint and viewpoint to obtain the multi-scale partial characteristics of the vehicle, and the vehicle multi-view features are learned from the single-view vehicle images through relational reasoning and attention mechanism. The two algorithms are combined to obtain the overall model, which not only preserves the details of the vehicle, but also effectively solves the problem of viewpoint variation. Comprehensive experimental results on two public datasets (VeRi-776 and VehicleID) indicate that the proposed URRNet can practically improve the model's representation ability and generalization ability, which is comparable to the state-of-the-art vehicle Re-ID methods.

Person re-identification: A retrospective on domain specific open challenges and future trends

Person Re-Identification (Re-ID) is a critical aspect of visual surveillance systems, which aims to automatically recognize and locate individuals across a multi-camera network with non-overlapping fields-of-view. Despite significant progress in recent years through the use of deep learning-based approaches, there remain many vision-related challenges, such as occlusion, pose, background clutter, misalignment, scale, viewpoint, low resolution & illumination, and cross-domain generalization across camera modalities, that hinder the accurate identification of individuals. The majority of the proposed approaches directly or indirectly aim to solve one or multiple of these existing challenges. To further advance the development of Re-ID solutions, a comprehensive review of the current approaches is necessary. However, no focused review currently exists that analyses and highlights specific aspects for further development. To fill this gap, we present a systematic challenge-specific literature survey of about 300 papers published between 2015 and 2022, which reviews Re-ID approaches from a solution-oriented perspective. This survey is the first of its kind to provide an in-depth analysis of the different approaches used to address the various challenges in Re-ID. Furthermore, our review highlights several prominent and diverse research trends in the Re-ID domain. These trends offer a visionary perspective regarding ongoing person Re-ID research, and they may eventually lead to the development of practical real-world solutions. We highlighted the AI ethics that must be followed while developing a Re-ID solution, and recently being practiced as well. Another exciting future dimension of person Re-ID research is the long-term Re-ID, which is still under evolution. Overall, our survey aims to serve as a valuable resource for researchers and practitioners working in the field of Re-ID and to inspire the development of innovative and effective Re-ID solutions.

Vulnerable underground entrance understanding for visual surveillance systems

Protecting critical infrastructure through visual surveillance is of vital importance, especially in underground entrance environments where large chunks of sloping glass ceilings are particularly susceptible to various types of terrorist activity. However, owing to the diversity of underground entrance environments, understanding them remains a challenge. Traditional 3D layout and object pose estimation that are evaluated on 3D point clouds or RGB-D data are energy-consuming and difficult to account for semantic information in environments. In this study, we present a methodology to understand underground entrance environments, and to recover their 3D reconstruction from a monocular camera. Clusters of sloping angle projections are extracted. Through their corresponding vanishing points (VPs), surfaces of sloping structures are estimated. Relative geometric constraints of different planes are built to bridge the gap between 2D sloping surfaces and 3D reconstruction without precise depth or point clouds. An underground entrance scene is approximated by Manhattan and sloping non-Manhattan structures in 3D reconstruction. The approach requires no prior training, and it requires neither the camera being calibrated nor the camera internal parameters being constant. Compared to the ground truth, the percentage of incorrectly understood pixels were measured and the results demonstrated that the method can successfully understand underground entrance scenes, meeting the requirements in safety monitoring for critical infrastructures from a resource-constrained surveillance camera.

Real-time pedestrian pose estimation, tracking and localization for social distancing.

The corona virus pandemic has introduced limitations which were previously not a cause for concern. Chief among them are wearing face masks in public and constraints on the physical distance between people as an effective measure to reduce the virus spread. Visual surveillance systems, which are common in urban environments and initially commissioned for security surveillance, can be re-purposed to help limit the spread of COVID-19 and prevent future pandemics. In this work, we propose a novel integration technique for real-time pose estimation and multiple human tracking in a pedestrian setting, primarily for social distancing, using CCTV camera footage. Our technique promises a sizeable increase in processing speed and improved detection in very low-resolution scenarios. Using existing surveillance systems, pedestrian pose estimation, tracking and localization for social distancing (PETL4SD) is proposed for measuring social distancing, which combines the output of multiple neural networks aided with fundamental 2D/3D vision techniques. We leverage state-of-the-art object and pose estimation algorithms, combining their strengths, for increase in speed and improvement in detections. These detections are then tracked using a bespoke version of the FASTMOT algorithm. Temporal and analogous estimation techniques are used to deal with occlusions when estimating posture. Projective geometry along with the aforementioned posture tracking is then used to localize the pedestrians. Inter-personal distances are calculated and locally inspected to detect possible violations of the social distancing rules. Furthermore, a "smart violations detector" is employed which estimates if people are together based on their current actions and eliminates false social distancing violations within groups. Finally, distances are intuitively visualized with the right perspective. All implementation is in real time and is performed on Python. Experimental results are provided to validate our proposed method quantitatively and qualitatively on public domain datasets using only a single CCTV camera feed as input. Our results show our technique to outperform the baseline in speed and accuracy in low-resolution scenarios. The code of this work will be made publicly available on GitHub at https://github.com/bilalze/PETL4SD.

Detective Eye: Exclusively for Face and Object Detection, Identification and Localization

Dynamic Visual surveillance System is basically used for examination and explanation of article approaches to acting. It contains static and moving article acknowledgment, video following to appreciate the events that occur in scene. The principal objective of this study paper is to choose the various methodologies in static and moving article area as well as following of moving things. Any video scene contains objects that not altogether firmly established by object area technique. There are various classes of recognized article like tree, fogs, individual and other moving things. Area for moving thing is a very going after for any video perception structure. Object Following is used to find the district where articles are available and condition of things in each packaging in more critical level application. In this paper, we used different procedures like establishment allowance, genuine system, and transient edge differencing for the recognizable proof of moving articles. We also portrayed different following methodologies like Point following, Diagram Following and Piece continuing in our review paper.

An End-to-End Human Abnormal Behavior Recognition Framework for Crowds With Mentally Disordered Individuals.

Abnormal or violent behavior by people with mental disorders is common. When individuals with mental disorders exhibit abnormal behavior in public places, they may cause physical and mental harm to others as well as to themselves. Thus, it is necessary to monitor their behavior using visual surveillance systems. However, it is challenging to automatically detect human abnormal behavior (especially for individuals with mental disorders) based on motion recognition technologies. To address these issues, in the current work, we propose an end-to-end abnormal behaviour detection framework from a new perspective in conjunction with the Graph Convolutional Network (GCN) and a 3D Convolutional Neural Network (3DCNN). Specifically, we first train a one-class classifier to extract features and estimate abnormality scores. To improve the performance of abnormal behavior detection, GCN is used to model the similarity between video clips for the correction of noisy labels. Then, based on this framework, GCN recognizes the normal behavior clips in the abnormal video and removes them, while the clips identified as abnormal behavior are retained. Finally, a 3D CNN is used to extract spatiotemporal features to classify different abnormal behaviors. In order to better detect the violent behavior of individuals with mental disorders, the paper focuses on the UCF-Crime dataset with various types of violent behaviors. By experimenting with this dataset, the classification accuracy reaches 37.9%, which is significantly better than that of the current state-of-the-art approaches.

AN EFFICIENT CNN ARCHITECTURE FOR OBJECT DETECTION AND CLASSIFICATION IN IMAGES USING DEEP LEARNING

Object Classification has received wide attention due to its real time applications using deep learning technique. It is a process of classifying objects into predefined and semantically meaningful categories. Due to less accuracy ,sometimes gives inaccurate result and wouldn’t be effective. Object Classification using CNN algorithm is a proposed system in which image is inserted and object is classified and displayed in UI based on trained model.Inthis,it extracts features from the image, undergoes three architectures and high accuracy output will be considered to overcome existing difficulties. Objects can be easily detected and identified by humans. The visual system of human is very fast and accurate and can perform complex tasks like object identification and detection very easily .Situation like where we have to find from so many things and trying to find that key will take long time and we have to face some difficulties. Object classification is very important for applications in automatic visual surveillance system. The process of classifying objects into predefined and semantically meaningful categories using its features is called object classification. In this paper we propose a new model for detection and classification of objects by taking the features to classify the detected objects using Deep Neural Network (DNN). Deep Neural Networks are capable of handling large higher dimensional data with billions of parameters as like human brain. Simulation results obtained illustrate that the proposed classifier model produces more accurate results for feature extraction and DNN for classification.

Keypoint Message Passing for Video-Based Person Re-identification

Video-based person re-identification~(re-ID) is an important technique in visual surveillance systems which aims to match video snippets of people captured by different cameras. Existing methods are mostly based on convolutional neural networks~(CNNs), whose building blocks either process local neighbor pixels at a time, or, when 3D convolutions are used to model temporal information, suffer from the misalignment problem caused by person movement. In this paper, we propose to overcome the limitations of normal convolutions with a human-oriented graph method. Specifically, features located at person joint keypoints are extracted and connected as a spatial-temporal graph. These keypoint features are then updated by message passing from their connected nodes with a graph convolutional network~(GCN). During training, the GCN can be attached to any CNN-based person re-ID model to assist representation learning on feature maps, whilst it can be dropped after training for better inference speed. Our method brings significant improvements over the CNN-based baseline model on the MARS dataset with generated person keypoints and a newly annotated dataset: PoseTrackReID. It also defines a new state-of-the-art method in terms of top-1 accuracy and mean average precision in comparison to prior works.

A Multimodal Fusion-Based LNG Detection for Monitoring Energy Facilities (Student Abstract)

Fossil energy products such as liquefied natural gas (LNG) are among Canada's most important exports. Canadian engineers devote themselves to constructing visual surveillance systems for detecting potential LNG emissions in energy facilities. Beyond the previous infrared (IR) surveillance system, in this paper, a multimodal fusion-based LNG detection (MFLNGD) framework is proposed to enhance the detection quality by the integration of IR and visible (VI) cameras. Besides, a Fourier transformer is developed to fuse IR and VI features better. The experimental results suggest the effectiveness of the proposed framework.

Towards Building a Visual Behaviour Analysis Pipeline for Suicide Detection and Prevention.

Understanding human behaviours through video analysis has seen significant research progress in recent years with the advancement of deep learning. This topic is of great importance to the next generation of intelligent visual surveillance systems which are capable of real-time detection and analysis of human behaviours. One important application is to automatically monitor and detect individuals who are in crisis at suicide hotspots to facilitate early intervention and prevention. However, there is still a significant gap between research in human action recognition and visual video processing in general, and their application to monitor hotspots for suicide prevention. While complex backgrounds, non-rigid movements of pedestrians and limitations of surveillance cameras and multi-task requirements for a surveillance system all pose challenges to the development of such systems, a further challenge is the detection of crisis behaviours before a suicide attempt is made, and there is a paucity of datasets in this area due to privacy and confidentiality issues. Most relevant research only applies to detecting suicides such as hangings or jumps from bridges, providing no potential for early prevention. In this research, these problems are addressed by proposing a new modular design for an intelligent visual processing pipeline that is capable of pedestrian detection, tracking, pose estimation and recognition of both normal actions and high risk behavioural cues that are important indicators of a suicide attempt. Specifically, based on the key finding that human body gestures can be used for the detection of social signals that potentially precede a suicide attempt, a new 2D skeleton-based action recognition algorithm is proposed. By using a two-branch network that takes advantage of three types of skeleton-based features extracted from a sequence of frames and a stacked LSTM structure, the model predicts the action label at each time step. It achieved good performance on both the public dataset JHMDB and a smaller private CCTV footage collection on action recognition. Moreover, a logical layer, which uses knowledge from a human coding study to recognise pre-suicide behaviour indicators, has been built on top of the action recognition module to compensate for the small dataset size. It enables complex behaviour patterns to be recognised even from smaller datasets. The whole pipeline has been tested in a real-world application of suicide prevention using simulated footage from a surveillance system installed at a suicide hotspot, and preliminary results confirm its effectiveness at capturing crisis behaviour indicators for early detection and prevention of suicide.

Suspicious activity recognition for monitoring cheating in exams

Video processing is getting special attention from research and industries. The existence of smart surveillance cameras with high processing power has opened the for making it conceivable to design intelligent visual surveillance systems. Know a day it is very possible to assure invigilators safety during the examination period. This work aims to distinguish the suspicious activities of students during the exam for surveillance examination halls. For this, a 63 layers deep CNN model is suggested and named "L4-BranchedActionNet". The suggested CNN structure is centered on the alteration of VGG-16 with added four blanched. The developed framework is initially turned into a pre-trained framework by using the SoftMax function to train it on the CUI-EXAM dataset. The dataset for detecting suspicious activity is subsequently sent to this pre-trained algorithm for feature extraction. Feature subset optimization is applied to the deep features that have been obtained. These extracted features are first entropy coded, and then an ant colony system (ACS) is used to optimize the entropy-based coded features. The configured features are then input into a variety of classification models based on SVM and KNN. With a performance of 0.9299 in terms of accuracy, the cubic SVM gets the greatest efficiency scores. The suggested model was further tested on the CIFAR-100 dataset, and it was shown to be accurate to the tune of 0.89796. The result indicates the suggested frameworks soundness.

Vehicle and Person Re-Identification With Support Neighbor Loss.

One of the key tasks for an intelligent visual surveillance system is to automatically re-identify objects of interest, e.g., persons or vehicles, from nonoverlapping camera views. This demand incurs the vast investigation of person re-identification (re-ID) and vehicle re-ID techniques, especially those deep learning-based ones. While most recent algorithms focus on designing new convolutional neural networks, less attention is paid to the loss functions, which are of vital roles as well. Triplet loss and softmax loss are the two losses that are extensively used, both of which, however, have limitations. Triplet loss optimizes the model to produce features with which samples from the same class have higher similarity than those from different classes. The problem of triplet loss is that the number of triplets to be constructed grows cubically with training samples, which causes scalability issue, unstable performance, and slow convergence. Softmax loss has favorable scalable property and is widely used for large-scale classification problems. However, since Softmax loss only aims to separate well training classes, its performance for re-ID tasks is not desirable because the model is tested to measure the similarity of samples from unseen classes. We propose the support neighbor (SN) loss, which avoids the limitations of the abovementioned two losses. Unlike triplet loss that is calculated based on triplets, SN loss is derived from K -nearest neighbors (SNs) of anchor samples. The SNs of an anchor are unique, containing more valuable contextual information and neighborhood structure of the anchor, and thus contribute to more stable performance and reliable embedding from image space to feature space. Based on the SNs, a softmax-like separation term and a squeeze term are proposed, which encourage interclass separation and intraclass compactness, respectively. Experiments show that SN loss surpasses triplet and softmax losses with the same backbone network and reaches the state-of-the-art performance for both person and vehicle re-ID using a ResNet50 backbone when combined with training tricks.

An Efficient Unusual Event Tracking in Video Sequence using Block Shift Feature Algorithm

The area of video technology is rapidly growing owing to advancements in intelligent video systems in sensor operations, higher bandwidth capacity, storage, and high-resolution displays. This led to the proliferation of video-based computing modeling to perform specific tasks on video sequences to gain more insight from the data. Visual tracking of events is a core component in video visual surveillance systems that classify and track moving objects to describe their behavioral aspects. The prime motive behind intelligent video systems is to perform efficient video analytics to meet the specific requirements of the user/use-cases. It involves a self-directed paradigm to understand event sequences, reducing the computational burden of characterizing the activities. The study incorporates a block-shift feature algorithm and introduces a novel computational research method for unusual event tracking in video sequences. The formulated approach employs a framework combining operational blocks to compute sequential operations such as block-matching from the dictionary of motion estimations. Before applying the learning model, the subsequent analysis procedure adds feature lexicon and dominant attributes to make the execution computationally efficient. Further, it uses a sparse-non negative factorization approach to organize the informative details into k possible finite clusters. The event detection outcome from the training datasets of video sequences shows better experimental results than the traditional highly cited related approach of unusual object detection and tracking.

Person Reidentification Based on Automotive Radar Point Clouds

Person reidentification (ReID) systems play a key role in intelligent visual surveillance systems and have widespread applications, for example, in public security. Usually, person ReID systems can identify a person with cameras. In this article, we focus on the relatively unexplored area of using low-cost automotive radar for the person ReID problem. Unlike the radar-based person identification, person ReID has some characteristics, such as the uncooperative scenes and the long-term robustness. Therefore, we design a new deep learning network to extract spatiotemporal information from 4-D radar point clouds. We also build a data set of radar point clouds collected from the real-world person ReID scenarios. The evaluation result shows that our method achieves 91% CMC-1 accuracy on the ReID task. Besides, for the person identification task, our method also achieves accuracies of 98% and 91% for 15 and 40 individuals, respectively. In addition, we discuss the potential of using radar for person ReID problems and intuitively explain the new method’s performance. Finally, we analyze the robustness and the influence of different parameters on the method and the contributions of different modules to the network model. The results of our experiment indicate that radar-based ReID not only preserves privacy but also outperforms camera-based ReID in some cases, such as in low-light environments or with substantial clothing changes.

A Robust Layout-Independent License Plate Detection and Recognition Model Based on Attention Method

Auto License Plate Detection and Recognition (ALPDR) is required in many industrial fields, including visual surveillance systems and vehicle registration control. Even though this field has recently demonstrated high performance according to rapid advances in Deep Learning (DL) based technologies, there are still two problems that most related works have not yet solved. One is layout-dependent and the other is the problem of recognition accuracy degradation due to unconstrained environment conditions. In this paper, we present a more accurate, flexible and layout-independent method to improve LP detection and recognition accuracy under various outdoor conditions. For this, we proposed a new framework with lightweight and efficient anchor-free detection networks, employing the idea of CenterNet and attention-based recognition networks with residual deformable block suitable for LPR tasks. Various experiments demonstrate the performance of the proposed method, using famous LP benchmark datasets such as CCPD, AOLP and VBLPD. The proposed method outperformed the conventional LP detection and recognition methods. Additionally, Korean handicapped parking card (KHPC) data was tested to prove the usefulness of this method for marked character data different to license plates.

무인 감시 Transformer

In a visual surveillance system, even the same object should exhibit different detection results depending on the surrounding environment configuration. To this end, the model for visual surveillance needs to detect an object by understanding the state of the object according to the environment on the image. In this study, for such visual surveillance, an object segmentation model applied with a transformer structure suitable for image processing was used to divide objects inside the image into foreground and background. A modified attention structure was presented for the corresponding transformer structure, and the results of object segmentation models according to the type of input data were compared.