Smart Surveillance System Research Articles

Dual-Stage Attribute Embedding and Modality Consistency Learning-Based Visible–Infrared Person Re-Identification

Visible–infrared person re-identification (VI-ReID) is an emerging technology for realizing all-weather smart surveillance systems. To address the problem of pedestrian discriminative information being difficult to obtain and easy to lose, as well as the wide modality difference in the VI-ReID task, in this paper we propose a two-stage attribute embedding and modality consistency learning-based VI-ReID method. First, the attribute information embedding module introduces the fine-grained pedestrian information in the attribute label into the transformer backbone, enabling the backbone to extract identity-discriminative pedestrian features. After obtaining the pedestrian features, the attribute embedding enhancement module is utilized to realize the second-stage attribute information embedding, which reduces the adverse effect of losing the person discriminative information due to the deepening of network. Finally, the modality consistency learning loss is designed for constraining the network to mine the consistency information between two modalities in order to reduce the impact of modality difference on the recognition results. The results show that our method reaches 74.57% mAP on the SYSU-MM01 dataset in All Search mode and 87.02% mAP on the RegDB dataset in IR-to-VIS mode, with a performance improvement of 6.00% and 2.56%, respectively, proving that our proposed method is able to reach optimal performance compared to existing state-of-the-art methods.

AI-driven behavior biometrics framework for robust human activity recognition in surveillance systems

The integration of artificial intelligence (AI) into human activity recognition (HAR) in smart surveillance systems has the potential to revolutionize behavior monitoring. These systems analyze an individual’s physiological or behavioral features to continuously monitor and identify any unusual or suspicious activity in video streams, thereby improving security and surveillance measures. Traditional surveillance systems often rely on manual human monitoring, which is resource-intensive, error-prone, and time-consuming. To address these limitations, computer vision-based behavior biometrics has emerged as a solution for secure video surveillance applications. However, implementing automated HAR in real-world scenarios is challenging due to the diversity of human behavior, complex spatiotemporal patterns, varying viewpoints, and cluttered backgrounds. To tackle these challenges, an AI-based behavior biometrics framework is introduced that is based on a dynamic attention fusion unit (DAFU) followed by a temporal-spatial fusion (TSF) network to effectively recognize human activity in surveillance systems. In the first phase of the proposed framework, a lightweight EfficientNetB0 backbone is enhanced by the DAFU to extract human-centric salient features using a unified channel-spatial attention mechanism. In the second phase, the DAFU features with fixed sequence lengths are passed to the proposed TSF network to capture the temporal, spatial, and behavioral dependencies in video data streams. The integration of Echo-ConvLSTM in the TSF further enhances accuracy and robustness by combining temporal dependencies from the echo state network with spatial and temporal dependencies from the convolutional long short-term memory (ConvLSTM). The proposed AI-based behavior biometrics framework is evaluated using four publicly available HAR datasets (UCF101, HMDB51, UCF50, and YouTube Action), yielding higher accuracies of 98.734%, 80.342%, 98.987%, and 98.927%, which demonstrate the superior performance when compared with state-of-the-art (SOTA) methods.

Evaluation of Elderly Fall Detection Systems using Data Analysis

As the global population continues to age, it is more important to ensure the safety and wellbeing of senior citizens. Falls are a frequent and dangerous concern for elders, frequently resulting in fatalities or very serious injuries. Modern technology, such as the Elderly Fall Detection System, have been created to solve this issue. This technology attempts to detect falls quickly and react to them, offering aid right away and maybe saving lives. These studies identify a fall when the tri-axial accelerometer reading from a wearable device exceeds the predetermined threshold. Less complexity and computational expense compared to other approaches is one of the main benefits of adopting threshold-based methods. Finding the right threshold value, however, to accurately identify all falls without confounding them with certain ADL, has proven to be a challenging issue. Early Intervention and Medical Aid: Being independent and able to live in one's own house are two things that many elderly people enjoy. Support for carers and peace of mind: detection systems not only aid the elderly but also provide their carers peace of mind. Cost-cutting: Falls among the elderly frequently lead to hospital stays, rehab, and higher healthcare expenses. The Weighted Product Model (WPM) is a technique for ranking and evaluating alternatives based on a variety of factors. It is an easy and obvious method that enables decision-makers to weigh the relative weight of many variables and make wise decisions. Smart surveillance system, Smart cane, Smart Carpet, Smart phone/watch and app. Unobtrusiveness, reliability, privacy, Cost. As the global population continues to age, it is more important to ensure the safety and wellbeing of senior citizens. Falls are a frequent and dangerous concern for elders, frequently resulting in fatalities or very serious injuries The Weighted Product Model (WPM) is a technique for ranking and evaluating alternatives based on a variety of factors Unobtrusiveness, reliability, privacy, Cost, smart phone is the highest and smart carpet is the lowest

Smart surveillance using deep learning

<span>Smart surveillance systems play an important role in security today. The goal of security systems is to protect users against fires, car accidents, and other forms of violence. The primary function of these systems is to offer security in residential areas. In today’s culture, protecting our homes is critical. Surveillance, which ranges from private houses to large corporations, is critical in making us feel safe. There are numerous machine learning algorithms for home security systems; however, the deep learning convolutional neural network (CNN) technique outperforms the others. The Keras, Tensorflow, Cv2, Glob, Imutils, and PIL libraries are used to train and assess the detection method. A web application is used to provide a user-friendly environment. The flask web framework is used to construct it. The flash-mail, requests, and telegram application programming interface (API) apps are used in the alerting approach. The surveillance system tracks abnormal activities and uses machine learning to determine if the scenario is normal or not based on the acquired image. After capturing the image, it is compared with the existing dataset, and the model is trained using normal events. When there is an anomalous event, the model produces an output from which the mean distance for each frame is calculated.</span>

IoT based tracking cattle healthmonitoring system using wireless sensors

Controlling the health of dairy cows is crucial for increasing global dairy food output. Dairy farmers are losing faith in the sector as cows suffer from a range of health issues, disease outbreaks that are unpredictable, and expensive breeding expenses. As a consequence, farmers must use efficient technology techniques for cow health monitoring in order to enhance milk output. This study looked at a range of wireless sensor-based automated dairy cow health monitoring systems. The fundamental purpose of wireless sensor network (WSN) based smart surveillance systems in agricultural optimization is to follow the health of dairy cows on a continual basis. This monitoring gadget must be installed in both local and remote farm areas so that interested farmers may monitor their cattle's movements throughout the day from several places. The data collected by the automated system would be kept in a database. Farmers may then obtain data using farm automation to execute effective farm management techniques. Furthermore, WSN is a low-cost device created exclusively for identifying illnesses in dairy cows. This achievement in sophisticated technology agricultural automation would assist to boost productivity by reducing human involvement. This article summarises all livestock tracking techniques, as well as the issues and challenges that they confront.

Handling illumination variation for motion detection in video through intelligent method: An application for smart surveillance system

Handling illumination variation for motion detection in video through intelligent method: An application for smart surveillance system

Study of image sensors for enhanced face recognition at a distance in the Smart City context

Smart monitoring and surveillance systems have become one of the fundamental areas in the context of security applications in Smart Cities. In particular, video surveillance for Human Activity Recognition (HAR) applied to the recognition of potential offenders and to the detection and prevention of violent acts is a challenging task that is still undergoing. This paper presents a method based on deep learning for face recognition at a distance for security applications. Due to the absence of available datasets on face recognition at a distance, a methodology to generate a reliable dataset that relates the distance of the individuals from the camera, the focal length of the image sensors and the size in pixels of the target face is introduced. To generate the extended dataset, the Georgia Tech Face and Quality Dataset for Distance Faces databases were chosen. Our method is then tested and applied to a set of commercial image sensors for surveillance cameras using this dataset. The system achieves an average accuracy above 99% for several sensors and allows to calculate the maximum distance for a sensor to get the required accuracy in the recognition, which could be crucial in security applications in smart cities.

CNN-ViT Supported Weakly-Supervised Video Segment Level Anomaly Detection.

Video anomaly event detection (VAED) is one of the key technologies in computer vision for smart surveillance systems. With the advent of deep learning, contemporary advances in VAED have achieved substantial success. Recently, weakly supervised VAED (WVAED) has become a popular VAED technical route of research. WVAED methods do not depend on a supplementary self-supervised substitute task, yet they can assess anomaly scores straightway. However, the performance of WVAED methods depends on pretrained feature extractors. In this paper, we first address taking advantage of two pretrained feature extractors for CNN (e.g., C3D and I3D) and ViT (e.g., CLIP), for effectively extracting discerning representations. We then consider long-range and short-range temporal dependencies and put forward video snippets of interest by leveraging our proposed temporal self-attention network (TSAN). We design a multiple instance learning (MIL)-based generalized architecture named CNN-ViT-TSAN, by using CNN- and/or ViT-extracted features and TSAN to specify a series of models for the WVAED problem. Experimental results on publicly available popular crowd datasets demonstrated the effectiveness of our CNN-ViT-TSAN.

IoT-Based Smart Surveillance System for High-Security Areas

The world we live in today is becoming increasingly less tethered, with many applications depending on wireless signals to ensure safety and security. Proactive security measures can help prevent the loss of property due to actions such as larceny/theft and burglary. An IoT-based smart Surveillance System for High-Security Areas (SS-HSA) has been developed to address this issue effectively. This system utilizes a Gravity Microwave Sensor (GMS), which is highly effective due to its ability to penetrate nonmetallic obstructions. Combining GMS with Arduino UNO is a highly effective technique for detecting suspected objects behind walls. The GMS can also be integrated with the global system for mobile (GSM) communications, making it an IoT-based solution. The SS-HSA system utilizes machine learning AI algorithms operating at a GMS frequency to analyze and calculate accuracy, precision, F1-Scores, and Recall. After a thorough evaluation, it was determined that the Random Forest Classifier achieved an accuracy rate of 95%, while the Gradient Boost Classifier achieved an accuracy rate of 94%. The Naïve Bayes Classifier followed closely behind with a rate of 93%, while the K Nearest Neighbor and Support Vector Machine both achieved an accuracy rate of 96%. Finally, the Decision Tree algorithm outperformed the others in terms of accuracy, presenting a value of 97%. Furthermore, in the studied machine learning AI algorithms, it was observed that the Decision Tree was optimal for SS-HSA.

Anomaly Prediction over Human Crowded Scenes via Associate-Based Data Mining and K-Ary Tree Hashing

Anomaly detection and behavioral recognition are key research areas widely used to improve human safety. However, in recent times, with the extensive use of surveillance systems and the substantial increase in the volume of recorded scenes, the conventional analysis of categorizing anomalous events has proven to be a difficult task. As a result, machine learning researchers require a smart surveillance system to detect anomalies. This research introduces a robust system for predicting pedestrian anomalies. First, we acquired the crowd data as input from two benchmark datasets (including Avenue and ADOC). Then, different denoising techniques (such as frame conversion, background subtraction, and RGB-to-binary image conversion) for unfiltered data are carried out. Second, texton segmentation is performed to identify human subjects from acquired denoised data. Third, we used Gaussian smoothing and crowd clustering to analyze the multiple subjects from the acquired data for further estimations. The next step is to perform feature extraction to multiple abstract cues from the data. These bag of features include periodic motion, shape autocorrelation, and motion direction flow. Then, the abstracted features are mapped into a single vector in order to apply data optimization and mining techniques. Next, we apply the associate-based mining approach for optimized feature selection. Finally, the resultant vector is served to the k-ary tree hashing classifier to track normal and abnormal activities in pedestrian crowded scenes.

Designing an Autonomous Triggering Control System via Motion Detection for IoT Based Smart Home Surveillance CCTV Camera

Security is essential in our daily lives, whether in a personal, home, business, or industrial setting. As a result, to ensure high security, a complicated surveillance system is required. This is typically not cost effective, necessitates more consumable electronic devices, requires more maintenance, and needs more storage space in settings. Hence, the goal of this research is to create a single, small, and simple IOT-based smart closed-circuit television (CCTV) control system. A low-cost smart security system can well be built using the Raspberry Pi microprocessor. The components involve are a camera, a sensor, and a Raspberry Pi board. The proposed smart surveillance system creates a comfortable and safe environment for households to initiate immediate recording of the surrounding upon triggering of sensors assemble at the specific entrance or exit points at home. Raspberry Pi microprocessor is used to be integrated with closed-circuit surveillance camera with motion detection, allowing smart automation of home surveillance system at a cost-effective manner. When the Raspberry Pi is connected to the Internet, the user may even remotely turn on or off the CCTV using the smartphone application. The system was tested with movement at the sensor from a range of distances based on the location setting. It can deliver the anticipated output such as alarm triggering, automated initiating recording of surrounding from the CCTV camera, sending an alert notification to the user's mobile phone, allowing monitoring of the location setting through the user's device, as well as turning off the CCTV camera through the user's device via the developed system.

Development of SmartTrack with Authentication for Tracking Objects Business Promotion

Abstract: One of the most important and difficult duties of a smart video surveillance system is object tracking. The major purpose of this is to pinpoint the precise location of a specific item and to highlight that location with a buzzer and light. We considered secrecy when designing. We must protect the private documents. Therefore, we need to offer users permission before granting access to the location, buzzer, and light. This system was specifically created with authorization and confidentiality in mind. The various object tracking techniques that the researchers used are thoroughly reviewed in this publication. The purpose of this essay isto review numerous ways to object tracking and to examine, study, and evaluate them

Secure Gait Recognition-Based Smart Surveillance Systems Against Universal Adversarial Attacks

Currently, the internet of everything (IoE) enabled smart surveillance systems are widely used in various fields to prevent various forms of abnormal behaviors. The authors assess the vulnerability of surveillance systems based on human gait and suggest a defense strategy to secure them. Human gait recognition is a promising biometric technology, but one significantly hindered because of universal adversarial perturbation (UAP) that may trigger system failure. More specifically, in this research study, the authors emphasize on sample convolutional neural network (CNN) model design for gait recognition and assess its susceptibility to UAPs. The authors compute the perturbation as non-targeted UAPs, which trigger a model failure and lead to an inaccurate label to the input sample of a given subject. The findings show that a smart surveillance system based on human gait analysis is susceptible to UAPs, even if the norm of the generated noise is substantially less than the average norm of the images. Later, in the next stage, the authors illustrate a defense mechanism to design a secure surveillance system based on human gait.

Application of Smart Surveillance System in National Security


 Surveillance has been considered essential in maintaining order and discipline since the dawn of time. This paper explores various domains revolving around the Smart surveillance system ranging from the primitive definition, and how it deviates from traditional surveillance to how we can implement the system on a fundamental level which can be scaled up to work on national-level systems. With the rising criminal activity in today’s world, there is a need for a proposed solution in surveillance applications that can examine human behavior and recognize people with criminal pasts and detect abnormal activities. This article focuses on proposing advancements in Surveillance Systems in Nepal such that Artificial Intelligence (AI) systems will be able to recognize people with criminal pasts and also keep track of a person's behaviour to predict if any acts of violence are about to take place or not. The research aims to explore the potential for such a system in Nepal and other places around the world, including questions such as: what specific types of AI Surveillance are the governments deploying and “The feasibility and challenges of adopting a Smart Surveillance System in Nepal”. The research methodology consists of secondary sources of data to collect the information. A comparative document study, a quick survey with ICT experts, journal articles, and research on current technologies, found that Nepal is not far behind in technological advancements though it lacks policy-level intervention on AI which shows Nepal can enhance its national security by adopting an AI-powered Surveillance System.

Privacy Preserved Video Summarization of Road Traffic Events for IoT Smart Cities

The purpose of smart surveillance systems for automatic detection of road traffic accidents is to quickly respond to minimize human and financial losses in smart cities. However, along with the self-evident benefits of surveillance applications, privacy protection remains crucial under any circumstances. Hence, to ensure the privacy of sensitive data, European General Data Protection Regulation (EU-GDPR) has come into force. EU-GDPR suggests data minimisation and data protection by design for data collection and storage. Therefore, for a privacy-aware surveillance system, this paper targets the identification of two areas of concern: (1) detection of road traffic events (accidents), and (2) privacy preserved video summarization for the detected events in the surveillance videos. The focus of this research is to categorise the traffic events for summarization of the video content, therefore, a state-of-the-art object detection algorithm, i.e., You Only Look Once (YOLOv5), has been employed. YOLOv5 is trained using a customised synthetic dataset of 600 annotated accident and non-accident video frames. Privacy preservation is achieved in two steps, firstly, a synthetic dataset is used for training and validation purposes, while, testing is performed on real-time data with an accuracy from 55% to 85%. Secondly, the real-time summarized videos (reduced video duration to 42.97% on average) are extracted and stored in an encrypted format to avoid un-trusted access to sensitive event-based data. Fernet, a symmetric encryption algorithm is applied to the summarized videos along with Diffie–Hellman (DH) key exchange algorithm and SHA256 hash algorithm. The encryption key is deleted immediately after the encryption process, and the decryption key is generated at the system of authorised stakeholders, which prevents the key from a man-in-the-middle (MITM) attack.

AI Based Smart Surveillance System

The AI Based smart surveillance system is gaining huge attention because of rise in demand for safety and security. The surveillance system is designed to analyse the video, image, and audio or any kind of surveillance data automatically without any human involvement. The developments that happened in recent years in the computer vision, sensor devices and Auto ML is playing a keen role in accrediting such intelligent system. There are many kinds of surveillance and security systems present in the market, but there is no live decision making and predictive analysis surveillance i.e., automated self-decision making system which helps the different public departments like police, health, fire and many more to track and reach the particular location where the incident is happened. In this proposed project we provide the AI based intelligent surveillance and security system which analyses and takes the decision immediately by itself based on the present parametric conditions as per the modules trained by us, It helps in providing the alerts quicker compared to the traditional alert system. The system which is proposed here will react automatically based on the situations occurs as it is trained in various modes. This paper is intending to provide the generalised outline of the AI based smart surveillance system and its functionalities. This paper also consists of information regarding the core processing steps of AI based surveillance system such as tracking, object detection and classification, and behavioural analysis.

IoT-based Dual Technology Motion Detector

Home security has become a prime concern in recent years. As technology is emerging every second, abundant IoT-based smart surveillance systems have been developed and implemented with many modern features to keep the home safe. The proposed project is an extended approach to the existing home security system. The project aims to reduce false alarm rates of the existing home security system by using a Dual Technology motion sensor. Dual-technology sensors are sensors that combine two different types of technology. The dual-technology sensor uses both PIR and ultrasonic sensors, activating the trigger only when both sensors detect any motion of an object. The proposed setup effectively reduces false alarms caused by inanimate objects. The trigger caused by the Dual Technology sensor sends a message alert to the specified phone number through the Twilio Programmable Messaging API. Ultrasonic technology is sensitive to motion toward and away from the detector; passive infrared technology (PIR) is sensitive to motion across the field of view. The future implications of the project are great as it can also be used in the Automation of lighting in homes, offices, and libraries.

Region-of-interest based video coding strategy for rate/energy-constrained smart surveillance systems using WMSNs

Video surveillance systems that involve embedded visual sensor nodes have significant constraints due to their limited energy sources. Reducing the power consumption of the in-node processing and the required bandwidth while maintaining a high QoS is still a challenging task. The difficulty increases when a smart task must be performed on the received video in the destination. In this context, this paper proposes an energy-efficient video coding strategy based on a new and fast region-of-interest (ROI) detection method. The lightweight ROI detection method segments the frame into four regions. And the coding strategy aims to extract two different classes of the ROI for coding and transmission using variable quality levels based on their relevance. Furthermore, the strategy aims to exclude the regions of lower importance and any non-ROI that has insignificant movement. We assess the strategy’s ability to perform object recognition tasks at the destination under quality degradation. The performance results using different datasets demonstrate a better trade-off between awareness of ROI quality, energy consumption and bandwidth savings for the proposed strategy compared to other methods. This results in 96% reduction of bandwidth and 93% reduction in energy for some sequences at the expense of a 1-4 dB decrease in PSNR when compared to MJPEG standard. While the recognition accuracy of the YOLOv3 model at the destination outperforms the other techniques by about 4% to 22%.

CNN hyper-parameter optimization for environmental sound classification

Environmental sounds are being used widely in our lives. It is especially used in tasks such as managing smart cities, location determination, surveillance systems, machine hearing, and environmental monitoring. The main method for this, environmental sound classification (ESC), has been increasingly studied in recent years. However, the classification of these sounds is more difficult than other sounds because there are too many parameters that generate noise. The study tried to find the convolutional neural network (CNN) model that gave the highest accuracy for ESC tasks with the optimization of hyper-parameters. For this purpose, the Particle Swarm Optimization (PSO) algorithm was rearranged to represent the CNN architecture. Thus, the hyper-parameters in CNN are represented exactly without any transformation during optimization. Studies were carried out on the ESC-10, ESC-50, and Urbansound8k data sets, which are state-of-art for ESC tasks. Some data augmentation techniques have been used for data sets in the training of CNN models. The CNN models, which were obtained with PSO, achieved success rates of 98.64 % for ESC-10, 93.71 % for ESC-50, and 98.45 % for Urbansound8k, respectively. These results are the best accuracy values obtained with the pure CNN model when compared with previous studies. As a result, it has been made possible to automatically design CNN models for the classification of urban sounds, giving high classification accuracy. Thus, researchers who do not know much about CNN design can use this method in their desired datasets without the need for expert knowledge.

Bio-Inspired Vision and Neuromorphic Image Processing Using Printable Metal Oxide Photonic Synapses

We propose a neuromorphic vision system that uses digitally printed metal oxide photonic synapses with optical tunability and temporally correlated plasticity. The neuromorphic vision system provides the first demonstration of encoding ambient light intensity in the time domain and captures optical images by encoding their pixel intensity into pulsatile signals in real time, analogous to a biological visual retina and optic nerve. The system can then process the information and form memory, emulating the image data processing in the human brain. The system can realize image-preprocessing functions to increase image contrast and reduce image background noise, thereby effectively improving the classification and recognition accuracy. Dynamic stimulation of visual cortical prosthetics shows that the system is capable of dynamic perception and memory to detect motion and mimic coherent visual perception. Furthermore, the photonic synapses with a two-terminal structure using printed semiconducting fiber arrays could also facilitate large-scale integration. The proposed system offers the potential to simplify neuromorphic visual circuits and may have applications in autonomous smart devices, such as driverless cars, smart surveillance systems, and intelligent healthcare.