Smart Surveillance System Research Articles

Solving the imbalanced dataset problem in surveillance image blur classification

Surveillance videos taken in unconstrained environments can be tampered with due to different environmental factors and malicious human activities. They often blur the video content and introduce difficulty in identifying the events in the scene. The problem is particularly acute for smart surveillance systems that need to make real-time decisions based on the video. Automatic detection and classification of the blur anomalies in the video are crucial to these systems. Traditional learning-based classification methods often face imbalance problems in the sample numbers and distributions among the data classes in the dataset that severely affect their training and hence the classification performance. In this paper, a new learning-based approach for surveillance image blur classification is proposed. The imbalanced dataset problem is tackled both at the data and algorithm levels. At the data level, two synthesizers are developed to generate the required negative surveillance images to balance the sample numbers for all classes. At the algorithm level, an attention-based structure making use of the special feature of the minority class is proposed to improve the classification accuracy. Our experiment results show that the proposed approach significantly outperforms state-of-the-art methods for blur classification while keeping the model size small for edge applications.

Smart Accident Reporting and Surveillance System

Smart Accident Reporting and Surveillance System

A WSN and vision based smart, energy efficient, scalable, and reliable parking surveillance system with optical verification at edge for resource constrained IoT devices

As urbanization accelerates, the demand for efficient parking surveillance solutions has increased. However, existing solutions often face challenges related to energy consumption, scalability, and reliability. This paper introduces a smart hybrid parking surveillance system integrating wireless sensor networks (WSNs) with vision based solution at the edge for resource constrained IoT devices to address these challenges. The solution leverages WSNs for periodic readings of parking space occupancy and introduces a low power sleep mode in the network for energy efficiency, along with optical verification strategies using computer vision models like R-CNN and Faster R-CNN FPN on ResNet50 and MobileNetv2 backbones for distinguishing between true and false positives in the WSN data for a greater accuracy in parking space occupancy. The system utilizes edge for computing on edge servers resulting in increased responsiveness of the system, reduced data transmission and real time processing of data. The proposed solution is formulated in such a way that it automatically switches between WSN and vision based sensing resulting in less energy consumption and longer lifespan of the system without compromising on accuracy. Through experimental results it is observed that models trained on the MobileNetv2 backbone demonstrated at least twice faster for both processing the images and training compared to those models trained on the ResNet backbone. On the other hand, both Faster R-CNN FPN (input resolution: 1440) and R-CNN (input resolution: 128) models trained on the MobileNetv2 backbone have slightly lower accuracies than the same models trained on the ResNet50 backbone.

Unveiling Users' Security and Privacy Concerns Regarding Smart Home IoT Products from Online Reviews

The Internet of Things (IoT) has revolutionized the global market with lifestyle products such as fitness trackers (FT), smart home speakers (SHS), and surveillance and security camera systems (SSCS). While offering convenience, these products also introduce potential security and privacy (S&P) risks to buyers, often going unnoticed. Consumers’ incomplete mental models of the risks involved and the information asymmetry between buyers and sellers only add to the problem. Understanding consumer concerns in online product reviews can play a crucial role in bridging the gap of such information asymmetry. By establishing a balanced flow of information between buyers and sellers, manufacturers can leverage genuine concerns expressed in reviews to enhance product features while educating users about misinformation in reviews. In this study, we collected FT, SHS, and SSCS product reviews from three Amazon domains: the US, the UK, and India. Using a keyword-based search method focused on S&P concerns, we discovered a considerable number of reviews expressing notable concerns regarding security and privacy. Our qualitative analysis revealed that data security is a common concern across all product types. Further, our quantitative analysis exposed significant geographic variations, with the concern ratio being higher in the US than in the UK for all device types and higher than in the Indian domain for security cameras. These findings highlight the need for tailored security measures and user awareness campaigns in different parts of the world to address the identified concerns effectively.

A smart surveillance system utilizing modified federated machine learning: Gossip‐verifiable and quantum‐safe approach

SummaryEdge computing has the capability to process data closer to its point of origin, leading to the development of critical autonomous infrastructures with frequently communicating peers. The proposed work aims to evaluate the effectiveness of security and privacy mechanisms tailored for distributed systems, particularly focusing on scenarios where the nodes are closed‐circuit television (CCTV) systems. Ensuring public safety, object tracking in surveillance systems is a vital responsibility. The workflow has been specifically crafted and simulated for the purpose of weapon detection within public CCTV systems, utilizing sample edge devices. The system's primary objective is to detect any unauthorized use of weapons in public spaces while concurrently ensuring the integrity of video footage for use in criminal investigations. The outcomes of prior research on distributed machine learning (DML) techniques are compared with modified federated machine learning (FML) techniques, specifically designed for being Gossip verifiable and Quantum Safe. The conventional federated averaging algorithm is modified by incorporating the secret sharing principle, coupled with code‐based McEliece cryptosystem. This adaptation is designed to fortify the system against quantum threats. The Gossip data dissemination protocol, executed via custom blockchain atop the distributed network, serves to authenticate and validate the learning model propagated among the peers in the network. It provides additional layer of integrity to the system. Potential threats to the proposed model are analyzed and the efficiency of the work is assessed using formal proofs. The outcomes of the proposed work demonstrate that the trustworthiness and consistency are meticulously preserved for both the model and data within the DML framework on the Edge computing platform.

Smart surveillance system using ESP32 and camera-based motion detection with IM technology

Smart surveillance system using ESP32 and camera-based motion detection with IM technology

드론을 활용한 스마트감리시스템의 발전방향에 관한 연구

Currently, Korea's smart construction industry is becoming more advanced, and technology that can promote safety by incorporating ICT (Information & Communication Technology) convergence technology is being applied and developed. This study aims to provide information on the construction site to supervisors and builders by providing analysis data to the construction site using drones, etc., and to suggest development directions through smart supervision system improvement. In this study, after deriving major items through prior research, a focus interview was conducted with 7 supervisors to select important items for each research problem, and a total of 150 people, including construction project managers, manager supervisors, safety managers, and workers, were surveyed. As a result of the analysis of the survey on the satisfaction of smart supervision systems using drones at construction sites, the operation of drones will be highly useful in the construction field in the future (8.7, 25.92%), the smart supervision system using drones reflects what builders and supervisors want (8.3, 24.86%), and the drone is consistent with the goal of using smart supervision (8.3, 24.73%) and the drone system level is excellent (8.2, 24.49%). Through this study, it was found that the smart surveillance system using drones would be very useful in the construction field, and it was found that the efficiency of safety, accuracy, and economy could be improved by using drones instead of visually measuring and checking the field. It is hoped that the research results of this paper will provide the information necessary to improve the quality of the surveillance system in the construction sector at home and abroad and to prevent safety accidents at construction sites.

A hybrid cascade-parallel discriminative-generative model for pipeline integrity threat detection in a smart fiber optic surveillance system

A hybrid cascade-parallel discriminative-generative model for pipeline integrity threat detection in a smart fiber optic surveillance system

IoT-based Tracking Cattle Health Monitoring System Using Wireless Sensors

Maintaining the health of dairy cows is essential to raising the production of dairy products worldwide. Due to the numerous health problems that cows face, the unpredictability of disease outbreaks, and the high cost of producing new animals, dairy farmers are beginning to lose faith in the industry. To increase milk production, producers must thus employ effective technological methods for monitoring the health of their cows. This research examined many automated wireless sensor-based dairy cow health monitoring systems. The primary goal of smart surveillance systems based on wireless sensor networks (WSNs) in agricultural optimization is to continuously monitor the health of dairy cows. For interested farmers to keep an eye on their cattle's whereabouts throughout the day from many locations, this monitoring device needs to be deployed in both local and remote agricultural areas. A database would hold the information gathered by the automated system. Farmers can then use farm automation to acquire data so they can implement efficient farm management practices. Moreover, WSN is a low-cost tool designed specifically to detect diseases in dairy cows. This technological advancement in agricultural automation would help increase output by lowering the need for human intervention. This page provides an overview of all livestock tracking methods along with a summary of the problems and difficulties they face

IoT based Women Safety Smart System Development

Designing an effective system for women's safety is essential for reducing crimes against women and prosecuting the guilty. Women harassment at workplace, public transport etc., has been a rising concern all over the world. An early action and proper mechanism could reduce the number of crimes against women. The development of a Women Safety Smart System within the Internet of Things (IoT) domain represents a significant advancement in leveraging technology to address the pressing issue of women's safety. This paper outlines the design, implementation, and potential impact of such a system, which integrates IoT devices, sensors, and data analytics to create a proactive safety infrastructure. Key components of the system include IoT-enabled wearables, smart surveillance and monitoring systems, a dedicated mobile application, and predictive analytics capabilities. These components work synergistically to provide real-time monitoring, timely intervention, and community engagement in ensuring the safety and well-being of women. By harnessing IoT technology, this system offers a scalable and adaptable solution to mitigate risks, empower women, and foster societal change. The paper discusses the benefits, challenges, and future directions of Women Safety Smart Systems in the IoT domain, highlighting their potential to transform safety paradigms and promote inclusivity and equality in urban environments.

IoT Approach for Ensuring Safety in smart Cities for Energy Saving

The security of city dwellers is of the utmost importance due to the increasing number of smart cities and the speed with which they are being developed. As a whole, smart city safety can be improved through the use of the Internet of Things (IoT), which is the focus of this article. Several safety concerns, such as public safety, emergency management, and disaster preparedness, can be monitored and addressed with the help of the Internet of Things (IoT) framework's network of interconnected devices, sensors, and data analytics. In order to improve situational awareness, reaction times, and overall safety results, this research dives into important Internet of Things (IoT) applications such smart sensors, real-time analytics, and surveillance systems. In addition to looking at potential energy savings, the study analyses the pros and cons of using an Internet of Things (IoT) safety infrastructure in smart cities.

Smart Surveillance System for Highways

Abstract: In today's world, everyone on the planet is concerned about security. Every citizen wishes to eliminate the possibility of a future threat to himself or his possessions. This tremendous increase in the need for security needs for individuals and organizations necessitates the installation of an appropriate and up-to-date surveillance system. Real-time video surveillance has become an essential component of the transportation system. It also prevents accidents and other difficult issues on highways for a transportation system. With numerous advancements, various machine learning and artificial intelligence-based techniques are progressing to handle such issues via smart surveillance systems. Various researchers have contributed various techniques for vehicle detection and tracking to the literature. These methods are both effective and time-consuming. For the safety and security of people and their resources, proper surveillance is critical. The innovation of an aerial surveillance system could be very efficient at addressing surveillance system challenges. Current systems are both costly and complex. A cost-effective and efficient solution that is easily accessible to anyone with a moderate budget is required. Highways are a critical component for transportation purpose, and it is necessary to make them as secure as possible.

Tracking illegal activities using video surveillance systems: a review of the current state of research

The current state of research on the use of the neural networks under martial law to identify offenders committing illegal acts, prevent acts of terrorism, combat sabotage groups in cities, track weapons and control traffic is considered. The methods of detecting illegal actions, weapons, face recognition and traffic violations using video surveillance cameras are analysed. It is proposed to introduce the studied methods into the work of “smart” video surveillance systems in Ukrainian settlements.
 The most effective means of reducing the number of offences is the inevitability of legal liability for offences, so many efforts in law enforcement are aimed at preventing offences. Along with public order policing by patrol police, video surveillance is an effective way to prevent illegal activities in society. Increasing the coverage area of cameras and their number helps to ensure public safety in the area where they are used. However, an increase in the number of cameras creates another problem which is the large amount of video data that needs to be processed. To solve the problem of video data processing, various methods are used, the most modern of which is the use of artificial intelligence to filter a large amount of data from video cameras and the application of various video processing algorithms. The ability to simultaneously process video data from many CCTV cameras without human intervention not only contributes to public safety, but also improves the work of patrol police. The introduction of smart video surveillance systems allows monitoring the situation in public places around the clock, even if there is no police presence in the area.
 In the reviewed studies of video surveillance systems, neural networks, in particular MobileNet V2, YOLO, mYOLOv4-tiny, are used to track illegal actions, criminals and weapons, which are trained on large amounts of video and photo data. It has been found that although neural networks used to require a lot of computing power, they can now be used in IoT systems and smartphones, and this contributes to the fact that more video surveillance devices can be used to monitor the situation.

Resource-aware fog service placement using deferred acceptance in edge computing

Fog computing aims at reducing network latency, bandwidth and processing delay by bringing computation resources closer to the data sources. Modern IoT applications, such as smart surveillance systems, remote health monitoring applications, ambient-aware applications etc., are time-critical and computationally intensive. For such applications, it is ideal to consider an intermediate layer of processing i.e a fog layer in addition to cloud. However, processing the workload of such IoT applications with an intermediate layer of fog nodes has several challenges. The significant challenges include the mobility of end-user nodes, dynamic patterns of workload generation leading to highly-variant resource requirements and sensitive quality of service (QOS) requirements. Hence a dynamic approach is required to precisely map the service requests from end-user devices to a suitable fog node. The optimality of a fog node for service placement depends on several factors, including location, resource utilization rates such as CPU, Memory and Disk utilization % and energy consumption. Several meta-heuristic techniques and evolutionary algorithms have been proposed in the past for optimal service placement among fog nodes. This paper investigates the applicability of a game theory-based approach called Deferred acceptance (DA) for the same. The DA algorithm, originally designed as a stable matching mechanism for the marriage market is adapted for service placement problems in fog-based IoT systems. The effectiveness of optimized service placement through the DA approach is validated experimentally. The experiment is carried out by creating a test bed that mimics a fog computing-based intelligent traffic management application. The proposed approach's performance is evaluated through parameters such as the startup time for the DA approach to produce outcomes, total execution time and handshake latency. The numerical results of the DA approach are compared with those of a simple scheduling algorithm ( FIFO) and a basic genetic algorithm-based approach. The DA approach appears to perform on par with the genetic algorithm in terms of start-up time, but exceeds the genetic algorithm with respect to stability of the outcomes produced.

Transfer Learning for Abnormal Object Detection

In today's world, smart surveillance plays an important role in protecting security and creating a safe living environment. For abnormal objects in the smart surveillance system, this is an important issue, requiring attention and timely response from managers and supervisors. To address this issue, the paper uses transfer learning techniques on modern object detection models to detect abnormal objects such as guns, knives, etc. in public places. We experimented with the transfer learning method on the DETR model with a small dataset, and the model results showed a fairly fast convergence speed. Through this method, we hope to help reduce the burden of public security monitoring and warning work for managers, while technicians can use transfer learning techniques that are deployed in practice.

Smart surveillance using IoT: a review

In today’s modern society, video surveillance is a growing trend and it can revolutionize many aspects of technology, especially in future smart cities that will transform traditional surveillance systems into intelligent and interconnected networks. It may be difficult for even well-trained employees to process and respond immediately to monitored data. Moreover, IoT-enabled surveillance systems overcome the challenges and flaws of conventional passive monitoring techniques by offering real-time surveillance and automated notifications for suspicious activity, intrusions, or anomalies. Therefore, the objective of this article is to provide an Internet of Things (IoT)-based smart surveillance system that may successfully connect the ecosystem, resulting in enhanced monitoring for smart city services. The primary goal of this study is to provide a comprehensive review of several IoT-based surveillance techniques used in various smart city applications. The categorization of tasks for each section is as follows: to present the historical context overview; to examine the significance of IoT and its application in smart cities; to present a standardized architecture for IoT-based smart city surveillance; to highlight an authoritative and thorough review of current IoT-based smart surveillance systems; and to identify various research issues. The methods used are: statistical graph or chart approach, schematic, and timeline diagram. Conclusions. This article outlines numerous research challenges for future video surveillance that may be addressed by researchers. In summary, this comprehensive review provides a valuable and streamlined resource for future researchers exploring smart city surveillance through the IoT.

Federated Learning for Crowd Counting in Smart Surveillance Systems

Federated Learning for Crowd Counting in Smart Surveillance Systems

Ensemble-Based Knowledge Distillation for Video Anomaly Detection

Video anomaly detection has become a vital task for smart video surveillance systems because of its significant potential to minimize the video data to be analyzed by choosing unusual and critical patterns in the scenes. In this paper, we introduce three novel ensemble and knowledge distillation-based adaptive training methods to handle robust detection of different abnormal patterns in video scenes. Our approach leverages the adaptation process by providing information transfer from multiple teacher models with different network structures and further alleviates the catastrophic forgetting issue. The proposed ensemble knowledge distillation methods are implemented on two state-of-the-art anomaly detection models. We extensively evaluate our methods on two public video anomaly datasets and present a detailed analysis of our results. Finally, we show that not only does our best version model achieve comparable performance with a frame-level AUC of 75.82 to other state-of-the-art models on UCF-Crime as the target dataset, but more importantly our approaches prevent catastrophic forgetting and dramatically improve our model’s performance.

ESTS‐GCN: An Ensemble Spatial–Temporal Skeleton‐Based Graph Convolutional Networks for Violence Detection

Surveillance systems are essential for social and personal security. However, monitoring multiple video feeds with multiple targets is challenging for human operators. Therefore, automatic and smart surveillance systems have been introduced to support or replace traditional surveillance systems and build safer communities. Advancements in artificial intelligence techniques, particularly in the field of computer vision, have boosted this area of research. Most existing works have focused on image‐based (RGB‐based) machine learning and deep learning algorithms for detecting anomalous and violent events. In this study, we propose a unique Ensemble Spatial–Temporal Skeleton‐Based Graph Convolutional Networks (ESTS‐GCNs) model for violence detection that automatically uses spatial and temporal data to detect violence in surveillance videos. Skeleton‐based algorithms are less sensitive to pixel‐based noise and background interference, making them excellent candidates for activity and anomaly detection. Our proposed ensemble‐based architecture utilizes Graph Convolutional Networks (GCNs) and comprises multiple spatial and temporal modules. Three different spatial pipelines are exploited: channel‐wise topologies, self‐attention mechanism, and graph attention networks. The models were trained and evaluated using two skeleton‐based datasets introduced by us: Skeleton‐based Real‐Life Violence Situations (RLVS) and NTU‐Violence (NTU‐V). Our model achieved a maximum accuracy of around 93% and outperformed existing models by more than 10%.

Global-Local Multiple Granularity Learning for Cross-Modality Visible-Infrared Person Reidentification.

Cross-modality visible-infrared person reidentification (VI-ReID), which aims to retrieve pedestrian images captured by both visible and infrared cameras, is a challenging but essential task for smart surveillance systems. The huge barrier between visible and infrared images has led to the large cross-modality discrepancy and intraclass variations. Most existing VI-ReID methods tend to learn discriminative modality-sharable features based on either global or part-based representations, lacking effective optimization objectives. In this article, we propose a novel global-local multichannel (GLMC) network for VI-ReID, which can learn multigranularity representations based on both global and local features. The coarse- and fine-grained information can complement each other to form a more discriminative feature descriptor. Besides, we also propose a novel center loss function that aims to simultaneously improve the intraclass cross-modality similarity and enlarge the interclass discrepancy to explicitly handle the cross-modality discrepancy issue and avoid the model fluctuating problem. Experimental results on two public datasets have demonstrated the superiority of the proposed method compared with state-of-the-art approaches in terms of effectiveness.