Video Surveillance Research Articles

Nighttime large-field video image change detection based on adaptive superpixel reconstruction and multi-scale singular value decomposition fusion

With the development of technology and the needs of social governance, surveillance equipment has been widely used. It is very mature to detect the change of surveillance video images in conventional scenes through video image change detection algorithms. However, in the large field of view environment at night, there are complex random noise and low signal-to-noise ratio in surveillance video images, which makes it difficult for people to find small moving targets. To this end, we propose a new method for nighttime large-field surveillance video image change detection based on adaptive superpixel reconstruction and multi-scale singular value decomposition fusion. The proposed method consists of two parts. On the one hand, an adaptive superpixel reconstruction method is used to reconstruct the two denoised difference images by selecting different segmentation parameters, and the edge information of the two reconstructed difference images is significantly enhanced. On the other hand, a multi-scale singular value decomposition fusion method is used to fuse the two difference images. The multi-scale singular value decomposition fusion obtains a robust difference image by selecting fusion rules at different scales and using the complementary information of different difference images, and the Fuzzy c-means (FCM) clustering algorithm is used to obtain the final changed image. Experimental results on a self-built nighttime large-field video image dataset with two resolutions show that the proposed method is superior to other algorithms in terms of detection accuracy and robustness.

Human behavior-based COVID-19 transmission in two dining spaces

Since December 2019, the COVID-19 pandemic has rapidly disseminated globally, posing significant threats to the world. The dining spaces are high-risk indoor environments for the transmission of SARS-CoV-2, posing challenges for intervention and control. This study, based on surveillance videos from two COVID-19 outbreak cases in restaurants, obtained real data on human behaviors of close contact and surface touch. A respiratory infectious disease transmission model was developed, incorporating four transmission routes: short-range airborne, long-range airborne, fomite and large droplet. The results indicate that diners and staff spent 21.9 %−28.7 % and 17.5 %−27.8 % of their time on speaking, respectively, while spending 85.9 %−90.7 % and 83.4 %−87.6 % of their time on surface touching. The primary transmission routes were short-range (contributing 5.8 %−70.9 %) and long-range airborne (contributing 28.4 %−93.0 %), with fomite and large droplet routes contributing less than 12.0 %. Staff-only mask wearing reduced infection risk by 12.8 %−31.8 %. It is recommended that mandatory mask wearing for staff is necessary, while diners should wear masks as much as possible, and that the equivalent ventilation rate of clean fresh air is suggested to 30.0 m3/ (h·person). This study provides a scientific support to make non-pharmaceutical interventions in dinning spaces.

Balancing Accuracy and Training Time in Federated Learning for Violence Detection in Surveillance Videos: A Study of Neural Network Architectures

Balancing Accuracy and Training Time in Federated Learning for Violence Detection in Surveillance Videos: A Study of Neural Network Architectures

Crowd dynamics analysis and behavior recognition in surveillance videos based on deep learning

Crowd dynamics analysis and behavior recognition in surveillance videos based on deep learning

Robust Mixed-Rate Region-of-Interest-Aware Video Compressive Sensing for Transmission Line Surveillance Video

Classic video compression methods usually suffer from long encode time and requires large memories, making it hard to deploy on edge devices; thus, video compressive sensing, which requires less resources during encoding, is receiving more attention. We propose a robust mixed-rate ROI-aware video compressive sensing algorithm for transmission line surveillance video compression. The proposed method compresses foreground targets and background frames separately and uses reversible neural network to reconstruct original frames. The result on transmission line surveillance video data shows that the proposed compressive sensing method can achieve 26.47, 34.71 PSNR and 0.6839, 0.9320 SSIM higher than existing methods on 1.5% and 15% measurement rates, and the proposed ROI extraction net can precisely retrieve regions under high noise levels. This research not only demonstrates the potential for a more efficient video compression technique in resource-constrained environments, but also lays a foundation for future advancements in video compressive sensing techniques and their applications in various real-time surveillance systems.

Vehicle trajectory extraction and integration from multi-direction video on urban intersection

With the gradual maturity of computer vision technology, using intersection surveillance videos for vehicle trajectory extraction has become a popular method to analyze vehicle conflicts and safety in urban intersection. However, many intersection surveillance videos have blind spots, failing to fully cover the entire intersection. Vehicles may also obstruct each other, resulting in incomplete vehicle trajectories. The angle of surveillance videos can also lead to inaccurate trajectory extraction. In response to these challenges, this study proposes an vehicle trajectory extraction and integration framework using surveillance videos collected from four entrance of urban intersection. The framework first employs the improved YOLOv5s model to detect the positions of vehicles. Then, we proposed an object tracking model MS-SORT to extract the trajectories in each surveillance video. Subsequently, the trajectories of each surveillance video are mapped into the same coordinate system. Then the integration of trajectories is achieved using space–time information and re-identification (ReID) methods. The framework extracts and integrates trajectories from four intersection surveillance videos, obtaining trajectories with significantly broader temporal and spatial coverage compared to those obtained from any single direction of surveillance video. Our detection model improved mAP by 1.3 percentage points compared to the basic YOLOv5s, and our object tracking model improved MOTA and IDF1 by 2.6 and 2.1 percentage points compared to DeepSORT. The trojectory integration method achieved 94.7 % of F1-Score and RMSE of 0.51 m. The average length and number of the extracted trajectories has increased by at least 47.6 % and 24.2 % respectively compared to trajectories extracted from a single video.

Two-timescale joint service caching and resource allocation for task offloading with edge–cloud cooperation

Task offloading with edge–cloud cooperation has emerged as a pivotal solution for meeting the intricate array of application coupled with dynamically evolving business demand in 6G business scenarios, such as traffic sensing, environmental monitoring, and video surveillance in smart cities. Nonetheless, effectively leveraging heterogeneous edge–cloud network resources for effective task offloading presents substantial challenges. Additionally, the inherent differences in system decision cycles escalate the complexity of the task offloading problem to a new dimension. In this study, we delve into a two-timescale joint service caching and resource allocation optimization for task offloading within edge–cloud cooperation aiming to maximize long-term network performance while adhering to energy constraints. We propose a novel edge–cloud cooperation task offloading scheme that supports both edge–cloud and edge–edge cooperation to effectively balance the edge–cloud and edge–edge loads, promoting the efficient co-utilization of all edge–cloud system resources. Furthermore, we devise an online two-timescale Lyapunov-based joint optimization framework for service caching, task offloading, and computing resource allocation. Our two-timescale decision-making framework can flexibly accommodate the inherent differences in the sensitive decision optimization periods, thereby mitigating the degradation of task offloading performance caused by frequent service caching updates. Finally, theoretical analysis confirms that our proposed algorithm can converge to an approximate optimal solution in polynomial time, and the superiority of our scheme is validated by extensive simulation experiments.

Personnel Monitoring in Shipboard Surveillance Using Improved Multi-Object Detection and Tracking Algorithm.

Detecting and tracking personnel onboard is an important measure to prevent ships from being invaded by outsiders and ensure ship security. Ships are characterized by more cabins, numerous equipment, and dense personnel, so there are problems such as unpredictable personnel trajectories, frequent occlusions, and many small targets, which lead to the poor performance of existing multi-target-tracking algorithms on shipboard surveillance videos. This study conducts research in the context of onboard surveillance and proposes a multi-object detection and tracking algorithm for anti-intrusion on ships. First, this study designs the BR-YOLO network to provide high-quality object-detection results for the tracking algorithm. The shallow layers of its backbone network use the BiFormer module to capture dependencies between distant objects and reduce information loss. Second, the improved C2f module is used in the deep layer of BR-YOLO to introduce the RepGhost structure to achieve model lightweighting through reparameterization. Then, the Part OSNet network is proposed, which uses different pooling branches to focus on multi-scale features, including part-level features, thereby obtaining strong Re-ID feature representations and providing richer appearance information for personnel tracking. Finally, by integrating the appearance information for association matching, the tracking trajectory is generated in Tracking-By-Detection mode and validated on the self-constructed shipboard surveillance dataset. The experimental results show that the algorithm in this paper is effective in shipboard surveillance. Compared with the present mainstream algorithms, the MOTA, HOTP, and IDF1 are enhanced by about 10 percentage points, the MOTP is enhanced by about 7 percentage points, and IDs are also significantly reduced, which is of great practical significance for the prevention of intrusion by ship personnel.

Masked contrastive graph representation learning for age estimation

Age estimation of face images is a crucial task with various practical applications in areas such as video surveillance and Internet access control. While deep learning-based age estimation methods like Convolutional Neural Networks (CNN), Multilayer Perceptrons (MLP), and Transformers have shown remarkable performance, they have limitations when modeling complex or irregular objects, and often contain redundant information. Although existing graph neural networks (GNNs) can alleviate this issue, they only considers the structural information of the image and ignores the semantic features, thus the feature representation capability of graph nodes is limited. To address these issues, this paper proposes a novel Masked Contrastive Graph Representation Learning (MCGRL) method for accurate age estimation of face images. Our approach leverages CNN to extract semantic features of the image, which are then partitioned into patches that serve as nodes in the graph. We use a masked graph convolutional network (GCN) to derive image-based node representations that capture rich structural information. Finally, we incorporate multiple losses to explore the complementary relationship between structural information and semantic features, which improves the feature representation capability of GCN. Experimental results on real-world face image datasets demonstrate the superiority of our proposed method over other state-of-the-art age estimation approaches. Our code is available at https://github.com/yuntaoshou/MCGRL

A Trusted Supervision Paradigm for Autonomous Driving Based on Multimodal Data Authentication

At the current stage of autonomous driving, monitoring the behavior of safety stewards (drivers) is crucial to establishing liability in the event of an accident. However, there is currently no method for the quantitative assessment of safety steward behavior that is trusted by multiple stakeholders. In recent years, deep-learning-based methods can automatically detect abnormal behaviors with surveillance video, and blockchain as a decentralized and tamper-resistant distributed ledger technology is very suitable as a tool for providing evidence when determining liability. In this paper, a trusted supervision paradigm for autonomous driving (TSPAD) based on multimodal data authentication is proposed. Specifically, this paradigm consists of a deep learning model for driving abnormal behavior detection based on key frames adaptive selection and a blockchain system for multimodal data on-chaining and certificate storage. First, the deep-learning-based detection model enables the quantification of abnormal driving behavior and the selection of key frames. Second, the key frame selection and image compression coding balance the trade-off between the amount of information and efficiency in multiparty data sharing. Third, the blockchain-based data encryption sharing strategy ensures supervision and mutual trust among the regulatory authority, the logistic platform, and the enterprise in the driving process.

Person re-identification from UAVs based on Deep hybrid features: Application for intelligent video surveillance

Person re-identification from UAVs based on Deep hybrid features: Application for intelligent video surveillance

Stand-off concealed firearm detection using motion tracking and convolutional neural networks

<p>The standoff detection of concealed firearms is crutial in managing public security in public spaces. Currently employed standoff concealed weapon detection techniques employ electromagnetic wave imaging which has been found to be extremely slow and may require expensive hardware and may not be applicable in public open spaces. Inorder to maintain safety in open spaces, artificial intelligence enabled video surveillance systems have been widely adopted. This poses an opportunity to explore video surveillance cameras as concealed weapon detectors. A review of existing video surveillance based automated weapon detection approaches discovered that the focus was on the detection of unconcealed firearms leaving a gap in the detection of concealed firearms. This study addresses the aforementioned gap by providing a standoff concealed firearm detection approach on video based on skeletal-based human motion tracking and convolutional neural networks. The motion of armed and unarmed persons was tracked using a depth camera and further classified using convolutional neural networks model. The developed model reported 100% accuracy, precision and recall scores. These results outperformed results obtained from traditional machine learning models therefore highlighting the superior capability of the proposed approach for concealed firearm detection on video to complement the efforts of human video surveillance operators.</p>

Safeguards-related event detection in surveillance video using semi-supervised learning approach

We develop a deep learning model employing a semi-supervised learning approach, which can detect automatically safeguards-related events in nuclear facility from surveillance video. Our model is designed after a comprehensive analysis of the trends in artificial intelligence-based models to identify abnormal events in video. Our model incorporates a reconstruction module and a prediction module independently. The reconstruction module is trained to generate video frames within a sliding window, while the prediction module is trained to predict future motion feature based on the motion features within the video frames in a sliding window. Each module utilizes an autoencoder with a memory module positioned between an encoder and an decoder of the autoencoder. We evaluate the model's performance using a benchmark dataset and a self-produced dataset obtained from facility related to pyroprocessing. Our model's performanace is comparable to or superior to that of the prevous models from the benchmark dataset analysis, and all the abnormal events can be detected without false positive error from the self-produced dataset analysis.

Delving into CLIP latent space for Video Anomaly Recognition

We tackle the complex problem of detecting and recognising anomalies in surveillance videos at the frame level, utilising only video-level supervision. We introduce the novel method AnomalyCLIP, the first to combine Vision and Language Models (VLMs), such as CLIP, with multiple instance learning for joint video anomaly detection and classification. Our approach specifically involves manipulating the latent CLIP feature space to identify the normal event subspace, which in turn allows us to effectively learn text-driven directions for abnormal events. When anomalous frames are projected onto these directions, they exhibit a large feature magnitude if they belong to a particular class. We also leverage a computationally efficient Transformer architecture to model short- and long-term temporal dependencies between frames, ultimately producing the final anomaly score and class prediction probabilities. We compare AnomalyCLIP against state-of-the-art methods considering three major anomaly detection benchmarks, i.e. ShanghaiTech, UCF-Crime, and XD-Violence, and empirically show that it outperforms baselines in recognising video anomalies. Project website and code are available at https://lucazanella.github.io/AnomalyCLIP/.

Crime Activity Detection in Surveillance Videos Based on Developed Deep Learning Approach

In modern communities, lots of offenders are prone to recidivism, hence, there is a requirement to inhibit such criminals, especially from impending socioeconomically disadvantaged and high-crime areas that experience elevated levels of criminal activity, involving drug-related offenses, violence, theft, and other forms of anti-social behavior. Consequently, surveillance cameras have been installed in relevant institutions, and further personnel have been provided to monitor videos using various surveillance apparatus. However, relying solely on monitoring with the naked eye and manual video processing falls short of accurately evaluating the footage acquired via such cameras. To handle the issues of conventional systems, there is a need for a system that is able to classify acquired images while supporting surveillance personnel actively. Therefore, in this paper, a deep-learning approach is developed to build a crime detection system. This developed approach includes various layers necessary to perform feature extraction and classification processes and make the system capable of efficiently and accurately detecting crime activities from surveillance video frames. Besides the proposed crime activity detection system, two deep-learning approaches (EfficientNet-B7, and MobileNet-V2) are trained and assessed on the popular UCF Crime and DCSASS datasets. Generally, the proposed detection system encompasses dataset preparation and pre-processing, splitting the pre-processed crime activity image dataset, and implementing the proposed deep learning approach and other pre-trained approaches.

Consistency-constrained unsupervised video anomaly detection framework based on Co-teaching

Intelligent video surveillance continues to be a vibrant research domain within the field of computer vision. However, existing representation learning frameworks primarily focus on static information extraction frame by frame such as appearance features, they often overlook the valuable dynamic information like optical flow feature inherent in the video data, which is most essential characteristics of sequence data. To mining dynamic features and bridge this gap, our paper introduces a novel anomaly detection framework that balance dynamic information with static information and construct a relationship between appearance features and corresponding optical flow features, where we sets strong consistency constraints, which reduce the loss between dynamic information and corresponding static information, and we leverages collaborative teaching network to ensure a consistent representation of both static and dynamic information for predict. The proposed framework consists of two sets of encoder–decoder pairs complemented by a memory storage module. Operating in parallel with the dual encoder network is a Co-teaching network, with the shared memory module serving as the cornerstone for collaborative training. The Consistency constrained condition guarantees the strong consistency of dynamic and static information in the learned representations. In our experimental phase, we present compelling results that showcase the superior performance of our algorithm across three publicly available datasets.

A Survey: Systems and Applications Based on Cloud Computing

mobile cloud computing is an emerging technology that combines the power of cloud computing with the ubiquity of smartphones. This technology enables smart devices to access cloud computing resources. It offers various advantages, such as increased storage capacity, improved processing power, extended battery life, higher storage spaces, and faster response times. Mobile cloud computing has the potential to revolutionize mobile computing by providing users with anytime, anywhere access to cloud-based services and applications. However, mobile cloud computing faces many challenges, such as security, privacy, and network issues. This paper presents an overview of mobile cloud computing, its benefits, and challenges. We employed a descriptive approach to collect data on various programs and systems that rely on cloud computing, including automated video surveillance systems, health applications, MISSION SWACHHTA, car sharing for delivery and parcel customers, banking applications and systems, e-commerce platforms, big data systems, e-learning systems, and logistic systems.

Design and development of a motion activated internet protocol video surveillance system for crowd monitoring

Internet Protocol (IP) cameras are powerful tools in security and crowd monitoring implementation owing to their efficient communication over a wide geographical area. However, for any authorized personnel to initiate recording based on an event, to report a security breach, or even to stop recording when an uninteresting scene is presented, such would have to sit at the reach of the control which is not always possible. In this proposal, we implement a motion activated internet protocol (MAIP) based video surveillance system using network video management for crowd monitoring. The system can activate itself to initiate video recording on detecting motion or event using its in-built features of the network cameras and Video Management Software (VMS). The system will result in an effective and efficient motion detection video surveillance with necessary features that allow for quick recording, conserved bandwidth, greater storage, space conservation, reduces CPU load on recording servers and also allows for integration with other systems.

Multi-scale transformer network for super-resolution of visible and thermal air images

Reference image-based Super-Resolution (RefSR) is introduced to improve the quality of a Low-resolution (LR) input image by leveraging the additional information provided by a High-Resolution (HR) reference image (Ref). While existing RefSR methods focus on thermal or visible flows separately, they often struggle to enhance the resolution of small objects such as Mini/Micro UAVs (Unmanned Aerial Vehicle) due to the resolution disparities between the input and reference images. To cope with these challenges when dealing with UAV early detection in context of video surveillance, we propose ThermoVisSR, a multiscale texture transformer for enhancing the Super-Resolution (SR) of visible and thermal images of Mini/Micro UAVs. Our approach tries to reconstruct the fine details of these objects while preserving their approximation (the body form and color of the different scene objects) already contained in the LR image. Hence, our model is divided up into two streams dealing separately with approximation and detail reconstruction. In the first one, we introduce a Convolution Neural Network (CNN) fusion backbone to extract the Low-Frequency (LF) approximation from the original LR image pairs. In the second one and to extract the details from the Ref image, our approach involves blending features from both visible and thermal sources to make the most of what each offer. Subsequently, we introduce the High-Frequency Texture Transformer (HFTT) across various resolutions of the merged features to ensure an accurate correspondence matching and significant transfer of High-Frequency (HF) patches from Ref to LR images. Moreover, to adapt the injection to the different bands well, we incorporate the separable software decoder (SSD) into the HFTT allowing to capture channel-specific details during the reconstruction phase. We validated our approach using a newly created dataset of Air images of Mini/Micro UAVs. Experimental results demonstrate that the proposed model consistently outperforms the state-of-the-art approaches on both qualitative and quantitative assessments.

F3DNN-Net: behaviours violence detection via fine-tuned fused feature based deep neural network from surveillance video

F3DNN-Net: behaviours violence detection via fine-tuned fused feature based deep neural network from surveillance video