Video Surveillance Research Articles

Accelerated Data Engine: A faster dataset construction workflow for computer vision applications in commercial livestock farms

Large-scale, high-quality dataset was the foundation of developing advanced artificial intelligence applications. However, creating such a benchmark dataset in a professional field, such as precision management of animals, was always a challenge because of the costly and labor-intensive process of annotation and review. This study introduced a novel workflow named Accelerated Data Engine (ADE), designed to efficiently produce representative and high-quality computer vision datasets from raw animal surveillance footage. By incorporating referring and grounding models (R&G models) as auto-annotators, along with a distillation mechanism for dataset-auditors, ADE significantly speeded up the dataset construction process. The new workflow received natural language inputs as referrals to identify animal instances, delineated their body shapes, and then refined the auto-annotated data through a selection process. To demonstrate the efficacy of ADE, three 30-minute surveillance video samples featuring pigs, sheep, and cattle were discussed in this study. The results indicated the R&G models effectively annotated animals across various farms, while distillation mechanisms could identify various detection errors, balance the data representations, refine annotations, and verify the data quality. Two high-quality cattle datasets (6.5 k and 486 frames), including 26 k and 2.5 k cattle instances, were generated through the ADE workflow from 24-hour surveillance videos on a commercial cattle farm and made publicly available. The proposed dataset has achievable performance between 74.6 %∼84.1 %. The ADE workflow saved 78.4 % of manual work compared to the traditional dataset construction workflow (approximately 141 h). This pioneering approach empowered the fast creation of benchmark animal datasets and would enhance computer vision applications in the livestock production industry in the future.

Enhancing Small Target Detection in Aerial Imagery with Bi-PAN-FPN and EDMOA-Optimized YOLOv8-s

Across the globe, people are working to build "smart cities" that will employ technology to make people's lives better and safer. Installing cameras at strategic spots across the city to monitor public spaces besides provide real-time footage to law enforcement besides other local authorities is a crucial part of smart city infrastructure, which includes video surveillance. A more effective answer is provided by deep learning algorithms, however research in this area still faces significant problems from changes in target size, form change, occlusion, and illumination circumstances as seen from the drone's perspective. In light of the aforementioned issues, this study presents a highly effective and resilient approach for aerial picture identification. To begin, the concept of Bi-PAN-FPN is presented to enhance the neck component of YOLOv8-s, taking into consideration the prevalent issue of small targets being easily misdetected or ignored in aerial photos. We achieve a more advanced and thorough feature fusion procedure much as feasible by completely considering and reusing multiscale features. To further reduce the amount of parameters in the model and prevent info loss during long-distance feature transfer, the benchmark model's backbone incorporates the GhostblockV2 structure in lieu of a portion of the C2f module. With the help of the Enhanced Dwarf Mongoose Optimization Algorithm (EDMOA), the suggested model's hyper-parameters are optimised. Lastly, a dynamic nonmonotonic focusing mechanism is employed in conjunction with WiseIoU loss as bounding box regression loss. The detector accounts for varying anchor box quality by utilizing "outlier" evaluations, thus improving the complete presentation of the detection task.

Pose-oriented scene-adaptive matching for abnormal event detection

For intelligent surveillance systems, abnormal event detection automatically analyses surveillance video sequences and detects abnormal objects or unusual human actions at the frame level. Due to the lack of labelled data, most approaches are semi-supervised based on reconstruction or prediction methods. However, these methods may not generalize well to unseen scene contexts. To address this issue, we present a novel self and mutual scene-adaptive matching method for abnormal event detection. In the framework, we propose synergistic pose estimation and object detection, which effectively integrates human pose and object detection information to improve pose estimation accuracy. Then, the poses are resized to reduce the spatial distance between the source and target domains. The improved pose sequences are further fed into a spatio-temporal graph convolutional network to extract the geometric features. Finally, the features are embedded in a clustering layer to classify action types and compute normality scores. The training data is taken from the training part of common video anomaly detection datasets: UCSD PED1 & PED2, CHUK Avenue, and ShanghaiTech Campus. The proposed framework is evaluated on video sequences with unseen scene contexts in the UCSD PED2 and ShanghaiTech Campus datasets. The detection accuracy and efficiency are also evaluated in detail, and the proposed method for abnormal event detection achieves the highest AUC performance, 84.6%, on the ShanghaiTech Campus dataset and relatively high AUC performance, 96.9% and 74.8%, on UCSD PED2 & PED1 datasets. Compared with other state-of-the-art works, the performance analysis and results confirm the robustness and effectiveness of our proposed framework for cross-scene abnormal event detection.

Tropisetron, an Antiemetic Drug, Exerts an Anti-Epileptic Effect Through the Activation of α7nAChRs in a Rat Model of Temporal Lobe Epilepsy.

Temporal lobe epilepsy (TLE), a prevalent chronic neurological disorder, affects millions of individuals and is often resistant to anti-epileptic drugs. Increasing evidence has shown that acetylcholine (ACh) and cholinergic neurotransmission play a role in the pathophysiology of epilepsy. Tropisetron, an antiemetic drug used for chemotherapy in clinic, has displayed potential in the treatment of Alzheimer's disease, depression, and schizophrenia in animal models. However, as a partial agonist of α7 nicotinic acetylcholine receptors (α7nAChRs), whether tropisetron possesses the therapeutic potential for TLE has not yet been determined. In this study, tropisetron was intraperitoneally injected into pilocarpine-induced epileptic rats for 3 weeks. Alpha-bungarotoxin (α-bgt), a specific α7nAChR antagonist, was applied to investigate the mechanism of tropisetron. Rats were assessed for spontaneous recurrent seizures (SRS) and cognitive function using video surveillance and Morris's water maze testing. Hippocampal impairment and synaptic structure were evaluated by Nissl staining, immunohistochemistry, and Golgi staining. Additionally, the levels of glutamate, γ-aminobutyric acid (GABA), ACh, α7nAChRs, neuroinflammatory cytokines, glucocorticoids and their receptors, as well as synapse-associated protein (F-actin, cofilin-1) were quantified. The results showed that tropisetron significantly reduced SRS, improved cognitive function, alleviated hippocampal sclerosis, and concurrently suppressed synaptic remodeling and the m6A modification of cofilin-1 in TLE rats. Furthermore, tropisetron lowered glutamate levels without affecting GABA levels, reduced neuroinflammation, and increased ACh levels and α7nAChR expression in the hippocampi of TLE rats. The effects of tropisetron treatment were counteracted by α-bgt. In summary, these findings indicate that tropisetron exhibits an anti-epileptic effect and provides neuroprotection in TLE rats through the activation of α7nAChRs. The potential mechanism may involve the reduction of glutamate levels, enhancement of cholinergic transmission, and suppression of synaptic remodeling. Consequently, the present study not only highlights the potential of tropisetron as an anti-epileptic drug but also identifies α7nAChRs as a promising therapeutic target for the treatment of TLE.

Harnessing uncertainty: A deep mechanistic approach for cautious diagnostic and forecast of Bovine Respiratory Disease

Bovine Respiratory Disease (BRD) is a prevalent infectious disease of respiratory tract in cattle, presenting challenges in accurate diagnosis and forecasting due to the complex interactions of multiple risk factors. Common methods, including mathematical epidemiological models and data-driven approaches such as machine learning models, face limitations such as difficult parameter estimation or the need for data across all potential outcomes, which is challenging given the scarcity and noise in observing BRD processes. In response to these challenges, we introduce a novel approach known as the Bayesian Deep Mechanistic method. This method couples a data-driven model with a mathematical epidemiological model while accounting for uncertainties within the processes. By utilising 265 lung ultrasound videos as sensor data from 163 animals across 9 farms in France, we trained a Bayesian deep learning model to predict the infection status (infected or non-infected) of an entire batch of 12 animals, also providing associated confidence levels. These predictions, coupled with their confidence levels, were used to filter out highly uncertain diagnoses and diffuse uncertainties into the parameterisation of a mathematical epidemiological model to forecast the progression of infected animals. Our findings highlight that considering the confidence levels (or uncertainties) of predictions enhances both the diagnosis and forecasting of BRD. Uncertainty-aware diagnosis reduced errors to 32 %, outperforming traditional automatic diagnosis. Forecast relying on veterinarian diagnoses, considered the most confident, had a 23 % error, whilst forecast taking into account diagnosis uncertainties had a close 27.2 % error. Building upon uncertainty-awareness, our future research could explore integrating multiple types of sensor data, such as video surveillance, audio recordings, and environmental parameters, to provide a comprehensive evaluation of animal health, employing multi-modal methods for processing this diverse data.

AONet: Attention network with optional activation for unsupervised video anomaly detection

AbstractAnomaly detection in video surveillance is crucial but challenging due to the rarity of irregular events and ambiguity of defining anomalies. We propose a method called AONet that utilizes a spatiotemporal module to extract spatiotemporal features efficiently, as well as a residual autoencoder equipped with an attention network for effective future frame prediction in video anomaly detection. AONet utilizes a novel activation function called OptAF that combines the strengths of the ReLU, leaky ReLU, and sigmoid functions. Furthermore, the proposed method employs a combination of robust loss functions to address various aspects of prediction errors and enhance training effectiveness. The performance of the proposed method is evaluated on three widely used benchmark datasets. The results indicate that the proposed method outperforms existing state‐of‐the‐art methods and demonstrates comparable performance, achieving area under the curve values of 97.0%, 86.9%, and 73.8% on the UCSD Ped2, CUHK Avenue, and ShanghaiTech Campus datasets, respectively. Additionally, the high speed of the proposed method enables its application to real‐time tasks.

Human behavior recognition method based on wearable devices

Abstract Human behavior recognition remains a prevalent subject of inquiry in contemporary scientific studies. Behavior recognition technology has penetrated into every aspect of our lives, mainly in video surveillance, health monitoring, and smart homes. Aiming at some people who need behavioral monitoring to prevent danger, a human behavior recognition method based on wearable devices is proposed, which first acquires the three-axis acceleration data of behavioral activities of such people through wearable devices and then performs sliding window segmentation and feature extraction on the acquired data and finally inputs the obtained features into the dual-directional long and short term memory framework (BiLSTM) model to complete the identification of human behavior. To affirm the method’s reliability, we performed activity detection tests on a dataset from UCI featuring a non-powered wearable sensor used by the senior population. The outcomes indicate that the method is proficient in identifying the routine daily activities of this demographic, demonstrating its practical utility.

Clustering Aided Weakly Supervised Training to Detect Anomalous Events in Surveillance Videos.

Formulating learning systems for the detection of real-world anomalous events using only video-level labels is a challenging task mainly due to the presence of noisy labels as well as the rare occurrence of anomalous events in the training data. We propose a weakly supervised anomaly detection system that has multiple contributions including a random batch selection mechanism to reduce interbatch correlation and a normalcy suppression block (NSB) which learns to minimize anomaly scores over normal regions of a video by utilizing the overall information available in a training batch. In addition, a clustering loss block (CLB) is proposed to mitigate the label noise and to improve the representation learning for the anomalous and normal regions. This block encourages the backbone network to produce two distinct feature clusters representing normal and anomalous events. An extensive analysis of the proposed approach is provided using three popular anomaly detection datasets including UCF-Crime, ShanghaiTech, and UCSD Ped2. The experiments demonstrate the superior anomaly detection capability of our approach.

An NFMF-DBiLSTM model for human anomaly detection system in surveillance videos

An NFMF-DBiLSTM model for human anomaly detection system in surveillance videos

Design of an Emergency Medical Information System for Mass Gatherings

BackgroundThe frequency of mass gatherings is increasing. Such events often involve many people and carry the risk of mass casualty incidents, which require substantial medical resources from various healthcare institutions. The current medical system, while meeting daily needs, struggles to address the demand for a high volume of emergency resources and real-time data exchange. ObjectiveThe aim of this study was to develop an emergency medical information system for mass gatherings. MethodsWe developed an emergency medical information system for mass gatherings. Based on a unified prehospital and intrahospital emergency data exchange protocol, we can directly standardize medical information data and provide data support for the evacuation decision support algorithms of multiple institutions. Wearable devices, vehicle-mounted devices, video calling systems and surveillance systems are connected to capture real-time scenes. ResultsWe constructed the system via mobile applications and online platforms and deployed it in 3 hospitals, 5 ambulances and 17 on-site medical locations. We constructed a set of electronic medical records covering the whole first aid process according to the basic principles of first aid. The simulation results show that the proposed algorithm is suitable for mass gatherings. The overall survival rate of patients can be improved by 5%, and the average evacuation efficiency of patients can be improved by 50%. Furthermore, in a real-world environment, this method can ensure patient survival and achieve good convergence. ConclusionsOur system is capable of providing robust medical information support for emergency medical services during large-scale assembly events, ensuring a visualized full-process emergency response and decision-making for the diversion and subsequent transport of a large patient population.

A Comprehensive Comparative Analysis of Deep Learning Architectures for Suspicious Activity Detection in Video Surveillance

Abstract: In the landscape of modern security infrastructure, video surveillance has evolved into a ubiquitous and indispensable tool for ensuring public safety and safeguarding critical assets. This research paper delves into an extensive examination of various deep learning architectures for the purpose of detecting suspicious activities in video surveillance. The investigation encompasses Convolutional Long Short-Term Memory (ConvLSTM), Convolutional Neural Network (CNN) combined with Long Short-Term Memory (LSTM), ConvLSTM, Bidirectional Long Short-Term Memory (BiLSTM), and diverse combinations thereof. Each model is rigorously trained and tested on a carefully curated dataset designed to encapsulate a spectrum of normal and suspicious activities like shooting and fighting. The study aspires to identify the most efficacious model for enhancing the accuracy of suspicious activity detection in complex and dynamic environments. Metrics such as accuracy, precision, recall, and F1 score will be rigorously assessed to ascertain the model’s comparative performances

Exploring Techniques for Abnormal Event Detection in Video Surveillance

This study explores and evaluates the effectiveness of various abnormal event detection techniques in video surveillance, addressing challenges such as intrusions, accidents, and suspicious activities. Through a systematic review of related papers, the study reveals the prevalence of traditional methods like background subtraction and motion detection despite their limitations in complex scenarios. It highlights the increasing use of deep learning techniques, particularly CNNs and RNNs, which show promise but require substantial labeled data. The findings underscore the importance of selecting proper detection techniques based on specific surveillance scenarios and emphasize the need for extensive labeled datasets for deep learning methods. The originality of this study lies in its comprehensive review and comparison of various abnormal event detection techniques, providing valuable insights and practical implications for advancing video surveillance systems.

Principles of suspicious activity detection and programs for analyzing suspicious activity using video surveillance cameras

This article delves into the principles of detecting suspicious activity, meticulously examining both logical and psychological facets of human behavior. It places a primary focus on diverse programs for analyzing suspicious activity using video surveillance cameras. A comprehensive overview is provided for key categories of suspicious activity, encompassing crowd clustering, rapid movement, intrusion into secure areas, and unattended items. Within the research framework, the advantages and drawbacks of leading programs for suspicious activity analysis through video surveillance are scrutinized. Furthermore, the article introduces the concept of an innovative program aimed at competing with existing solutions. This software aspires to amalgamate the strengths of competitors while avoiding their shortcomings. The research goal is to identify the potential for creating a highly effective software product capable of successfully competing and meeting the requirements of modern surveillance systems.

Anomaly detection in crowded scenes: technologies, challenges and opportunities

The paper discusses advancements in intelligent video surveillance systems, particularly focused on anomaly detection in crowded environments. These systems aim to enhance public safety by automatically detecting unusual behavior and potential threats in real-time. Traditional video surveillance, relying heavily on human monitoring, faces limitations like reduced concentration and slow response times. In contrast, intelligent surveillance uses machine learning and AI algorithms to process vast amounts of video data, identifying patterns that deviate from normal behavior. Crowd anomaly detection is essential in densely populated areas like transportation hubs, stadiums, and public squares. The diversity of anomalies, ranging from minor disruptions to serious threats such as theft or terrorist attacks, presents a challenge for these systems. Anomalies can be difficult to detect due to their unpredictable nature, and what constitutes an anomaly varies depending on the context. The paper highlights the need for robust systems that can adapt to various environmental conditions and distinguish between normal variations and genuine threats. While supervised machine learning models show promise, they often require large amounts of labeled data, which is difficult to obtain in real-world settings. Unsupervised models and deep learning techniques, such as Convolution Neural Networks, have been effective in analyzing crowd behavior and detecting anomalies. However, these methods still face challenges, including scalability, high false positive rates, and the need for real-time processing in large-scale environments. The paper concludes by addressing the limitations of current crowd anomaly detection methods, such as their computational costs, ethical concerns, and inability to detect novel anomalies. It suggests directions for future research, including the integration of advanced learning techniques to improve system performance and scalability

Accurate crowd counting for intelligent video surveillance systems

The paper presents a novel deep learning approach for crowd counting in intelligent video surveillance systems, addressing the growing need for accurate monitoring of public spaces in urban environments. The demand for precise crowd estimation arises from challenges related to security, public safety, and efficiency in urban areas, particularly during large public events. Existing crowd counting techniques, including feature-based object detection and regression-based methods, face limitations in high-density environments due to occlusions, lighting variations, and diverse human figures. To overcome these challenges, the authors propose a new deep encoder-decoder architecture based on VGG16, which incorporates hierarchical feature extraction with spatial and channel attention mechanisms. This architecture enhances the model’s ability to manage variations in crowd density, leveraging adaptive pooling and dilated convolutions to extract meaningful features from dense crowds. The model’s decoder is further refined to handle sparse and crowded scenes through separate density maps, improving its adaptability and accuracy. Evaluations of the proposed model on benchmark datasets, including Shanghai Tech and UCF CC 50, demonstrate superior performance over state-of-the-art methods, with significant improvements in mean absolute error and mean squared error metrics. The paper emphasizes the importance of addressing environmental variability and scale differences in crowded environments and shows that the proposed model is effective in both sparse and dense crowd conditions. This research contributes to the advancement of intelligent video surveillance systems by providing a more accurate and efficient method for crowd counting, with potential applications in public safety, transportation management, and urban planning.

Tasks of visual crowd analysis in intelligent video surveillance systems

Modern urban population growth creates challenges for public safety, in particular due to crowds. This stimulates the development of new crowd management methods that require automated analysis. Visual crowd analysis based on computer vision technologies is a key tool for solving these problems. The development of deep learning has significantly improved the monitoring systems used for urban surveillance, social distancing control, transportation and event management. However, crowd analysis remains challenging due to occlusions, scale variations, unpredictable movement patterns, and complex behavior. To overcome these challenges, new algorithms, models, and large-scale datasets are needed to enable real-time analysis. The main tasks include people counting, object detection, motion analysis, behavior recognition, and anomaly detection. Deep neural networks and transfer learning significantly increase the accuracy and adaptability of such systems, which helps to improve public safety and manage the flow of people.

Intelligent Video Analytics for Abnormal Event Detection

Surveillance cameras have become a primary tool for monitoring human movement and preventing unwanted or unintended activities. In today's world, security management professionals increasingly rely on video surveillance to combat crime and mitigate incidents that could adversely affect society. The number of surveillance camera installations has dramatically increased in both the private and public sectors, serving to monitor public activities effectively. Video surveillance is considered one of the most efficient methods for ensuring security. Installing a surveillance camera allows the captured footage to be transmitted to security personnel; however, merely having video footage is insufficient for identifying abnormal activities. To effectively analyze this footage, it is essential to integrate an intelligent system.This paper aims to design and implement an Intelligent Video Analytics Model (IVAM), also known as the Human Object Detection (HOD) method, for analyzing and detecting video-based anomalies and abnormal human activities. IVAM can be deployed alongside surveillance cameras in various public locations such as Institutions, airports, hospitals, shopping malls, and railway stations, allowing for the automatic identification of unusual events. The IVAM was experimented with using MATLAB software, and the results were thoroughly verified. The performance of IVAM was assessed by comparing the obtained results with those from existing approaches, demonstrating that IVAM outperforms contemporary methods concerning accurate anomaly detection, lower error rates, and higher classification accuracy.

Pedestrian Re-Identification Based on Weakly Supervised Multi-Feature Fusion

This article proposes a weakly supervised multi-feature fusion pedestrian re-identification method, which introduces a multi-feature fusion mechanism to extract feature information from different layers into the same feature space and fuse them into the deep and shallow joint features. The goal is to fully utilize the rich information in the image and improve the performance and robustness of the pedestrian re-identification model. Secondly, by matching the target character with unprocessed surveillance videos, one only needs to know that the identity of a person appears in the video, without annotating the identity of a person in any of the frames of the video during the training process. This simplifies the annotation of training images by replacing accurate annotations with broad annotations; that is, it puts the pedestrian identities that appeared in the video in one package and assigns a video-level label to each package. This greatly reduces the annotation work and transforms this weakly supervised pedestrian re-identification challenge into a multi-instance and multi-label learning problem. The experimental results show that the method proposed in this paper is effective and can significantly improve mAP.

Enhancing Inland Waterway Safety and Management through Machine Learning-Based Ship Detection

Efficient ship detection is essential for inland waterway management. Recent advances in artificial intelligence have prompted research in this field. This study introduces a real-time ship detection model utilizing computer vision and the YOLO object detection framework. The model is designed to identify and locate common inland waterway vessels, such as container ships, passenger vessels, barges, ferries, canoes, fishing boats, and sailboats. Data augmentation techniques were employed to enhance the model's ability to handle variations in ship appearance, weather, and image quality. The system achieved a mean Average Precision (mAP) of 98.4%, with precision and recall rates of 96.6% and 95.0%, respectively. These results demonstrate the model's effectiveness in practical applications. Its ability to generalize across diverse vessel types and environmental conditions suggests its potential integration into video surveillance for improved maritime safety, traffic control, and search and rescue operations.

Optimized deep maxout for crowd anomaly detection: A hybrid optimization-based model

ABSTRACT Monitoring Surveillance video is really time-consuming, and the complexity of typical crowd behaviour in crowded situations makes this even more challenging. This has sparked a curiosity about computer vision-based anomaly detection. This study introduces a new crowd anomaly detection method with two main steps: Visual Attention Detection and Anomaly Detection. The Visual Attention Detection phase uses an Enhanced Bilateral Texture-Based Methodology to pinpoint crucial areas in crowded scenes, improving anomaly detection precision. Next, the Anomaly Detection phase employs Optimized Deep Maxout Network to robustly identify unusual behaviours. This network’s deep learning capabilities are essential for detecting complex patterns in diverse crowd scenarios. To enhance accuracy, the model is trained using the innovative Battle Royale Coalesced Atom Search Optimization (BRCASO) algorithm, which fine-tunes optimal weights for superior performance, ensuring heightened detection accuracy and reliability. Lastly, using various performance metrics, the suggested work’s effectiveness will be contrasted with that of the other traditional approaches. The proposed crowd anomaly detection is implemented in Python. On observing the result showed that the suggested model attains a detection accuracy of 97.28% at a learning rate of 90%, which is much superior than the detection accuracy of other models, including ASO = 90.56%, BMO = 91.39%, BES = 88.63%, BRO = 86.98%, and FFLY = 89.59%.