Abnormal Behavior Detection Research Articles

Research on Object Detection in Complex Scenarios Based on ASA‐YOLOv5

The applications of target detection in complex scenarios cover a wide range of fields, such as pedestrian and vehicle detection in self‐driving cars, face recognition and abnormal behavior detection in security monitoring systems, hazardous materials safety detection in public transportation, and so on. These applications demonstrate the importance and the prospect of wide application of target detection techniques in solving practical problems in complex scenarios. However, in these real scenes, there are often problems such as mutual occlusion and scale change. Therefore, how to accurately identify the target in the real complex scenarios has become a big problem to be solved. In order to solve the above problem, the paper proposes a novel algorithm, Adaptive Self‐Attention‐YOLOv5 (ASA‐YOLOv5), which is built upon the YOLOv5s algorithm and demonstrates effectiveness for target identification in complex scenarios. First, the paper implements a fusion mechanism between the trunk and neck networks, enabling the fusion of features across different levels through upsampling and downsampling. This fusion process mitigates detection errors caused by feature loss. Second, the Shuffle Attention mechanism is introduced before upsampling and downsampling to suppress noise and amplify essential semantic information, further enhancing target identification accuracy. Lastly, the Adaptively Spatial Feature Fusion (ASFF) module and Receptive Field Blocks (RFBs) module are added in the head network, and it can improve feature scale invariance and expand the receptive field. The ability of the model to detect the target in the complex scene is improved effectively. Experimental results indicate a notable improvement in the model's mean Average Precision (mAP) by 2.1% on the COCO dataset and 0.7% on the SIXray dataset. The proposed ASA‐YOLOv5 algorithm can enhance the effectiveness for target detection in complex scenarios, and it can be widely used in real‐world settings. © 2024 Institute of Electrical Engineers of Japan and Wiley Periodicals LLC.

A Dual Transformer-Based Deep Learning Model for Passenger Anomaly Behavior Detection in Elevator Cabs

Effective detection of abnormal behaviors within elevator cabins is critical to ensure elevator safety. While existing deep learning based anomaly detection methods mainly focus on convolutional neural networks for spatial feature extraction and recurrent networks for temporal feature learning, recent advancements in the Transformer architecture have demonstrated its power in time series predictions, and extended its capabilities to vision detection tasks. In this study, we present a duel transformer-based framework that can proficiently detect falling and fighting events in elevator cabs. The proposed solution leverages the vision transformer (ViT) to extract frame-level spatial features, followed by a temporal Transformer to identify abnormalities in surveillance videos. A comprehensive comparison between the proposed transformer-based method and other traditional recurrent neural network variants is carried out to validate the effectiveness of the method.

ACD-Net: An Abnormal Crew Detection Network for Complex Ship Scenarios.

Abnormal behavior of crew members is an important cause of frequent ship safety accidents. The existing abnormal crew recognition algorithms are affected by complex ship environments and have low performance in real and open shipborne environments. This paper proposes an abnormal crew detection network for complex ship scenarios (ACD-Net), which uses a two-stage algorithm to detect and identify abnormal crew members in real-time. An improved YOLOv5s model based on a transformer and CBAM mechanism (YOLO-TRCA) is proposed with a C3-TransformerBlock module to enhance the feature extraction ability of crew members in complex scenes. The CBAM attention mechanism is introduced to reduce the interference of background features and improve the accuracy of real-time detection of crew abnormal behavior. The crew identification algorithm (CFA) tracks and detects abnormal crew members' faces in real-time in an open environment (CenterFace), continuously conducts face quality assessment (Filter), and selects high-quality facial images for identity recognition (ArcFace). The CFA effectively reduces system computational overhead and improves the success rate of identity recognition. Experimental results indicate that ACD-Net achieves 92.3% accuracy in detecting abnormal behavior and a 69.6% matching rate for identity recognition, with a processing time of under 39.5 ms per frame at a 1080P resolution.

Anomaly Detection and Analysis in Nuclear Power Plants

Industries are increasingly adopting digital systems to improve control and accessibility by providing real-time monitoring and early alerts for potential issues. While digital transformation fuels exponential growth, it exposes these industries to cyberattacks. For critical sectors such as nuclear power plants, a cyberattack not only risks damaging the facility but also endangers human lives. In today’s digital world, enormous amounts of data are generated, and the analysis of these data can help ensure effectiveness, including security. In this study, we analyzed the data using a deep learning model for early detection of abnormal behavior. We first examined the Asherah Nuclear Power Plant simulator by initiating three different cyberattacks, each targeting a different system, thereby collecting and analyzing data from the simulator. Second, a Bi-LSTM model was used to detect anomalies in the simulator, which detected it before the plant’s protection system was activated in response to a threat. Finally, we applied explainable AI (XAI) to acquire insight into how distinctive features contribute to the detection of anomalies. XAI provides valuable explanations of model behavior by revealing how specific features influence anomaly detection during attacks. This research proposes an effective anomaly detection technique and interpretability to better understand counter-cyber threats in critical industries, such as nuclear plants.

SH-YOLO: Small Target High Performance YOLO for Abnormal Behavior Detection in Escalator Scene

SH-YOLO: Small Target High Performance YOLO for Abnormal Behavior Detection in Escalator Scene

A Review of Abnormal Crowd Behavior Recognition Technology Based on Computer Vision

Abnormal crowd behavior recognition is one of the research hotspots in computer vision. Its goal is to use computer vision technology and abnormal behavior detection models to accurately perceive, predict, and intervene in potential abnormal behaviors of the crowd and monitor the status of the crowd system in public places in real time, to effectively prevent and deal with public security risks and ensure public life safety and social order. To this end, focusing on the abnormal crowd behavior recognition technology in the computer vision system, a systematic review study of its theory and cutting-edge technology is conducted. First, the crowd level and abnormal behaviors in public places are defined, and the challenges faced by abnormal crowd behavior recognition are expounded. Then, from the dimensions based on traditional methods and based on deep learning, the mainstream technologies of abnormal behavior recognition are discussed, and the design ideas, advantages, and limitations of various methods are analyzed. Next, the mainstream software tools are introduced to provide a comprehensive reference for the technical framework. Secondly, typical abnormal behavior datasets at home and abroad are sorted out, and the characteristics of these datasets are compared in detail from multiple perspectives such as scale, characteristics, and uses, and the performance indicators of different algorithms on the datasets are compared and analyzed. Finally, the full text is summarized and the future development direction of abnormal crowd behavior recognition technology is prospected.

CSSDet: small object detection via cross-scale feature enhancement on drone-view images

ABSTRACT Object detection on drone-view images is vital for applications like intelligent transportation, abnormal behavior detection, and urban surveillance. However, the diverse perspectives and altitudes from which Unmanned Aerial Vehicle (UAV) capture scenes pose challenges due to multi-scale differences and small object sizes. To address this, we propose CSSDet, emphasizing multi-scale object detection and small object enhancement. CSSDet integrates an additional detection head for small objects and employs a bidirectional weighted feature pyramid network for effective cross-scale feature fusion, enhancing global perceptual capability. Additionally, we introduce coordinate attention and Involution modules to mitigate information loss. Experiments are conducted on two publicly available datasets, VisDrone and UAVDT. Quantitative results from the experiments indicate that the proposed CSSDet achieves optimal performance across a wide range of evaluation metrics, with particular strengths demonstrated in multi-scale object detection and small-size object detection. Qualitative results showcase its remarkable effectiveness in various aspects of object detection, including small object detection, objects at different scales and categories, multi-perspective scenarios, and complex scenes. The substantial experimental findings confirm that CSSDet attains comprehensive object detection performance on drone-view images. The source code is available at https://github.com/Gui-Cheng/CSSDet.

A Two-Stream Method for Human Action Recognition Using Facial Action Cues.

Human action recognition (HAR) is a critical area in computer vision with wide-ranging applications, including video surveillance, healthcare monitoring, and abnormal behavior detection. Current HAR methods predominantly rely on full-body data, which can limit their effectiveness in real-world scenarios where occlusion is common. In such situations, the face often remains visible, providing valuable cues for action recognition. This paper introduces Face in Action (FIA), a novel two-stream method that leverages facial action cues for robust action recognition under conditions of significant occlusion. FIA consists of an RGB stream and a landmark stream. The RGB stream processes facial image sequences using a fine-spatio-multitemporal (FSM) 3D convolution module, which employs smaller spatial receptive fields to capture detailed local facial movements and larger temporal receptive fields to model broader temporal dynamics. The landmark stream processes facial landmark sequences using a normalized temporal attention (NTA) module within an NTA-GCN block, enhancing the detection of key facial frames and improving overall recognition accuracy. We validate the effectiveness of FIA using the NTU RGB+D and NTU RGB+D 120 datasets, focusing on action categories related to medical conditions. Our experiments demonstrate that FIA significantly outperforms existing methods in scenarios with extensive occlusion, highlighting its potential for practical applications in surveillance and healthcare settings.

A Novel WTG Method for Predicting Ship Trajectories in the Fujian Inshore Area Based on AIS Data

The increasing congestion in major global maritime routes poses significant threats to international maritime safety, exacerbated by the proliferation of large, high-speed vessels. To improve the detection of abnormal ship behavior, this research employed automatic identification system (AIS) data for ship trajectory forecasting. Traditional methods primarily focus on spatial and temporal correlations but often lack accuracy and reliability. In this study, ship path predictions were enhanced using the WTG model, which combines wavelet transform, temporal convolutional networks (TCN), and gated recurrent units (GRU). Initially, wavelet decomposition was applied to deconstruct the input trajectory time series. The TCN and GRU modules then extracted features from both the time series and the decomposed data. The predicted elements were reassembled using a multi-head attention mechanism and a pooling layer to produce the final predictions. Comparative experiments demonstrated that the WTG model surpasses other models in the accuracy of ship trajectory prediction. The model proposed in this study proves to be reliable for forecasting ship paths, which is crucial for marine traffic management and ensuring safe navigation.

Research on Anomaly Detection of Ship Behavior Based on Data Mining

With the increase in the number of ships and the intensifying maritime traffic, maritime accidents such as ship collisions and grounding occur frequently, posing serious threats to people's lives and property. Therefore, timely detection and correction of abnormal ship behaviors are of great significance for improving navigation safety. The Automatic Identification System (AIS) can provide real-time information on ships' positions, speeds, headings, and more, offering a rich data resource for data mining. Through the analysis of these data, it is possible to deeply explore the navigation patterns and characteristics of abnormal behaviors of ships. This paper applies data mining technology to the detection of abnormal ship behaviors, which not only improves the accuracy and efficiency of detection but also provides maritime regulatory authorities with richer decision-making support.

Temporally enhanced abnormal behavior detection based on multi-channel coupling

Abnormal behavior detection in surveillance videos based on deep learning has becomea hot topic in current research. However, due to the complexity and variability of crowd movement and external environment, detecting abnormal behavior faces great challenges. Current abnormal behavior recognition models have limitations in feature extraction and pay insufficient attention to dynamic temporal features. To address these problems, a spatiotemporal enhancement anomaly detection method based on multi-channel coupling is proposed. Based on the SlowFast network, a multi-channel coupled temporal enhancement module and a spatial enhancement module are introduced respectively to extract more discriminative static and dynamic feature information. A large number of experiments are carried out on three benchmark datasets (Violent Flow, Hockey Fight, and Real-life Violence Situations). The results show that the prediction accuracy of the proposed abnormal behavior recognition method reaches 95.3%、97.3%and respectively 94%, thus verifying the effectiveness of the method.

Discrete Variational Feature Transfer Learning for Skeleton-based Abnormal Behavior Detection

Discrete Variational Feature Transfer Learning for Skeleton-based Abnormal Behavior Detection

Image quilting heuristic compressed sensing video privacy protection coding for abnormal behavior detection in private scenes

For video intelligence applications in private scenes such as home environments, traditional image processing methods are usually based on clear raw data and are prone to privacy leakage. Therefore, our team proposed multilayer compressed sensing (MCS) encoding to reduce image quality for visual privacy protection (VPP). However, the way in which MCS coding is implemented leads to unavoidable information loss. On this basis, inspired by the image quilting (IQ) algorithm, an image quilting heuristic MCS (IQ-MCS) coding method is proposed in this paper to improve the problem of faster information loss in the MCS coding process, which means that a similar privacy protection effect is achieved at lower coding layers, thus obtaining better application performance. To evaluate the level of VPP, a VPP evaluation algorithm is proposed that is more in line with subjective assessment. Finally, a correlation model between the VPP level and the performance of smart applications is established to balance the relationships between them, taking the detection of abnormal human behavior in private scenes as an example. The model can also provide a reference for the evaluation of other privacy protection methods.

Abnormal behavior detection in industrial control systems based on CNN

With the widespread application of Internet of Things technology in industrial control systems, abnormal behavior detection has become a key task to ensure the safety and stable operation of the system. We propose a multi-branch convolutional fusion neural network method to improve the accuracy and efficiency of abnormal behavior detection in Internet of Things industrial control systems. This method achieves efficient detection of abnormal behavior in video data by combining ResNet152 for spatial feature extraction and GRU for temporal feature extraction. Unlike traditional methods, this method adopts a multi-branch structure, which can simultaneously capture multi-scale feature information, significantly enhancing the richness of feature expression and the accuracy of detection. Experimental results show that on the UCF-Crime dataset, the accuracy of this method reaches 85.76%, which is significantly better than that of traditional methods. In addition, on the larger UCF-101 dataset, the accuracy of this method reaches 92.21%, further verifying its excellent generalization performance. Compared with the C3D network, this method improves the accuracy by nearly 6% while maintaining a high processing speed. These results show that the proposed method has great potential in practical applications.

SSRLM: A self-supervised representation learning method for identifying one ship with multi-MMSI codes

It is of great practical significance for maritime safety and shipping development to achieve “One Ship with One Maritime Mobile Service Identity (MMSI) Code” in accordance with regulations. However, the current “One Ship with Multiple MMSI Codes” violation identification method lacks practicability and generalization. Aiming at the these problems, this paper proposes a self-supervised representation learning model for the “One Ship with Multiple MMSI Codes” abnormal behavior recognition task, named SSRLM. Firstly, we construct a new dataset about “One Ship with Multiple MMSI Codes”, named HN_MulMI. Secondly, the SSRLM model learns the contextual interdependencies among different trajectory features in the HN_MulMI dataset through an unsupervised pre-training scheme, and extracts a dense vectorial representation of the ship’s motion trajectories. Finally, by using the unique characteristics of the “One Ship with Multiple MMSI Codes” trajectory sequence, the SSRLM model captures its feature dependencies through a fine-tuning scheme to accomplish the abnormal behavior detection task. Experimental results in the two scenarios of HN_MulMI test set demonstrate that the proposed model outperform the state-of-the-art models in recent years, the average precision, recall and F1-score rates is up to about 99.26% and 93.5%. Meanwhile, we also conducted comparative experiments on three public datasets to verify the generalization performance, achieving 90.31%, 93.2% and 95.14% results in the F1 score evaluation indicators, further confirming the good recognition performance of the SSLLM model. Finally, our model has been applied to the actual scene and achieved good results.

A Trusted Supervision Paradigm for Autonomous Driving Based on Multimodal Data Authentication

At the current stage of autonomous driving, monitoring the behavior of safety stewards (drivers) is crucial to establishing liability in the event of an accident. However, there is currently no method for the quantitative assessment of safety steward behavior that is trusted by multiple stakeholders. In recent years, deep-learning-based methods can automatically detect abnormal behaviors with surveillance video, and blockchain as a decentralized and tamper-resistant distributed ledger technology is very suitable as a tool for providing evidence when determining liability. In this paper, a trusted supervision paradigm for autonomous driving (TSPAD) based on multimodal data authentication is proposed. Specifically, this paradigm consists of a deep learning model for driving abnormal behavior detection based on key frames adaptive selection and a blockchain system for multimodal data on-chaining and certificate storage. First, the deep-learning-based detection model enables the quantification of abnormal driving behavior and the selection of key frames. Second, the key frame selection and image compression coding balance the trade-off between the amount of information and efficiency in multiparty data sharing. Third, the blockchain-based data encryption sharing strategy ensures supervision and mutual trust among the regulatory authority, the logistic platform, and the enterprise in the driving process.

Abnormal behavior detection mechanism using deep learning for zero-trust security infrastructure

Abnormal behavior detection mechanism using deep learning for zero-trust security infrastructure

NABNet: Deep Learning-Based IoT Alert System for Detection of Abnormal Neck Behavior.

The excessive use of electronic devices for prolonged periods has led to problems such as neck pain and pressure injury in sedentary people. If not detected and corrected early, these issues can cause serious risks to physical health. Detectors for generic objects cannot adequately capture such subtle neck behaviors, resulting in missed detections. In this paper, we explore a deep learning-based solution for detecting abnormal behavior of the neck and propose a model called NABNet that combines object detection based on YOLOv5s with pose estimation based on Lightweight OpenPose. NABNet extracts the detailed behavior characteristics of the neck from global to local and detects abnormal behavior by analyzing the angle of the data. We deployed NABNet on the cloud and edge devices to achieve remote monitoring and abnormal behavior alarms. Finally, we applied the resulting NABNet-based IoT system for abnormal behavior detection in order to evaluate its effectiveness. The experimental results show that our system can effectively detect abnormal neck behavior and raise alarms on the cloud platform, with the highest accuracy reaching 94.13%.

Research on 3D convolutional autoencoder enhanced metro abnormal behavior detection based on multi-level attentional memory

Research on 3D convolutional autoencoder enhanced metro abnormal behavior detection based on multi-level attentional memory

Laboratory Abnormal Behavior Detection Based on Multimodal Information Fusion

The traditional laboratory anomaly detection methods mainly focus on the hidden dangers caused by chemical leaks and other items, ignoring the impact of abnormal behaviors such as incorrect operations and improper behavior on safety in the laboratory. This paper proposes a laboratory abnormal behavior detection method based on multimodal information fusion. The method generates a dense optical flow field of RGB image sequences based on optical flow theory and global smoothing constraints, and mines motion mode information. Meanwhile, the contour modal information of behavior is captured through convolution and adjacency matrix operations. Using decision level and proximity functions to integrate student behavior motion mode information and contour mode information, and using the maximum value as the behavior detection result. The experimental results show that the method can effectively detect abnormal behavior in the laboratory environment, with small detection errors and a specificity close to 1.00, effectively ensuring the safety of the laboratory environment.