Event Detection In Video Surveillance Research Articles

Self-Supervised Masked Convolutional Transformer Block for Anomaly Detection.

Anomaly detection has recently gained increasing attention in the field of computer vision, likely due to its broad set of applications ranging from product fault detection on industrial production lines and impending event detection in video surveillance to finding lesions in medical scans. Regardless of the domain, anomaly detection is typically framed as a one-class classification task, where the learning is conducted on normal examples only. An entire family of successful anomaly detection methods is based on learning to reconstruct masked normal inputs (e.g. patches, future frames, etc.) and exerting the magnitude of the reconstruction error as an indicator for the abnormality level. Unlike other reconstruction-based methods, we present a novel self-supervised masked convolutional transformer block (SSMCTB) that comprises the reconstruction-based functionality at a core architectural level. The proposed self-supervised block is extremely flexible, enabling information masking at any layer of a neural network and being compatible with a wide range of neural architectures. In this work, we extend our previous self-supervised predictive convolutional attentive block (SSPCAB) with a 3D masked convolutional layer, a transformer for channel-wise attention, as well as a novel self-supervised objective based on Huber loss. Furthermore, we show that our block is applicable to a wider variety of tasks, adding anomaly detection in medical images and thermal videos to the previously considered tasks based on RGB images and surveillance videos. We exhibit the generality and flexibility of SSMCTB by integrating it into multiple state-of-the-art neural models for anomaly detection, bringing forth empirical results that confirm considerable performance improvements on five benchmarks: MVTec AD, BRATS, Avenue, ShanghaiTech, and Thermal Rare Event.

Review on Deep Learning Approaches for Anomaly Event Detection in Video Surveillance

In the last few years, due to the continuous advancement of technology, human behavior detection and recognition have become important scientific research in the field of computer vision (CV). However, one of the most challenging problems in CV is anomaly detection (AD) because of the complex environment and the difficulty in extracting a particular feature that correlates with a particular event. As the number of cameras monitoring a given area increases, it will become vital to have systems capable of learning from the vast amounts of available data to identify any potential suspicious behavior. Then, the introduction of deep learning (DL) has brought new development directions for AD. In particular, DL models such as convolution neural networks (CNNs) and recurrent neural networks (RNNs) have achieved excellent performance dealing with AD tasks, as well as other challenging domains like image classification, object detection, and speech processing. In this review, we aim to present a comprehensive overview of those research methods using DL to address the AD problem. Firstly, different classifications of anomalies are introduced, and then the DL methods and architectures used for video AD are discussed and analyzed, respectively. The revised contributions have been categorized by the network type, architecture model, datasets, and performance metrics that are used to evaluate these methodologies. Moreover, several applications of video AD have been discussed. Finally, we outlined the challenges and future directions for further research in the field.

An Efficient CNN and BI-LSTM Model for Abnormal Event Detection in Video Surveillance

Abnormal event detection plays a complex and important task in intelligent transportation and Computer vision. At present, the existing abnormal event detection models based on deep learning mainly focus on data represented by a vectorial form, which pay little attention to the impact of the internal structure characteristics of feature vector. In order to address the above issues, we propose an abnormal event detection hybrid modulation method via feature extraction calibrating classification in video surveillance scenes in this paper. Our main contribution is to calibrate the classification of an abnormal event by constructing feature expectation subgraphs. First, we employ convolutional neural network and long short-term memory models to extract the spatiotemporal features of video frame, and then construct the feature expectation subgraph for each key frame of every video, which could be used to capture the internal sequential and topological relational characteristics of structured feature vector. This model included a pre-trained CNN for extracting spatio-temporal features from each individual frame selected from a series of frames, which is then passed to multi-layer Bi-directional Long Short-term Memory (BD-LSTM) that possesses the capability of accurately classifying the abnormal events in complex surveillance scenes of the road. Finally, the experiments and performance of frame level and pixel level values on a common UMN dataset are estimated and evaluated.

An On-Line and Adaptive Method for Detecting Abnormal Events in Videos Using Spatio-Temporal ConvNet

We address in this paper the problem of abnormal event detection in video-surveillance. In this context, we use only normal events as training samples. We propose to use a modified version of pretrained 3D residual convolutional network to extract spatio-temporal features, and we develop a robust classifier based on the selection of vectors of interest. It is able to learn the normal behavior model and detect potentially dangerous abnormal events. This unsupervised method prevents the marginalization of normal events that occur rarely during the training phase since it minimizes redundancy information, and adapt to the appearance of new normal events that occur during the testing phase. Experimental results on challenging datasets show the superiority of the proposed method compared to the state of the art in both frame-level and pixel-level in anomaly detection task.

Research on Abnormal Event Detection in Video Surveillance Based on Displacement of Feature Point and GLCM Texture Features

The current anomaly detection algorithm uses more than a single feature to detect that causes trouble about the scene feature description and has the average accuracy of about 92%. This paper presented a new video anomaly detection algorithm using displacement feature and texture feature. The algorithm used pyramid LK optical flow method combined with Harris corner to extract the average displacement feature of moving target. GLCM algorithm and Fast Fourier Transform were introduced to get the texture feature of video images. Finally through SVM algorithms for two kinds of features classification training, the highest accuracy could reach 97.65%. Experiments show that the new algorithm has a higher accuracy and more practical.

Learning Multimodal Deep Representations for Crowd Anomaly Event Detection

Anomaly event detection in crowd scenes is extremely important; however, the majority of existing studies merely use hand-crafted features to detect anomalies. In this study, a novel unsupervised deep learning framework is proposed to detect anomaly events in crowded scenes. Specifically, low-level visual features, energy features, and motion map features are simultaneously extracted based on spatiotemporal energy measurements. Three convolutional restricted Boltzmann machines are trained to model the mid-level feature representation of normal patterns. Then a multimodal fusion scheme is utilized to learn the deep representation of crowd patterns. Based on the learned deep representation, a one-class support vector machine model is used to detect anomaly events. The proposed method is evaluated using two available public datasets and compared with state-of-the-art methods. The experimental results show its competitive performance for anomaly event detection in video surveillance.

DeepDetect: A Cascaded Region-Based Densely Connected Network for Seismic Event Detection

Automatic event detection from time series signals has wide applications, such as abnormal event detection in video surveillance and event detection in geophysical data. Traditional detection methods detect events primarily by the use of similarity and correlation in data. Those methods can be inefficient and yield low accuracy. In recent years, because of the significantly increased computational power, machine learning techniques have revolutionized many science and engineering domains. In this study, we apply a deep-learning-based method to the detection of events from time series seismic signals. However, a direct adaptation of the similar ideas from 2D object detection to our problem faces two challenges. The first challenge is that the duration of earthquake event varies significantly; The other is that the proposals generated are temporally correlated. To address these challenges, we propose a novel cascaded region-based convolutional neural network to capture earthquake events in different sizes, while incorporating contextual information to enrich features for each individual proposal. To achieve a better generalization performance, we use densely connected blocks as the backbone of our network. Because of the fact that some positive events are not correctly annotated, we further formulate the detection problem as a learning-from-noise problem. To verify the performance of our detection methods, we employ our methods to seismic data generated from a bi-axial "earthquake machine" located at Rock Mechanics Laboratory, and we acquire labels with the help of experts. Through our numerical tests, we show that our novel detection techniques yield high accuracy. Therefore, our novel deep-learning-based detection methods can potentially be powerful tools for locating events from time series data in various applications.

Clustering Based on Two Layers for Abnormal Event Detection in Video Surveillance

Abnormal event detection has attracted great research attention in video surveillance. In this paper, the authors presented a robust method of trajectories clustering for abnormal event detection. This method is based on two layers and benefits from two well-known clustering algorithms: the agglomerative hierarchical clustering and the k-means clustering. Facing to the challenges related to the trajectories, e.g., different sizes, the authors introduce a preprocessing step to unify their sizes and reduce their dimensionality. The experimental results show the performance and accuracy of their proposed method.

Three-Pronged Compensation and Hysteresis Thresholding for Moving Object Detection in Real-Time Video Surveillance

Moving object extraction is the core of event detection in video surveillance. Although many related methods have been proposed to extract moving objects, even advanced applications still encounter cavity problems, which are false detection and deficiencies resulting from cavities inside the body or fragmented foreground objects. In this paper, an entirely new structure for extracting moving objects is proposed. This scheme is based on the concepts of hysteresis thresholding and motion compensation, which constitute spatial and temporal compensations, respectively. Experimental results show that the proposed scheme can outperform modern moving object detection methods in terms of precision, recall, F-measure, and other measurements.

Toward Abnormal Trajectory and Event Detection in Video Surveillance

In this paper, we present a unified approach for abnormal behavior detection and group behavior analysis in video scenes. Existing approaches for abnormal behavior detection do either use trajectory-based or pixel-based methods. Unlike these approaches, we propose an integrated pipeline that incorporates the output of object trajectory analysis and pixel-based analysis for abnormal behavior inference. This enables to detect abnormal behaviors related to speed and direction of object trajectories, as well as complex behaviors related to finer motion of each object. By applying our approach on three different data sets, we show that our approach is able to detect several types of abnormal group behaviors with less number of false alarms compared with existing approaches.

Learning deep event models for crowd anomaly detection

Abnormal event detection in video surveillance is extremely important, especially for crowded scenes. In recent years, many algorithms have been proposed based on hand-crafted features. However, it still remains challenging to decide which kind of feature is suitable for a specific situation. In addition, it is hard and time-consuming to design an effective descriptor. In this paper, video events are automatically represented and modeled in unsupervised fashions. Specifically, appearance and motion features are simultaneously extracted using a PCANet from 3D gradients. In order to model event patterns, a deep Gaussian mixture model (GMM) is constructed with observed normal events. The deep GMM is a scalable deep generative model which stacks multiple GMM-layers on top of each other. As a result, the proposed method acquires competitive performance with relatively few parameters. In the testing phase, the likelihood is calculated to judge whether a video event is abnormal or not. In this paper, the proposed method is verified on two publicly available datasets and compared with state-of-the-art algorithms. Experimental results show that the deep model is effective for abnormal event detection in video surveillance.

Use of feedback strategies in the detection of events for video surveillance

The authors present a feedback-based approach for event detection in video surveillance that improves the detection accuracy and dynamically adapts the computational effort depending on the complexity of the analysed data. A core feedback structure is proposed based on defining different levels of detail for the analysis performed and estimating the complexity of the data being analysed. Then, three feedback-based analysis strategies are defined (based on this core structure) and introduced in the processing stages of a typical video surveillance system. A rule-based system is designed to manage the interaction between these feedback-strategies. Experimental results show that the proposed approach slightly increases the detection reliability, whereas highly reduces the computational effort as compared to the initially developed surveillance system (without feedback strategies) across a variety of multiple video surveillance scenarios operating at real time.