Detection In Surveillance Videos Research Articles

Multi-Channel Generative Framework and Supervised Learning for Anomaly Detection in Surveillance Videos.

Recently, most state-of-the-art anomaly detection methods are based on apparent motion and appearance reconstruction networks and use error estimation between generated and real information as detection features. These approaches achieve promising results by only using normal samples for training steps. In this paper, our contributions are two-fold. On the one hand, we propose a flexible multi-channel framework to generate multi-type frame-level features. On the other hand, we study how it is possible to improve the detection performance by supervised learning. The multi-channel framework is based on four Conditional GANs (CGANs) taking various type of appearance and motion information as input and producing prediction information as output. These CGANs provide a better feature space to represent the distinction between normal and abnormal events. Then, the difference between those generative and ground-truth information is encoded by Peak Signal-to-Noise Ratio (PSNR). We propose to classify those features in a classical supervised scenario by building a small training set with some abnormal samples of the original test set of the dataset. The binary Support Vector Machine (SVM) is applied for frame-level anomaly detection. Finally, we use Mask R-CNN as detector to perform object-centric anomaly localization. Our solution is largely evaluated on Avenue, Ped1, Ped2, and ShanghaiTech datasets. Our experiment results demonstrate that PSNR features combined with supervised SVM are better than error maps computed by previous methods. We achieve state-of-the-art performance for frame-level AUC on Ped1 and ShanghaiTech. Especially, for the most challenging Shanghaitech dataset, a supervised training model outperforms up to 9% the state-of-the-art an unsupervised strategy.

Computer Vision based Early Electrical Fire-detection in Video Surveillance oriented for Building environment

This work presents autonomous electrical fire-detection and localization using computer vision based techniques. The proposed work uses YOLO v2 to extract the electrical fire features more effectively than other conventional and machine learning approaches. This working model is tested on commercial and residential building as well as indoor and outdoor environments. This framework has achieved high detection accuracy and low false alarm rate. Besides, the proposed frame work can be used for early real-time electrical fire detection in surveillance videos and we present experimental results for electrical fire localization in CCTV footage using the deep learning architecture proposed in this work.

Real Time Fire detection and Localization in Video sequences using Deep Learning framework for Smart Building

This work presents autonomous electrical fire-detection and localization using computer vision based techniques. The proposed work uses YOLO v2 to extract the electrical fire features more effectively than other conventional and machine learning approaches. This working model is tested on commercial and residential building as well as indoor and outdoor environments. This framework has achieved high detection accuracy and low false alarm rate. Besides, the proposed frame work can be used for early real-time electrical fire detection in surveillance videos and we present experimental results for electrical fire localization in CCTV footage using the deep learning architecture proposed in this work.

Threshold Modeling of Basic Statistical Methods for Effective Motion Detection in Video Surveillance

This paper empirical investigates two basic statistical methods namely Adaptive Median and Adaptive Mean for motion detection in video surveillance for the optimization of parameters namely threshold and the refresh rate of background frame used in these methods. Experimentation shows that the optimum choice of parameters majorly affects the quality of motion detection. The performance of methods for different parameters is measured using precision, recall and f1-score. PR curves are also drawn which are based on precision and recall values to show the effect of different parameters. Test data includes six data sets from different scenarios of ‘CDNet2012’. Experimental results verify that for every method there are fixed values of parameters with slight variations which gives better result of object motion. These parameter values can be used or adapted for future experimentation on these methods with respect to each scenario.

Detection and Localization of Abnormalities in Surveillance Video Using Timerider-Based Neural Network

Abstract Automatic anomaly detection in surveillance videos is a trending research domain, which assures the detection of the anomalies effectively, relieves the time-consumed by the manual interpretation methods without the requirement of the domain knowledge about the anomalous object. Accordingly, this research work proposes an effective anomaly detection approach, named, TimeRide Neural network (TimeRideNN), by modifying the standard RideNN using the Taylor series such that an extra group of rider, named as timerider, is included in the standard rider optimization algorithm. Initially, the face in the videos is subjected to face detection using the Viola Jones algorithm. Then, the object tracking is performed using the knocker and holoentropy-based Bhattacharya distance, which is a modification of the Bhattacharya distance using the knocker and holoentropy. After that, the features, such as object-level features and speed-level features of the objects, are extracted and the features are employed to the proposed TimeRideNN classifier, which declares the anomalous objects in the video. The experimentation of the proposed anomaly detection method is done using the UCSD dataset (Ped1), subway dataset and QMUL junction dataset, and the analysis is performed based on accuracy, sensitivity and specificity. The proposed TimeRideNN classifier obtains the accuracy, sensitivity and specificity of 0.9724, 0.9894 and 0.9691, respectively.

Motion-shape-based deep learning approach for divergence behavior detection in high-density crowd

We propose a novel method of abnormal crowd behavior detection in surveillance videos. Mainly, our work focuses on detecting crowd divergence behavior that can lead to serious disasters like a stampede. We introduce a notion of physically capturing motion in the form of images and classify crowd behavior using a convolution neural network (CNN) trained on motion-shape images (MSIs). First, the optical flow (OPF) is computed, and finite-time Lyapunov exponent (FTLE) field is obtained by integrating OPF. Lagrangian coherent structure (LCS) in the FTLE field represents crowd-dominant motion. A ridge extraction scheme is proposed for the conversion of LCS-to-grayscale MSIs. Lastly, a supervised training approach is utilized with CNN to predict normal or divergence behavior for any unknown image. We test our method on six real-world low- as well as high-density crowd datasets and compare performance with state-of-the-art methods. Experimental results show that our method is not only robust for any type of scene but also outperform existing state-of-the-art methods in terms of accuracy. We also propose a divergence localization method that not only identifies divergence starting (source) points but also comes with a new feature of generating a ‘localization mask’ around the diverging crowd showing the size of divergence. Finally, we also introduce two new datasets containing videos of crowd normal and divergence behaviors at the high density.

Adaptive Technique for Brightness Enhancement of Automated Knife Detection in Surveillance Video with Deep Learning

Detecting knives in surveillance videos are very urgent for public safety. In general, the research in identifying dangerous weapons is relatively new. Knife detection is a very challenging task because knives vary in size and shape. Besides, it easily reflects lights that reduce the visibility of knives in a video sequence. The reflection of light on the surface of the knife and the brightness on its surface makes the detection process extremely difficult, even impossible. This paper presents an adaptive technique for brightness enhancement of knife detection in surveillance systems. This technique overcomes the brightness problem that faces the steel weapons and improves the knife detection process. It suggests an automatic threshold to assess the level of frame brightness. Depending on this threshold, the proposed technique determines if the frame needs to enhance its brightness or not. Experimental results verify the efficiency of the proposed technique in detecting knives using the deep transfer learning approach. Moreover, the most four famous models of deep convolutional neural networks are tested to select the best in detecting knives. Finally, a comparison is made with the-state-of-the-art techniques, and the proposed technique proved its superiority.

Online anomaly detection in surveillance videos with asymptotic bound on false alarm rate

Anomaly detection in surveillance videos is attracting an increasing amount of attention. Despite the competitive performance of recent methods, they lack theoretical performance analysis, particularly due to the complex deep neural network architectures used in decision making. Additionally, online decision making is an important but mostly neglected factor in this domain. Much of the existing methods that claim to be online, depend on batch or offline processing in practice. Motivated by these research gaps, we propose an online anomaly detection method in surveillance videos with asymptotic bounds on the false alarm rate, which in turn provides a clear procedure for selecting a proper decision threshold that satisfies the desired false alarm rate. Our proposed algorithm consists of a multi-objective deep learning module along with a statistical anomaly detection module, and its effectiveness is demonstrated on several publicly available data sets where we outperform the state-of-the-art algorithms. All codes are available at https://github.com/kevaldoshi17/Prediction-based-Video-Anomaly-Detection-.

Anomaly3D: Video anomaly detection based on 3D-normality clusters

Abnormal behavior detection in surveillance videos is necessary for public monitoring and safety. In human-based surveillance systems, it requires continuous human attention and observation, which is a difficult task. The autonomous detection of such events is of essential significance. However, due to the scarcity of labeled data and the low occurrence probability of these events, abnormal event detection is a challenging vision problem. In this paper, we introduce a novel two-stage architecture for detecting anomalous behavior in videos. In the first stage, we propose a 3D Convolutional Autoencoder (3D-CAE) architecture to extract spatio-temporal features from normal event training videos. In 3D-CAE, the encoder and decoder architectures are based on 3D convolutions, which can learn both appearance and the motion features effectively in an unsupervised manner. In the second stage, we group the 3D spatio-temporal features into different normality clusters, and then remove the sparse clusters to represent a stronger pattern of normality. From these clusters, one-class SVM classifier is used to distinguish between normal and abnormal events based on the normality scores. Experimental results on four different benchmark datasets show significant performance improvement compared to state-of-the-art approaches while providing results in real-time.

Abnormal driving behavior detection based on kernelization-sparse representation in video surveillance

The detection of abnormal driving behaviors based on video surveillance systems is an important part of Intelligent Transportation System (ITS), which can help reduce disturbances on traffic flow and improve traffic safety. First, the study proposes a novel nonlinear sparse reconstruction method for abnormal driving behavior detection in video surveillance. A hybrid kernel function formed by convexly combining a local kernel of radial basis function (RBF) and a global kernel of homogeneous polynomial is been applied in sparse reconstruction method. Then, a novel Hybrid Kernel Orthogonal Matching Pursuit (HKOMP) algorithm is designed to solve the proposed sparse reconstruction model. Finally, the performance of the abnormal detection method is tested on two datasets i.e. stop sign dataset and car parking dataset. In addition, comparative experiments with five classical methods are carried out. The experimental results indicate that the proposed method outperforms other five comparison methods in terms of accuracy.

Cross-Epoch Learning for Weakly Supervised Anomaly Detection in Surveillance Videos

Weakly Supervised Anomaly Detection (WSAD) in surveillance videos is a complex task since usually only video-level annotations are available. Previous work treated it as a regression problem by giving different scores on normal and anomaly events. However, the widely used mini-batch training strategy may suffer from the data imbalance between these two types of events, which limits the model's performance. In this work, a cross-epoch learning (XEL) strategy associated with a hard instance bank (HIB) is proposed to introduce additional information from previous training epochs. Two new losses are proposed for XEL to achieve a higher detection rate as well as a lower false alarm rate of anomaly events. Moreover, the proposed XEL can be directly integrated into any existing WSAD framework. Experimental results of three XEL embedded models have shown promising AUC improvement (3%∼7%) on two public datasets, surpassing the state-of-the-art methods. Our code is available at: https://github.com/sdjsngs/XEL-WSAD .

Integrated Multiscale Appearance Features and Motion Information Prediction Network for Anomaly Detection.

The rise of video-prediction algorithms has largely promoted the development of anomaly detection in video surveillance for smart cities and public security. However, most current methods relied on single-scale information to extract appearance (spatial) features and lacked motion (temporal) continuity between video frames. This can cause a loss of partial spatiotemporal information that has great potential to predict future frames, affecting the accuracy of abnormality detection. Thus, we propose a novel prediction network to improve the performance of anomaly detection. Due to the objects of various scales in each video, we use different receptive fields to extract detailed appearance features by the hybrid dilated convolution (HDC) module. Meanwhile, the deeper bidirectional convolutional long short-term memory (DB-ConvLSTM) module can remember the motion information between consecutive frames. Furthermore, we use RGB difference loss to replace optical flow loss as temporal constraint, which greatly reduces the time for optical flow extraction. Compared with the state-of-the-art methods in the anomaly-detection task, experiments prove that our method can more accurately detect abnormalities in various video surveillance scenes.

An Efficient CNN and BI-LSTM Model for Abnormal Event Detection in Video Surveillance

Abnormal event detection plays a complex and important task in intelligent transportation and Computer vision. At present, the existing abnormal event detection models based on deep learning mainly focus on data represented by a vectorial form, which pay little attention to the impact of the internal structure characteristics of feature vector. In order to address the above issues, we propose an abnormal event detection hybrid modulation method via feature extraction calibrating classification in video surveillance scenes in this paper. Our main contribution is to calibrate the classification of an abnormal event by constructing feature expectation subgraphs. First, we employ convolutional neural network and long short-term memory models to extract the spatiotemporal features of video frame, and then construct the feature expectation subgraph for each key frame of every video, which could be used to capture the internal sequential and topological relational characteristics of structured feature vector. This model included a pre-trained CNN for extracting spatio-temporal features from each individual frame selected from a series of frames, which is then passed to multi-layer Bi-directional Long Short-term Memory (BD-LSTM) that possesses the capability of accurately classifying the abnormal events in complex surveillance scenes of the road. Finally, the experiments and performance of frame level and pixel level values on a common UMN dataset are estimated and evaluated.

Deep multi-level feature pyramids: Application for non-canonical firearm detection in video surveillance

The epidemic of gun violence worldwide necessitates the need for an active-based video surveillance network to combat this crime. In this context, autonomously detecting handguns is crucial in capturing firearm-related crimes. However, current object detectors using deep learning are unable to capture handguns at different scales in an unconstrained environment. Hence, this paper puts forward an enhanced deep multi-level feature pyramid network that addresses the difficulty in inferring handguns from a non-canonical perspective. We first construct a dataset containing handguns in an unconstrained environment for representation learning. The dataset is constructed from a set of 250 recorded videos and with over 2500 distinct labeled frames. Crucially, these labeled frames account for the absence of a proper video surveillance-based handgun dataset. We then train the dataset on a multi-level multi-scale object detector, i.e., M2Det. We further improve the performance of M2Det by: (1) Enhancing the base features by concatenating shallow, medium and deep features from the backbone according to its relative receptive field; (2) Implementing generalized intersection-over-union as its localization loss; and (3) Integrating Focal Loss as its classification loss to improve detection of small-scale handguns. Experiments on a challenging video surveillance test dataset demonstrate that the proposed model achieves 87.42% accuracy. In addition, we implement adaptive surveillance image partitioning to redetect handguns at specific regions. This method potentially solves the challenge of sporadically poor real-world handgun classifications. This model is capable of pioneering non-canonical handgun detection for active-based video surveillance systems. The dataset and trained models are available at:https://github.com/MarcusLimJunYi/Monash-Guns-Dataset.

RETRACTED: Efficient anomaly detection in surveillance videos based on multi layer perception recurrent neural network

Surveillance frameworks actualized in true environment are strong in nature. As the environment is uncertain and dynamic, the surveillance turns out to be increasingly perplexing when contrasted with a static and controlled environment. Effective anomaly identification in the video surveillance is a difficult issue because of spilling, video noise, anomalies, and goals. This examination work proposes a background deduction approach dependent on Maximally Stable Extremal Region (MSER) highlight extraction technique with the ongoing profound learning structure of Multi-layer perception recurrent neural network (MLP-RNN) that is fit for distinguishing multiple objects of various sizes by pixel-wise foreground investigating framework. The proposed algorithm takes as information a reference (without anomaly) and an objective edge, both transiently adjusted, and outputs a segmentation guide of same spatial goals where the featured pixels meaning the recognized anomalies, which ought to be all the components not present in the reference outline. Besides, examine the advantages of various remaking strategies to the reestablish unique picture goals and exhibit the improvement of leftover designs over the littler and more straightforward models proposed by past comparable works. The simulation results are shows serious execution in the tried dataset, just as constant handling ability as compared with existing methods.

Anomaly Event Detection in Security Surveillance Using Two-Stream Based Model

Anomaly event detection has been extensively researched in computer vision in recent years. Most conventional anomaly event detection methods can only leverage the single-modal cues and not deal with the complementary information underlying other modalities in videos. To address this issue, in this work, we propose a novel two-stream convolutional networks model for anomaly detection in surveillance videos. Specifically, the proposed model consists of RGB and Flow two-stream networks, in which the final anomaly event detection score is the fusion of those of two networks. Furthermore, we consider two fusion situations, including the fusion of two streams with the same or different number of layers respectively. The design insight is to leverage the information underlying each stream and the complementary cues of RGB and Flow two-stream sufficiently. Two datasets (UCF-Crime and ShanghaiTech) are used to validate the effectiveness of proposed solution.

Abnormal event detection in surveillance videos based on low-rank and compact coefficient dictionary learning

In this paper, a novel approach to abnormal event detection in crowded scenes is presented based on a new low-rank and compact coefficient dictionary learning (LRCCDL) algorithm. First, based on the background subtraction and binarization of surveillance videos, we construct a feature space by extracting the histogram of maximal optical flow projection (HMOFP) feature of the foreground from a normal training frame set. Second, in the training stage, a new joint optimization of the nuclear-norm and l2, 1-norm is applied to obtain a compact coefficient low-rank dictionary. Third, in the detection stage, l2, 1-norm optimization is utilized to obtain the reconstruction coefficient vectors of the testing samples. Note that the l2, 1-norm forces the reconstruction vectors of all the testing samples to compactly surround the same center in the training stage, such that the reconstruction errors of abnormal testing samples are different from those of normal ones. Finally, a reconstruction cost (RC) is introduced to detect abnormal frames. Experimental results on both global and local abnormal event detection show the effectiveness of our algorithm. Based on comparisons with state-of-the-art methods employing various criteria, the proposed algorithm achieves comparable detection results.

Autocorrelation of gradients based violence detection in surveillance videos

Abstract Automated monitoring of videos is becoming mandatory due to its widespread applications over public and private domains. Especially, research over detecting anomalous human behavior in crowded scenes has created much attention among computer vision researchers. Understanding patterns in crowded scenes is always challenging due to the rapid movement of the crowd, occlusions and cluttered backgrounds. In this work, we explore spatio-temporal autocorrelation of gradient-based features to efficiently recognize violent activities in crowded scenes. A discriminative classifier is then used to recognize violent actions in videos. Experimental results have shown improved performance of the proposed approach when compared to existing state-of-art-approaches.

Effective object proposals: size prediction for pedestrian detection in surveillance videos

Though detectors based on deep networks have witnessed a significant boost in pedestrian detection performance, small-scale pedestrian detection remains to be a challenging task. To this end, the authors propose a scale prediction method to eliminate the dependence of most existing detectors on pre-defined anchor boxes. Due to projective transformation, a pedestrian standing afar will appear smaller than one standing closeby. For a stationary surveillance camera, different blocks on the image correspond to the views of various depths leadings to diverse sizes of object proposals. A neural network structure is developed to empirically estimate the size of a pedestrian candidate in the pixel coordinates given its central location. Comprehensive sets of experiments on two real-world datasets demonstrate that the proposed method achieves superior performance with state-of-the-art methods. Besides, significant performance advantages on small pedestrian detection are observed.

A novel technique for automated concealed face detection in surveillance videos.

Face detection perceives great importance in surveillance paradigm and security paradigm areas. Face recognition is the technique to identify a person identity after face detection. Extensive research has been done on these topics. Another important research problem is to detect concealed faces, especially in high-security places like airports or crowded places like concerts and shopping centres, for they may prevail security threat. Also, in order to help effectively in preventing the spread of Coronavirus, people should wear masks during the pandemic especially in the entrance to hospitals and medical facilities. Surveillance systems in medical facilities should issue warnings against unmasked people. This paper presents a novel technique for concealed face detection based on complexion detection to challenge a concealed face assumption. The proposed algorithm first determine of the existence of a human being in the surveillance scene. Head and shoulder contour will be detected. The face will be clustered to cluster patches. Then determination of presence or absent of human skin will be determined. We proposed a hybrid approach that combines normalized RGB (rgb) and the YCbCr space color. This technique is tested on two datasets; the first one contains 650 images of skin patches. The second dataset contains 800 face images. The algorithm achieves an average detection rate of 97.51% for concealed faces. Also, it achieved a run time comparable with existing state-of-the-art concealed face detection systems that run in real time.