Video Anomaly Detection Using Variational Autoencoder

Abstract

Video anomaly detection in the surveillance video is one of the essential components of the intelligent video surveillance system. However, anomaly detection remains an ill-defined problem, despite the diverse applications due to its rareness and equivocal nature. A Long Short Term Memory - Variational Autoencoder (LSTM-VAE) model is proposed to detect video anomalies. The model consists of a spatial encoder comprised of convolutional layers, a temporal encoder as well as a decoder comprised of Convolutional LSTM (ConvLSTM), and a spatial decoder consisting of transposed convolution layers. The generative model is trained only on normal video clips with the objective of minimizing the reconstruction error. Then, the trained model is tested on the test video sequences comprised of both normal and abnormal incidents. The reconstruction error corresponding to the test frame sequences having video anomalies will be very high as the model is not trained to reconstruct them. Subsequently, the corresponding frames will have a low regularity score. An appropriate threshold regularity score is set to segregate the anomaly frames from the normal ones. Frames having a regularity score less than the set threshold value are considered as anomalous frames. The model is developed by using one of the publicly available bench-marked video anomaly datasets, i.e., UCSD Ped2. The performance metrics of the proposed model are promising.