DAST-Net: Dense visual attention augmented spatio-temporal network for unsupervised video anomaly detection

Abstract

This paper introduces an innovative end-to-end trainable framework named Dense Attention-aware Spatio-Temporal Network (DAST-Net) for video anomaly detection. The framework adeptly leverages both spatial and temporal data in an unsupervised manner, eliminating the need for manually crafted features. To enhance spatial feature representation, DAST-Net incorporates visual attention-aware residual connections within the denser residual network (DenserResNet), deviating from traditional identity skip connections. The rationale behind this connection choice is to augment the contextual understanding of features across various scales. For capturing temporal patterns, the framework employs a Convolutional LSTM Autoencoder (ConvLSTM-AE) module, enabling effective learning and representation of temporal dependencies in video data. Consequently, discriminating features from attention modules are combined with the features extracted by the ConvLSTM-AE module, enhancing visual recognition capabilities for both spatial and temporal aspects. Our proposed architecture outperforms state-of-the-art methods on four benchmark datasets, showcasing AUC scores of 85.4% on Ped1, 97.9% on Ped2, 89.8% on Avenue, and 73.7% on the ShanghaiTech dataset. The results demonstrate the performance of our method in identifying unusual events in video data.