Human-related anomalous event detection via memory-augmented Wasserstein generative adversarial network with gradient penalty

Abstract

Timely detection of human-related anomaly in surveillance videos is a challenging task. Generally, the irregular human motion and action patterns can be regarded as abnormal human-related events. In this paper, we utilize the skeleton trajectories to learn the regularities of human motion and action in videos for anomaly detection. The skeleton trajectories are decomposed into global and local feature sequences, which are utilized to provide human motion and action information, respectively. Then, the global and local sequences are modeled as two separate sub-processes with our proposed Memory-augmented Wasserstein Generative Adversarial Network with Gradient Penalty (MemWGAN-GP). In each sub-process, the pre-trained MemWGAN-GP is employed to predict future feature sequences from corresponding input past sequences and reconstruct the input sequences simultaneously. The predicted and reconstructed feature sequences are compared with their groundtruth to identify anomalous sequences. The MemWGAN-GP integrates the autoencoder with a WGAN model to boost the reconstruction and prediction ability of the autoencoder. Besides, a memory module is employed in MemWGAN-GP to overcome high capacity of the autoencoder for anomalies reconstruction and prediction. Experimental results on four challenging datasets demonstrate advantages of the proposed method over other state-of-the-art algorithms.