Human-related anomalous event detection via spatial-temporal graph convolutional autoencoder with embedded long short-term memory network

Abstract

Automatic detection of human-related anomalous events in surveillance videos is challenging, owing to unclear definition of anomalies and insufficiency of training data. Generally, the irregular human motion patterns can be regarded as human-related abnormal events. Therefore, we propose a novel method to operate directly on sequences of human skeleton graphs for discovering the normal patterns of human motion. The sequence of skeleton graphs is decomposed into two sub-components: global movement and local posture sequences. The global component is utilized to compute local component. The local component sequences are then input to our network for capturing normal spatial-temporal motion patterns of human skeleton. Our network is established on a Spatial-temporal Graph Convolutional Autoencoder (ST-GCAE) and embedded with Long Short-Term Memory (LSTM) network in hidden layers for exploring the temporal cues, which is thus called Spatial-temporal Graph Convolutional Autoencoder with Embedded Long Short-Term Memory Network (STGCAE-LSTM). Different from traditional autoencoder, STGCAE-LSTM owns a single-encoder-dual-decoder architecture, which is capable of reconstructing the input and predicting the unseen future simultaneously. Then, samples that deviate from normal patterns are detected as anomalies with fusion of reconstruction and prediction errors. Experimental results on four challenging datasets demonstrate advantages of our method over other state-of-the-art algorithms.