Multi-task learning for video anomaly detection

Abstract

We propose a multi-task learning framework for video anomaly detection based on a novel pipeline. Our model contains two crossing streams, one stream employs the backbone of Attention-R2U-net for future frame prediction, while the other is designed based on an encoder–decoder network to reconstruct the input optical flow maps. In addition, the latent layers of the two streams are merged together and assigned with a Deep SVDD-based loss at each location individually. Through the combination of these three tasks, the two-stream-crossing pipeline can be trained end-to-end to provide a comprehensive evaluation for the anomaly targets. Experimental results on several popular benchmark datasets show that our model outperforms the state-of-the-art competing models, which can be applied to different types of anomalous targets and meanwhile achieves remarkable precision.