Anomaly detection with dual-stream memory network

Abstract

Anomaly detection is an essential but challenging task. Existing DNN-based approaches tend to ignore the impact of network history state on extracting spatio-temporal correlations between video events. To address this problem, a Dual-Stream Memory Network (DSM-Net) has been proposed. It leverages historical information from the network to create a dual-stream memory module serving as complementary knowledge for the anomaly detection network. The memory module performs writing and reading in the form of a queue of data features. The writing records the historic information of video events through a moving average encoder, and the reading uses optical flow to uncover behavioral patterns in RGB images. Using a memory sharing strategy, the semantic information of the appearance branch and the motion branch can be integrated to reinforce the network. Results demonstrate that the proposed method on various standard datasets performs favorably when compared to existing methods.