Self Attention based Bi-directional Long Short-Term Memory Auto Encoder for Video Anomaly Detection

Abstract

In view of the existing algorithm ignoring global features and focusing only on local features, a Self Attention based Bi-directional long short-term memory Auto Encoder (SABiAE) is proposed. In this method, in order to solve problem of insufficient attention to some region in existing methods, the self attention based encoder is used to extract the global appearance features; in order to reduce the loss of relevant target features, the self attention based bi-directional long short-term memory network is extract the global temporal features. By using SABiAE to capture the global information between video frames, the decoder is used to reconstruct the global appearance and temporal features into a reconstructed frame. Thus, the reconstruction error between the reconstructed frame and the actual frame is calculated to judge the anomaly. Experimental results on two public datasets, UCSD ped2 and CUHK Avenue, show that the proposed method can accurately detect abnormal events in video, with frame level AUC of 95.6% and 84.7% respectively.