An abnormal event detection method based on the Riemannian manifold and LSTM network

Abstract

In this paper, we propose an abnormal event detection method based on the combination of Riemannian manifold and LSTM network. We divide the video into a series of non overlapping regions, and extract HOG and HOF as original appearance and motion features. Because the existing abnormal event detection methods pay little attention to the changes rate of appearance and motion, we propose a new feature called Riemannian manifold distance feature that can represent the topological relationship of features at different moment in the same position and capture the change rate of the current feature. Once the change rate of the current features changes, the topological relationship of features will change. We use ISOMAP algorithm to embed manifold into low dimensional Euclidean space. In the training phase, the dimension-reduced features of the same position at different historical moment are used as the input of each time step of the LSTM network, and the dimension-reduced feature of the current moment is used as expected output. And in this stage, we only use the normal video, so the LSTM will learn the changing rules of the features contained in normal events. In the testing phase, we use LSTM network to predict current dimension-reduced HOG and HOF features. After that, we can obtain the Riemannian manifold distance feature of actual dimension-reduced feature and predicted dimension-reduced feature. In order to detect abnormal events, we propose an abnormal score, which is used to measure whether the appearance/ movement and the change rate of them are close to the expectation. The abnormal score can detect the abnormal event according to dimension-reduced appearance feature, dimension-reduced motion feature, appearance Riemannian manifold distance feature and motion Riemannian manifold distance feature. Experiments on four benchmark datasets show the competitive performance of our method with the state-of-the-art methods.