Optical flow-motion history image (OF-MHI) for action recognition

Abstract

The motion history image (MHI) is a global spatiotemporal representation for video sequences. It is computationally very simple and efficient. It has been widely used for many real-time action recognition tasks. However, the conventional MHI assigns a fixed motion strength to each detected foreground point and then updates it with a small constant for the background point. Local body parts with different movement speeds and durations will then have the same intensity in the MHI. Similar actions may generate indistinguishable MHI patterns. In this paper, we propose a new motion history representation that incorporates both optical flow and a revised MHI. The motion strength of each pixel point is adaptively accumulated by the optical flow length at that location. It is then exponentially updated over time. It can better describe local movements of body parts in the global temporal template. The motion duration is implicitly given by the update rate for better description of various actions in the scene. For action classification, a set of training action samples are first collected and form the basis templates. An action sequence is then constructed as the linear combination of the basis templates. The coefficients of the combination give the feature vector. The Euclidean distance is finally used to evaluate the similarity between the feature vectors. Experimental results on the widely used KTH and Weizmann datasets have shown that the proposed scheme yields 100 % recognition rates on both test datasets with a fast processing rate of 47 fps on $$200\times 150$$ images.