Human action recognition using high-order feature of optical flows

Abstract

Optical flow is widely used in human action recognition. However, the influence of complex background on optical flow often leads to low recognition efficiency. To deal with this issue, an optical flow-based physical feature-driven action recognition framework is proposed in this paper. We first calculate the original dense optical flow field. Then, for reducing computational burden, joint action relevance that can eliminate the pseudo-optical flow in complex background is developed. After that, a more state flow field is obtained by local spatial–temporal thermal diffusion processing. On this basis, we design a feature descriptor that takes the divergence, curl and gradient features of flow field into consideration. Finally, we adopt Fisher vector to encode descriptors for classification. Experimental on HMDB51, KTH and UCF101 datasets proves that actions in complex background can be recognized accurately by the proposed framework, which outperforms the already developed methods.