Global motion estimation with iterative optimization-based independent univariate model for action recognition

Abstract

Motion information used in the existed video action recognition schemes is mixing of global motion(GM) and local motion(LM). In fact, GM & LM have their respective semantic concepts. Thus, it is promising to decouple GM and LM from the mixed motions. Numerous efforts have been made on the design of global motion models for video encoding, video dejittering, video denoising, and so on. Nevertheless, some of the models are too basic to cover the camera motions in action recognition while others are over-complicated. In this paper, we focus on the characteristic of the action recognition and propose a novel independent univariate GM model. It ignores camera rotation, which appears rarely in action recognition videos, and represents the GM in x and y direction respectively. Furthermore, GM is position invariant because it is from the universal camera motion. Pixels with global motions are subjected to the same parametric model and pixels with mixed motion can be seen as outliers. Motivated by this, we develop an iterative optimization scheme for GM estimation which removes the outlier points step by step and estimates global motions in a coarse-to-fine manner. Finally, the LM is estimated through a Spatio-temporal threshold-based method. Experimental results demonstrate that the proposed GM model makes a better trade-off between the model complexity and the robustness. And the iterative optimization scheme is more effective than the existed algorithms. The compared experiments using four popular action recognition models on UCF-101 (for action recognition) and NCAA (for group activity recognition) demonstrate that local motions are more effective than the mixed motions.