Realistic action recognition with salient foreground trajectories

Abstract

Human action recognition has great potential in many applications relevant to artificial intelligence, which can accelerate some research on expert and intelligent systems, such as feature selection. To improve the performance on human action recognition in realistic scenarios, a novel Salient Foreground Trajectory extraction method based on saliency detection and low-rank matrix recovery is proposed to learn the discriminative features from complicated video context. Specifically, a new trajectory saliency combining appearance saliency and motion saliency is proposed to divide the dense trajectories into salient trajectories and non-salient ones. The salient trajectories are approximately corresponding to the interested foreground region, while the non-salient subset is mainly composed of the dominating background trajectories. Furthermore, according to the low rank property of background motion, if the video has background motion, the background trajectory subspace is further constructed on the non-salient trajectory subset via low-rank matrix recovery method. Then the possible background trajectories in the salient subset could be subtracted. Finally, the resulting salient foreground trajectory features are encoded by the approach of Bag of Features or Fisher Vector for action classification. Experiments on KTH, UCF Sports and Olympic Sports have shown that the proposed Salient Foreground Trajectory method is effective and achieves comparable results to the state of the art.