Human action recognition with graph-based multiple-instance learning

Abstract

A new approach to human action recognition from realistic videos is presented in this paper. First, an affine motion model is utilized to compensate background motion for the purpose of extracting dense foreground trajectories. Then, a trajectory spectral embedding is introduced to split up foreground action into multiple spatio-temporal action parts for constructing a mid-level representation. To deal with over-segmentation, a novel density discontinuity detector is proposed for the sake of generating semantically salient action parts. Finally, to handle the ambiguity in the training set, action classification is formulated within the multiple-instance learning framework, which a spatio-temporal graph model is incorporated into. Extensive experiments show that the proposed approach achieves competitive results to state of the art on UCF Sports, Kisses/Slaps, YouTube, and Hollywood datasets.