Action recognition using edge trajectories and motion acceleration descriptor

Abstract

This paper presents a method for action recognition based on edge trajectories. First, to exploit long-term motion information for action representation more effectively, we propose to track edge points across video frames to extract spatiotemporal edge trajectories and use the ones derived from the edge points located on the boundaries of action-related area to describe actions. Second, besides the existing shape, histogram of oriented gradients, histogram of optical flow and motion boundary histogram, a new trajectory descriptor named histogram of motion acceleration is introduced, which is computed using the temporal derivative of the optical flow in the spatiotemporal neighborhood centered along a trajectory and describes the temporal relative motion of actions. Finally, using Fisher vector to encode trajectory descriptors and MKL-based multi-class SVM to predict action labels, we evaluate the proposed approach on seven benchmark datasets, namely KTH, ADL, UT-Interaction, UCF sports, YouTube, HMDB51 and UCF101. The experimental results demonstrate the effectiveness of our method.