Superpixel appearance and motion descriptors for action recognition

Abstract

This paper introduces a novel video representation based on superpixel segmentation and appearance and motion descriptors. Superpixel represents a very useful preprocessing step for a wide range of computer vision applications, as they group pixels into perceptually meaningful atomic regions which can be used for recognizing complex motion patterns. We construct a novel video representation in terms of superpixel-based histograms of oriented gradients (HOG), histograms of optical flow (HOF) and motion boundary histograms (MBH) descriptors, and integrate such representations with a bag-of-features (BoF) model for classification. The proposed approach is evaluated in the context of action classification on a challenging benchmark dataset: UCF Sports dataset and it achieves 87.9% generalization accuracy. The experimental results demonstrate the advantage of superpixel-based descriptors compared to other approaches for human action recognition.