Recognizing micro actions in videos by learning multi-layer local features

Abstract

Recognizing micro actions such as slight head shaking or hand clapping from videos can be challenging since they only involve small movements of local body parts. In this paper, we propose to fuse features from both higher-level and lower-level layers of convolutional neural networks for improving the accuracy of micro-action recognition. Deep features in higher-level layers have been shown to be effective in recognizing general actions, such as biking and jumping, that involve relatively large movements. Different from features in higher-level layers, features in lower-level layers are usually of higher resolution and can help capture small motions in micro actions. In this paper, we employ class-discriminative information as a guidance in lower-level layers to learn local features that are highly associated with micro-action regions. In the experiments, we evaluate the proposed method on two micro-action video datasets and achieve new state-of-the-art performance. We also test the proposed method on two general-action video datasets with promising performance.