Adaptive Slice Representation for Human Action Classification

Abstract

Common action recognition methods describe an action sequence along with its time axis, i.e., first extracting features from the $x y$ plane, and then modeling the dynamic changes along with the time axis. Other than the ordinary $x y$ plane-based representation, other views, e.g., $x t$ slice-based representation, may be more efficient to distinguish different actions. In this paper, we investigate different slicing views of the spatiotemporal volume to organize action sequences and propose an efficient slice representation for human action recognition. First, a minimum average entropy principle is proposed to select the optimal slicing angle for each action sequence adaptively. This allows the foreground pixels to be distributed in the fewest slices so as to reduce more uncertainty caused by the information dispersed in different slices. Then, the obtained slice sequence is transformed into a pair of 1-D signals to describe the distribution of foreground pixels along the time axis. Finally, the mel frequency cepstrum coefficient features are calculated to describe the spectrum characteristics of the 1-D signals over time. Thus, a 3-D spatiotemporal action volume is efficiently transformed into a low-dimensional spectrum features. Extensive experiments on the 2-D human action data sets (the UIUC and the WEIZMANN) as well as the Microsoft Research (MSR) Action3-D depth data set demonstrate the effectiveness of the slice-based representation, where the recognition performance can reach to the state-of-the-art level with high efficiency.