Double-layer conditional random fields model for human action recognition

Abstract

The conditional random fields (CRFs) model, as one of the most successful discriminative approaches, has received renewed attention recently for human action recognition. However, the existing CRFs model formulations have typically limited capabilities to capture higher order dependencies among the given states and deeper intermediate representations within the target states, which are potentially useful and significant to model the complex action recognition scenarios. In this paper, we present a novel double-layer CRFs (DL-CRFs) model for human action recognition in the graphical model framework. In problem formulation, an augmented top layer as the high-level and global variable is designed in the DL-CRFs model, with the global perception perspective to acquire higher-order dependencies between the target states. Meanwhile, we exploit the additional intermediate variables to explicitly perceive the intermediate representations between the target states and observation features. We then propose to decompose the DL-CRFs model in two parts, that are the top linear-chain CRFs model and the bottom one, in order to execute ease inference both during the parameter learning phase and test time. Lastly, the assumed DL-CRFs model parameters can be learned with block-coordinate primal–dual Frank–Wolfe algorithm with gap sampling scheme in a structured support vector machine framework. Experimental results and discussions on two public benchmark datasets demonstrate that the proposed approach performs better than other state-of-the-art methods in several evaluation criteria.