Low-Rank Tensor Subspace Learning for RGB-D Action Recognition.

Abstract

RGB-D action data inherently equip with extra depth information to improve performance of action recognition compared with RGB data, and many works represent the RGB-D data as a third-order tensor containing a spatiotemporal structure and find a subspace with lower dimension. However, there are two main challenges of these methods. First, the dimension of subspace is usually fixed manually, which may not describe the samples well in the subspace. Second, preserving local information by finding the intra-class and inter-class neighbors from a manifold is highly time-consuming. In this paper, we learn a tensor subspace, whose dimension is learned automatically by low-rank learning, for RGB-D action recognition. Particularly, the tensor samples are factorized to obtain three projection matrices (PMs) by Tucker Decomposition, where all the PMs are performed by nuclear norm in a close-form to obtain the tensor ranks, which are used as tensor subspace dimension. In addition, we extract the discriminant and local information from a manifold using a graph constraint. This graph preserves the local knowledge inherently, which is faster than the previous way of calculating both the intra-class and inter-class neighbors of each sample. We evaluate the proposed method on four widely used RGB-D action datasets including MSRDailyActivity3D, MSRActionPairs, MSRActionPairs skeleton, and UTKinect-Action3D datasets, and the experimental results show higher accuracy and efficiency of the proposed method.