Hypergraph Neural Network for Skeleton-Based Action Recognition.

Abstract

Recently, skeleton-based human action recognition has attracted a lot of research attention in the field of computer vision. Graph convolutional networks (GCNs), which model the human body skeletons as spatial-temporal graphs, have shown excellent results. However, the existing methods only focus on the local physical connection between the joints, and ignore the non-physical dependencies among joints. To address this issue, we propose a hypergraph neural network (Hyper-GNN) to capture both spatial-temporal information and high-order dependencies for skeleton-based action recognition. In particular, to overcome the influence of noise caused by unrelated joints, we design the Hyper-GNN to extract the local and global structure information via the hyperedge (i.e., non-physical connection) constructions. In addition, the hypergraph attention mechanism and improved residual module are induced to further obtain the discriminative feature representations. Finally, a three-stream Hyper-GNN fusion architecture is adopted in the whole framework for action recognition. The experimental results performed on two benchmark datasets demonstrate that our proposed method can achieve the best performance when compared with the state-of-the-art skeleton-based methods.