Knowledge embedded GCN for skeleton-based two-person interaction recognition

Abstract

Two-person interaction recognition with skeleton data has attracted much attention in computer vision. Recently, graph convolutional network (GCN) based methods, which model the skeleton data in the form of graph, have achieved remarkable performance. However, the topology of graph in existing methods, denoted as naturally connected graph, is predefined based on the natural connection of each person. It ignores the correlations between two persons and cannot be suitable for different actions. In this paper, we design two graphs by exploiting the knowledge for two-person interaction recognition. A knowledge-given graph is constructed to build the direct connection between two persons. Meanwhile, a knowledge-learned graph is proposed to build the adaptive correlations, which is unique for each input sample. Moreover, we further propose the knowledge embedded graph convolution network (K-GCN) to exploit the complementarity among knowledge-given, knowledge-learned and naturally connected graphs for two-person interaction recognition. In addition, multi-level scheme is proposed to model the joint-level and part-level information simultaneously, which further enhances the performance. To verify the effectiveness of the proposed method, consecutive ablation studies are performed on two prevalence datasets, SBU and NTU Interaction. Experimental results show that our method achieves the state-of-the-art performance for two-person interaction recognition on both of them.