Sequential inter-hop graph convolution neural network (SIhGCN) for skeleton-based human action recognition

Abstract

Skeleton-based human action recognition has attracted a lot of attention due to its capability and potential to provide more information than just using the sequence of RGB images. The use of Graph Convolutional Neural Network (GCN) becomes more popular since it can model the human skeleton very well. However, the existing GCN architectures ignore the different levels of importance on each hop during the feature aggregation and use the final hop information for further calculation, resulting in considerable information loss. Besides, they use the standard Laplacian or adjacency matrix to encode the property of a graph into a set of vectors which has a limitation in terms of graph invariants. In this work, we propose a Sequential Inter-hop Graph Convolution Neural Network (SIhGCN) which can capture salient graph information from every single hop rather than the final hop only and our work utilizes the normalized Laplacian matrix which provides better representation since it relates well to graph invariants. The proposed method is validated on two large datasets, NTU-RBG + D and Kinetics, to demonstrate the superiority of our proposed method.