Skeleton-Based ST-GCN for Human Action Recognition With Extended Skeleton Graph and Partitioning Strategy

Abstract

Skeleton-based Graph Convolutional Networks (GCN) for human action and interaction recognition have received considerable attention of researchers due to its compact and view-invariant nature of skeleton data. However, the static skeleton graph topology in conventional GCNs does not reflect the implicit relationships of non-adjacent joints, which contain vital latent information for a skeleton pose in an action sequence. Moreover, traditional tri-categorical node partitioning strategy discards much of the motion dependencies along temporal dimension for non-physically connected edges. We propose an extended skeleton graph topology along with extended partitioning strategy to extract much of the non-adjacent joint relational information in the model for robust discriminative features. Extended skeleton graph represents joints as vertices and weighted edges represent intrinsic and extrinsic relationships between physically connected and non-physically connected joints respectively. Furthermore, extended partitioning strategy divides the input graph for GCN as five-categorical fixed-length tensor to encompass maximal motion dependencies. Finally, the extended skeleton graph and partitioning strategy are realized by adopting Spatio-Temporal Graph Convolutional Network (ST-GCN). The experiments carried out over three large scale datasets NTU-RGB+D, NTU-RGB+D 120 and Kinetics-Skeleton show improved performance over conventional state-of-the-art ST-GCNs.