Multi-Branch Spatial-Temporal Attention Graph Convolution Network for Skeleton-based Action Recognition

Abstract

Skeleton-based human action recognition has attracted considerable research interest due to its robustness to dynamic environments and complex backgrounds. Models based on graph neural networks have achieved great success in this field. However, most of these graph neural network based methods adopt a single-branch or two-stream structure with limited input features. Due to the separation of space and time, many methods cannot pay attention to the critical information of human skeleton sequence, and multi-scale spatial-temporal dependent attention modeling becomes the key to modeling. This paper proposes a novel multi-branch Spatio-temporal attention graph convolutional neural network (MB-STAGCN) to recognize human actions from skeleton data. The proposed model adopts a multi-branch structure, fuses three-stream features, and then inputs them into the backbone network to extract features from multiple perspectives to a greater extent. Besides, we design a new temporal convolutional block for multi-scale extraction of information in the backbone network. We also propose an attention block named Spatio-temporal Concat Attention (STCatAtt) to capture critical information. Experiments on benchmark datasets show remarkable performance for human action recognition, demonstrating the effectiveness of our method.