Kernel Attention Based Multi-scale Adaptive Graph Convolutional Neural Network for Skeleton-Based

Abstract

Graph convolution network is widely used in skeleton-based action recognition tasks. To improve the adaptability of the graph, the model should learn the structure of the graph automatically. The existing adaptive strategies nested by methods of action recognition merely depend on the spatial features of skeleton without considering the temporal factors. Such a model is not sufficient to generate a graph structure better suited to model the spatio-temporal correlation of skeletons. Furthermore, the features are easily overlooked during training and there is no suitable feature enhancer for skeleton-based data. To improve these defects, the paper (1) proposed a novel adaptive graph module by aggregating spatio-temporal features of skeleton sequences and (2) designed a new action recognition model with a powerful feature enhancer (kernel attention) called KA-AGCN. Finally, on three large-scale action recognition datasets NTU-RGBD 60, NTU-RGBD 120 and Skeleton-Kinetics, our results outperform most of the advanced methods.