Multi-granularity spatial temporal graph convolution network with consecutive attention for human motion prediction

Abstract

Human motion prediction is attracting increasing attention for its numerous potential applications in fields including autonomous driving, video surveillance and virtual reality. However, accurate motion prediction is challenging due to the complex spatial dependencies, dynamic temporal correlations and high dimension of human pose sequences. Existing graph-based methods rarely consider positional and channel information of the feature map, resulting in lower prediction accuracy. Therefore, we propose a novel multi-granularity spatial temporal graph convolution network with consecutive attention (MSTCA) for human motion prediction. Firstly, a multi-granularity spatial convolution network is introduced to capture spatial joint features through multiple kernel sizes. Then, consecutive attention module is proposed to capture both positional and channel information of the feature map. Next, MSTCA uses multi-granularity temporal convolutional network to extract temporal correlations with multiple receptive fields and predict future poses. Finally, a decoder composed of a 2D convolution layer and several PRelu layers integrates the output of the whole model. Experimental results on the GTA-IM and PROX datasets demonstrate that our method significantly improves the accuracy of human motion prediction in comparison to the existing approaches.