A human-like action learning process: Progressive pose generation for motion prediction

Abstract

Human motion prediction is crucial for the human–robot inter-action and self-driving. We human beings learn an action with two stages, i.e., the approximation stage and the adjustment stage. While the first stage learns an approximate pose in general, the second stage learns more details. Based on this observation, this paper proposes a new two-stage framework to predict the upcoming human motion poses progressively based on a historical sequence. In the approximation stage, we adopt the motion attention network to derive an approximation of the upcoming poses. In the adjustment stage, we propose a novel fine-tuning network that can extract both spatial and temporal features to enhance the prediction accuracy. To imitate the action learning procedure of our humans, the adjustment stage in our progressive prediction framework continuously adjusts the pose many times via a series of fine-tuning networks. Extensive experiments on three benchmark datasets (i.e., Human3.6M, CMU-Mocap, and 3DPW) show that our proposed method can outperform state-of-the-art methods in both short-term and long-term predictions.