Tag-driven 3D human motion generation with transformer VAE

Abstract

Animating natural and controllable 3D human motion is a crucial but challenging task in the field of computer animation. The traditional methods using motion capture tend to be slow due to complicated manual procedures, while the use of neural networks for content generation, such as images, has proven to be both effective and efficient. Therefore, this paper proposes a transformer-based variational auto-encoder (Transformer-VAE) to generate realistic human motion based on the given action tag and an initial pose. The VAE is used to determine the probabilistic distribution of human motion, while the transformer architecture is used to learn the spatial-temporal relationship in motion sequences. The given initial pose, worked as a condition, greatly helps to generate appropriate motion by narrowing the search space. And an adapted scheduled sampling strategy is also employed to alleviate the train-test discrepancy in autoregressive generation. Experiments on constructed datasets show that our approach can generate diverse human motions based on specified tags and achieves superior performance compared to state-of-the-art methods in some metrics.