PSPDNet: Part-aware shape and pose disentanglement neural network for 3D human animating meshes

Abstract

Disentangled representations of shape and pose are essential for animating human body meshes in computer animation, computer games, and virtual reality applications. While recent deep neural networks have achieved impressive effectiveness, their performance in terms of interpretability, reconstruction precision, and fine-grained control is not satisfactory. To address these issues, we propose the Part-aware Shape and Pose Disentanglement neural network (PSPDNet), a framework for disentangling the shape and pose of 3D human meshes with the same connectivity. PSPDNet utilizes part mesh autoencoders to learn representations of different human body parts, enhancing the interpretability of the latent codes by corresponding them with local motions. While mesh autoencoders alone can decouple shape and pose information from animated meshes, they fail to control local motions. In addition, PSPDNet employs an additional rotation-translation module to remove global rigid motion, i.e., rotation and translation, from the sequence. Finally, we propose a novel loss function which includes disentanglement loss and alignment loss to train PSPDNet in an unsupervised manner. Our experiments show that PSPDNet greatly improves disentangled representation with strong interpretability, insensitivity to global rigid transformation, and locality of editing and controlling.