AnimePose: Multi-person 3D pose estimation and animation

Abstract

3D animation of human body movement is quite challenging as it involves using a huge setup with several motion trackers all over the persons body to track the movements of every limb. This is time- consuming and may cause the person discomfort in wearing exoskeleton body suits with motion sensors. In this work, we present a trivial yet effective solution to generate simple 3D animation of human movement of multiple persons from a 2D video using deep learning. Although significant improvement has been achieved recently in 3D human pose estimation, most of the prior works work well in case of single person pose estimation and multi-person pose estimation is still a challenging problem. In this work, we firstly propose a supervised multi-person 3D pose estimation and animation framework namely AnimePose for a given input RGB video sequence. The pipeline of the proposed system consists of various modules: i) Multi-Person 2D pose estimation, ii) Depth Map estimation, iii) Lifting 2D poses to 3D poses, iv) Person trajectory prediction and human pose tracking. Our proposed system produces comparable results on previous state-of-the-art 3D multi-person pose estimation methods on publicly available dataset MuPoTS-3D dataset and it also outperforms previous competing human pose tracking methods by a significant margin of 11.7% performance gain on MOTA score on Posetrack 2018 dataset.