Robust 3D Human Pose Estimation Guided by Filtered Subsets of Body Keypoints

Abstract

We propose a novel hybrid human 3D body pose estimation method that uses RGBD input. The method relies on a deep neural network to get an initial 2D body pose. Using depth information from the sensor, a set of 2D landmarks on the body are transformed in 3D. Then, a multiple hypothesis tracker uses the obtained 2D and 3D body landmarks to estimate the 3D body pose. In order to safeguard from observation errors, each human pose hypothesis considered by the tracker is constructed using a gradient descent optimization scheme that is applied to a subset of the body landmarks. Landmark selection is driven by a set of geometric constraints and temporal continuity criteria. The resulting 3D poses are evaluated by an objective function that calculates densely the discrepancy between the 3D structure of the rendered 3D human body model and the actual depth observed by the sensor. The quantitative experiments show the advantages of the proposed method over a baseline that directly uses all landmark observations for the optimization, as well as over other recent 3D human pose estimation approaches.