Towards Efficient 3D Pose Retrieval and Reconstruction from 2D Landmarks

Abstract

In this paper, we deal with the most challenging task of recovering the 3D human pose from just a single monocular image, that may be a synthetic image or a real internet image. The retrieval and reconstruction of the articulated 3D pose, both are prerequisites for the analysis of the people in images/videos. We address both tasks together and propose an efficient framework for search & retrieval of the 3D poses from the motion capture dataset and then a novel data-driven approach to estimate the final 3D human pose from a still image. For 3D pose retrieval, we design and devise multiple feature sets based on the subsets of 2D joint locations for the global similarity search into the MoCap dataset. We resolve the 2D-3D cross model retrieval issue efficiently by projecting both, the 3D feature sets derived from the motion capture data, and the 2D feature sets from the image pose, to a normalized 2D pose space. As a result, we retrieve 3D similar poses conveniently from the MoCap dataset. For 3D pose reconstruction, we exploit retrieved nearest neighbours to learn 3D local pose model in low dimensional principle component analysis (PCA) space, which is further constrained by prior and control energies. We also capitalize retrieved nearest neighbours in order to estimate the camera parameters. At the end, we evaluate our approach on a wide variety of 2D synthetic images generated from the benchmark motion capture datasets like CMU and HDM05 by random camera parameters as well as on the real internet images.