Motion Tracking, Retrieval and 3D Reconstruction from Video

Abstract

The work at hand presents a novel data-driven framework for 3D full body human motion reconstruction from uncalibrated monocular video data. To this end, we develop a knowledge base by taking 2D samples of the motion capture library from dierent viewing directions. This allows later steps to handle 2D query videos without any information on the viewing direction. We detect and track features from input video sequences by utilizing low-level image based feature detection techniques like MSER and SURF. This process is stabilized by back projection of high-level 3D prior information obtained from the motion capture library to the image plane. Extraction of suitable feature sets from both, input control signals and motion capture data, enables us to retrieve the best relevant prior poses from the motion capture library by employing fast motion retrieval techniques. Finally, 3D motion sequences are reconstructed by non-linear energy minimization, that takes into account multiple prior terms. Furthermore, we propose a method to estimate camera parameters from input video itself and sampling of motion capture library.