A Global Optimization Approach to High-detail Reconstruction of the Head

Abstract

The paper presents an approach for reconstructing head-and-shoulder portraits of people from calibrated stereo images with a high level of geometric detail. In contrast to many existing systems, our reconstructions cover the full head, including hair. This is achieved using a global intensity-based optimization approach which is stated as a parametric warp estimation problem and solved in a robust Gauss-Newton framework. We formulate a computationally efficient warp function for mesh-based estimation of depth which is based on a well known image-registration approach and adapted to the problem of 3D reconstruction. We address the use of sparse correspondence estimates for initializing the optimization as well as a coarse-to-fine scheme for reconstructing without specific initialization. We discuss issues of regularization and brightness constancy violations and show various results to demonstrate the effectiveness of the approach.