Sparse transfer for facial shape-from-shading

Abstract

We present an image-based 3D face shape reconstruction method which transfers shape cues inferred from source face images to guide the reconstruction of the target face. Specifically, a sparse face shape adaption mechanism is used to generate a target-specific reference shape by adaptively and selectively combining source face shapes. This reference shape can also facilitate the reconstruction optimization for the target shape. As an off-line process, each source shape has been derived from a set of given sufficient source images (more than 9) based on a non-Lambertian reflectance model. Such a process allows for the existence of cast shadow and specularity, and more accurately infers the source shape. Guided by the target-specific reference shape, the shape of a target face can be estimated using a small number of images (even only one). The proposed reconstruction method refers to a lighting estimation and an albedo estimation for the target face. No standard 3D shape (such as the high-precision scanned 3D face) is required in the reconstruction process. Compared to the state-of-the-arts including the Photometric Stereo, Tensor Spline, the single reference based method, and the GEM algorithm, the proposed sparse transfer model can produce visually better facial details and obtain smaller reconstruction errors.