3D face reconstruction via landmark depth estimation and shape deformation

Abstract

Since a human face could be represented by a few landmarks with less redundant information, and calculated by a linear combination of a small number of prototypical faces, we propose a two-step 3D face reconstruction approach including landmark depth estimation and shape deformation. The proposed approach allows us to reconstruct a realistic 3D face from a 2D frontal face image. First, we apply a coupled dictionary learning method based on sparse representation to explore the underlying mappings between pair of 2D and 3D training landmarks. In the method, a weighted l1 norm sparsity function is introduced to better pursuit the l0 norm sparsity. Then, the depth of the landmarks could be estimated. Second, we propose a novel shape deformation method to reconstruct the 3D face by combining a small number of most relevant deformed faces which are obtained by the estimated landmarks. The sparsity regulation is also introduced to find the relevant faces in the second step. The proposed approach could explore the distributions of 2D and 3D faces and the underlying mappings between them well, because human faces are represented by low-dimensional landmarks, and their distributions are described by sparse representations. Moreover, it is much more flexible since we can make any change in any step. Extensive experiments are conducted on BJUT_3D database, and the results validate the effectiveness of the proposed approach.