Monocular 3D facial shape reconstruction from a single 2D image with coupled-dictionary learning and sparse coding

Abstract

Monocular 3D face reconstruction from a single image has been an active research topic due to its wide applications. It has been demonstrated that the 3D face can be reconstructed efficiently using a PCA-based subspace model for facial shape representation and facial landmarks for model parameter estimation. However, due to the limited expressiveness of the subspace model and the inaccuracy of landmark detection, most existing methods are not robust to pose and illumination variation. To overcome this limitation, this work proposes a coupled-dictionary model for parametric facial shape representation and a two-stage framework for 3D face reconstruction from a single 2D image by using facial landmarks. Motivated by image super-resolution, the proposed coupled-model consists of two dictionaries for the sparse and the dense 3D facial shapes, respectively. In the first stage, the sparse 3D face is estimated from facial landmarks by using partial least-squares regression. In the second stage, the dense 3D face is reconstructed by 3D super-resolution on the estimated sparse 3D face. Comprehensive experimental evaluations demonstrate that the proposed coupled-dictionary model outperforms the PCA-based subspace model in 3D face modeling accuracy and that the proposed framework achieves much lower reconstruction error on facial images with pose and illumination variations compared to state-of-the-art algorithms. Moreover, qualitative analysis demonstrates that the proposed method is generalizable to different types of data, including facial images, portraits, and facial sketches.