CRNet:3D Face Reconstruction with Contour Map Regression Network

Abstract

2D face images represent faces with incomplete information. 3D face reconstruction from a single 2D image is a challenging problem with application value. The single feature extraction method distorts the generated 3D face image. In this paper, we use contour-based face segmentation method to reconstruct 3D face image. We focus on the edge and contour information of the face when using contour lines to segment the face. Different from the global 3D face reconstruction method, we combine the global and local face information to carry out 3D face reconstruction. Our method: First of all, we do contour segmentation for human faces and extract the features of the segmented images. Second, we learn the local binary features of each keypoint in a complete face image, then combine the features and use linear regression to detect the keypoints. Thirdly, we use Convolutional Neural Networks to learn the regression 3D Morphable Model coefficient and significantly improve the quality and efficiency of reconstruction. We regressed the coefficients of the 3D deformable model from 2D images to present face alignment for 3D face reconstruction. We carry out feature mapping between 2D face and 3D face image, and monitor and verify 3D face model through mapping relationship. Our method can not only reconstruct face images from all angles, but also reduce face deformities. We made face images fit better under different expressions and postures.