Large-View 3D Color Face Reconstruction from Dingle Image via UV Location Map and CGAN

Abstract

Image-based 3D reconstruction has been a hot topic in computer vision research. Since 3D face model can convey more information, it possesses expansive application prospects in comparison with the 2D face image, such as more precise identity information recognition signs, more accurate emotional expression media, etc. In order to reconstruct a realistic 3D color face model from a single large-view 2D face image, a simple but effective method is proposed. First, the encoder-decoder network is applied to generate the 2D UV location map from the original RGB image. Then a simple convolution neural network is applied to regress the 3D face from the UV location map. Finally, a network based on conditional generative adversarial networks (CGAN) is proposed to make up for the missing of UV texture caused by self-occlusion in the case of large pose. Compared with existing 3D face reconstruction methods in Stirling/ESRC 3D Face Database, proposed method can improve the accuracy. More important, a complete and realistic 3D face model can be reconstructed from the large-view face image even in the increasingly complex background.