Collaborative Completion of Facial Texture and Displacement in UV Space

Abstract

When the input face has large pose, it’s hard for single view 3D face reconstruction methods to estimate complete facial texture and geometric details due to self-occlusion. Considering the correlation between the feature distributions of texture and geometric details, TDGAN is proposed, a collaborative completion model with GAN structure which could complete texture and geometric details collaboratively in a uniform framework. Firstly, the texture and geometric details are mapped into the UV space. Then, the collaborative completion is performed in a generative adversarial network which includes a generator, two global discriminators and two local discriminators. Finally, to exploit the common structures of texture and geometric details, a consistency constraint module is incorporated to the framework. The complete and consistent texture and geometric details UV maps are synthesized. Experiments on the currently largest 3D face dataset demonstrate that the proposed collaborative completion method could produce more high quality results than the independent UV completion methods.