3D Face Cartoonizer: Generating Personalized 3D Cartoon Faces from 2D Real Photos with a Hybrid Dataset

Abstract

AbstractCartoon face is a prevalent kind of stylized face, which is widely used in movies, TVs and advertisements. Although plenty of methods have been proposed to generate 2D cartoon faces, it is still challenging to learn personalized 3D cartoon faces directly from 2D real photos. To solve this problem, we contribute the first 3D cartoon face hybrid dataset with both large amounts of low-quality and a small number of high-quality face triplets. Each triplet contains a 2D real face, as well as its corresponding 2D and 3D cartoon faces. To leverage the hybrid dataset, we propose Recon2AGen which first pretrains our network with low-quality triplets in a reconstruction-then-generation manner and then finetunes it with high-quality triplets in an adversarial manner. In this way, we solve the 2D-to-3D ambiguity and the real-to-cartoon transformation by disentangling the task into three progressively learned sub-tasks. And the hybrid dataset is fully explored to achieve generalizable and high accuracy results. Extensive experiments show that our generated 3D cartoon faces are of high quality and can be easily edited and animated, enabling extensive practical applications. Code and dataset will be available at https://github.com/mingsjtu/3DCartoonGenerator.Keywords3D face generationCartoon faceDataset