"Deepfake" Portrait Image Generation

Abstract

With the prevailing of deep learning technology, especially generative adversarial networks (GAN), generating photo-realistic facial images has made huge progress. Image generation techniques have many good applications such as data augmentation, entertainment, augmented/virtual reality as well as bad applications like Deepfake, which has caused huge concerns in the society. In this talk, we mainly review general image generation techniques, particularly describing a few of our recent work on high-quality portrait/facial image generation. Our work can be divided into 3D based approaches and GAN based 2D approaches. For 3D based approaches, we will explain how to model facial geometry, reflectance and lighting explicitly, and then show how such 3D modelling knowledge can be used in portrait manipulation. For 2D GAN based approaches, we will present a framework for pluralistic facial image generation from a masked facial input, where all the previous approaches only aim to produce one output. At the end, some suggestions will be provided for detecting deepfake from a generation point of view.