CtrlFaceNet: Framework for geometric-driven face image synthesis

Abstract

Abstract In this work, we introduce a novel framework based on Generative Adversarial Networks to control the pose, expression and facial features of a given face image using another face image. It can then be used for data augmentation, pose invariant face identification, face verification, and lightweight image editing. Generating new realistic face images with controllable poses, facial features, and expressions is a challenging generative learning problem due to skin tone variations, the identity preservation problem, necessity to deal with unseen large poses, and the absence of ground truth images in the training process. We make the following contributions. First, we present a network, CtrlFaceNet that can control a source face image while preserving the identity and skin tone. Second, we introduce a method for training the framework in fully self-supervised mode using a large-scale dataset of unconstrained face images. Third, we show that the style loss function can be used to preserve the skin tone of the source image. The experimental results show that our approach outperforms all other baselines. Furthermore, to the best of our knowledge, we are the first to train such a model using large-scale dataset of unconstrained face images.