Unsupervised face frontalization using disentangled representation-learning CycleGAN

Abstract

Face frontalization aims to normalize profile face images to frontal ones for pose-invariant face-related recognition tasks. Current works have achieved outstanding results in face frontalization by using deep learning techniques. However, paired training data is usually required to train a deep face frontalization model and it is always difficult and time-consuming to acquire this kind of data. To solve this problem, we propose an unsupervised face frontalization framework, named Disentangled Representation-learning Cycle Generative Adversarial Network (DRCycleGAN). The model can be trained with unpaired data through embedding face images onto two spaces, identity feature space and pose feature space, and jointly inputting the identity feature and the pose feature to the generators to implement a paired forward and backward mapping (i.e., face frontalization and its inverse process). To adapt to the face frontalization task, a semantic-level cycle consistency loss is proposed, which implements consistency constraint supervision by measuring high-level semantic feature differences. Extensive experiment results demonstrate that the proposed method can achieve promising face frontalization performance and improve the pose-invariant face recognition performance.