Unsupervised face Frontalization for pose-invariant face recognition

Abstract

Face frontalization aims to normalize profile faces to frontal ones for pose-invariant face recognition. Current works have achieved promising results in face frontalization by using deep learning techniques. However, training deep models of face frontalization usually needs paired training data which is undoubtedly costly and time-consuming to acquire. To address this issue, we propose a Pose Conditional CycleGAN (PCCycleGAN) to generate authentic and identity-preserving frontal face images for pose-invariant face recognition. First, through coupling with a pair of inverse mappings, constraining with cycle consistent loss and using conditional pose label to control specific face pose generation, PCCycleGAN can be trained with unpaired samples. Second, pixel-level loss, feature space perception loss, and identity preserving loss are introduced in PCCycleGAN to help synthesize realistic and identity-preserving frontal face images. Extensive experiments on both constrained Multi-PIE dataset and unconstrained LFW and IJB-A datasets are conducted on face synthesis and pose-invariant face recognition. Results demonstrate that the proposed face frontalization model can synthesize frontal faces with high image quality as well as maintaining the identity information in both the constrained and unconstrained environments. In addition, our method enhances the performance of face recognition on the Multi-PIE, LFW and IJB-A datasets and achieves competitive face recognition performance on LFW and IJB-A datasets.