Self-Supervised Frontalization and Rotation Gan with Random Swap for Pose-Invariant Face Recognition

Abstract

The task of pose invariant face recognition (PIFR) has recently seen important improvements by introducing generative adversarial networks (GANs). These GAN-based models synthesize a frontal face image from an image in any pose to boost recognition performance. Most of these methods, however, require the ground-truth frontal face image during optimization as they rely on supervised or semi-supervised learning. In this work, we introduce the self-supervised Frontalization and Rotation GAN (FRGAN), which can synthesize a frontal face image from a non-frontal face image. For self-supervision, the synthesized image is rotated back to the original pose based on reconstruction and adversarial losses. To improve performance, the FRGAN uses the Random Swap, a parameter-free data augmentation strategy that swaps key facial regions between the input image and its reconstructed version to force the generator to synthesize more realistic images. Our qualitative and quantitative experiments on benchmark datasets confirm the strong performance of the FRGAN compared to the state-of-the-art (SOTA).