FIN-GAN: Face illumination normalization via retinex-based self-supervised learning and conditional generative adversarial network

Abstract

Illumination normalization is one of the most challenging issues for facial analysis. To be specific, the variation of environmental illumination influences the visual quality of an image and drastically degrades the performance of face recognition, detection, landmark and other related applications. Retinex theory provides an important concept for processing illumination variation, which supposes that a face image can be decomposed into [an invariant reflectance component and a variant illumination component. Enlighten by this theory, in this paper, we put forward a novel deep learning approach which combines self-supervised learning and adversarial training for face illumination normalization (FIN-GAN). The proposed FIN-GAN framework can be implemented by two steps. Firstly, self-supervised learning is employed to decompose the original face image into the illumination component and the illumination-invariant component with Retinex constraint. Then, we employ the conditional generative adversarial network for face image reconstruction. For network optimization, we design the combined loss to ensure visual quality and preserve identity information. Experiments are performed on Extended-YaleB, CAS-PEAL, CMU-PIE and Multi-PIE datasets. Through multiple quantitative criteria, we demonstrate that the proposed FIN-GAN obtains promising performance in face illumination normalization.