Super-resolution of very low-resolution face images with a wavelet integrated, identity preserving, adversarial network

Abstract

Super-resolution of face images, known as Face Hallucination (FH), has been excessively studied in recent years. Modern FH methods use deep Convolution Neural Networks (CNN) with a pixel-wise MSE loss function to infer high-resolution facial images. The MSE-oriented approaches generate over-smooth results, particularly when dealing with very low-resolution images. Recently, Generative Adversarial Networks (GANs) have successfully been exploited to synthesize perceptually more pleasant images. However, the GAN-based models do not guarantee identity preservation during face super-resolution. To address these challenges, we have proposed a novel Wavelet-integrated, Identity Preserving, Adversarial (WIPA) approach. Specifically, we present Wavelet Prediction blocks attached to a Baseline CNN network to predict wavelet missing details of facial images. The extracted wavelet coefficients are concatenated with original feature maps in different scales to recover fine details. Unlike other wavelet-based FH methods, this algorithm exploits the wavelet-enriched feature maps as complementary information to facilitate the hallucination task. We introduce a wavelet prediction loss to push the network to generate wavelet coefficients. In addition to the wavelet-domain cost function, a combination of perceptual, adversarial, and identity loss functions has been utilized to achieve low-distortion and perceptually high-quality images while maintaining identity. The extensive experiments prove the superiority of the proposed approach over the state-of-the-art methods by achieving PSNR of 25.16 dB for CelebA dataset and verification rate of 86.1% for LFW dataset; both conducted on 8X magnification factor.