Facial Image Synthesis and Super-Resolution With Stacked Generative Adversarial Network

Abstract

Image synthesis and super-resolution (SR) have always been a hot spot for computer vision and image processing research. Since the development of Deep Learning, especially after the Deep Convolutional Generative Adversarial Network (DC-GAN) methods, facial image synthesis and SR problem had been solved in many circumstances. But most of the existing works were focused on natural-looking of the synthesized result rather than keeping facial information of the original image. Our paper presented an end-to-end method of getting high-resolution photo-realistic facial images from low-resolution (LR) in-the-wild images without losing the facial identity details. The pipeline used a flexible stacked GAN structure for the SR process with different target image resolutions on different upscaling factors. To avoid getting blur or nonsensical image output and realize the flexibility, “U-Net” architecture and upsampling layers with residual learning blocks were stacked. The stacked network structure makes applying different loss functions in different parts of the network possible, which helps to solve the two problems of keeping identical facial details of the LR input image and generating high-quality output images simultaneously. By using 3 different loss functions in different positions of the stacked network separately, through experimental comparison, we found the best stacked residual block parameters which could get the best output image quality. Experimental results also explicated that the network had a good SR ability compare to state of the art methods in different resolution and upscaling factor.