Resolution-Preserving Generative Adversarial Networks for Image Enhancement

Donghyeon Lee,Kyujoong Lee,Sangheon Lee,Hoseong Lee,Hyuk-Jae Lee

doi:10.1109/access.2019.2934320

Abstract

Generative adversarial networks (GANs) are used for image enhancement such as single image super-resolution (SISR) and deblurring. The conventional GANs-based image enhancement suffers from two drawbacks that cause a quality degradation due to a loss of detailed information. First, the conventional discriminator network adopts strided convolution layers which cause a reduction in the resolution of the feature map, and thereby resulting in a loss of detailed information. Second, the previous GANs for image enhancement use the feature map of the visual geometry group (VGG) network for generating a content loss, which also causes visual artifacts because the maxpooling layers in the VGG network result in a loss of detailed information. To overcome these two drawbacks, this paper presents a proposal of a new resolution-preserving discriminator network architecture which removes the strided convolution layers, and a new content loss generated from the VGG network without maxpooling layers. The proposed discriminator network is applied to the super-resolution generative adversarial network (SRGAN), which is called a resolution-preserving SRGAN (RPSRGAN). Experimental results show that RPSRGAN generates more realistic super-resolution images than SRGAN does, and consequently, RPSRGAN with the new content loss improves the average peak signal-to-noise ratio (PSNR) by 0.75 dB and 0.32 dB for super-resolution images with the scale factors of 2 and 4, respectively. For deblurring, the visual appearance is also significantly improved, and the average PSNR is increased by 1.54 dB when the proposed discriminator and content loss are applied to the deblurring adversarial network.

Highlights

Generative adversarial networks (GANs) [1] consist of a discriminator network (D) and a generator network (G), where G aims to transform the input noisy samples (z) into realistic data samples and D discerns how much realistic data sample G produces as output
This paper focuses on the architecture of the discriminator network, so the baseline network (BNet) [46] is selected for the comparison because the sizes of BNet and resolution-preserving SRGAN (RPSRGAN) are similar
The resolutionpreserving discriminator network architecture maintains the detailed information of an input image and generates the image from the generator network without a loss of detailed information

Summary

INTRODUCTION

Generative adversarial networks (GANs) [1] consist of a discriminator network (D) and a generator network (G), where G aims to transform the input noisy samples (z) into realistic data samples and D discerns how much realistic data sample G produces as output. The reduction of a feature map improves its accuracy because the VGG19 network only keeps the global information required for the classification This means that it removes detailed local information of an image, which is essential for the high-quality SISR. Application of these two methods, i.e., the proposed architecture of D and the proposed content loss, to SISR achieves improvement of the visual similarity between the super-resolution and original images. CONVOLUTIONAL NEURAL NETWORK-BASED IMAGE ENHANCEMENT SISR aims to generate a super-resolution image (I SR), an output image of CNNs, that does not lose the detailed information of the original high-resolution image (I HR). An enhanced deep super-resolution network (EDSR) [8] adopts residual blocks [23] to further improve the visual quality of an image over the existing CNN-based SISR methods. Nah et al [16] and Noroozi et al [17] propose deblurring operations using multi-scale CNNs

GENERATIVE ADVERSARIAL NETWORKS FOR IMAGE ENHANCEMENT

REESOLUTION-PRESERVING

REESOLUTION-PRESERVING CONTENT LOSS

CONCLUSION