Back-projection-based progressive growing generative adversarial network for single image super-resolution

Abstract

Recent advanced deep learning studies have shown the positive role of feedback mechanism in image super-resolution task. However, current feedback mechanism only calculates residual errors of images with the same resolution without considering the useful features that may be carried by different resolution features. In this paper, to explore the potential of feedback mechanism, we design a new network structure (progressive up- and downsampling back-projection units) to construct a generative adversarial network for single image super-resolution and use progressive growing methodologies to train it. Unlike previous feedback structure, we use progressively increasing scale factor to build up- and down-projection units, which aims to learn fruitful features across scales. This method allows us to get more meaningful information from early feature maps. Additionally, we train our network progressively; in the process of training, we start from single layer network structure and add new layers as the training goes on. By this mean, the training process can be greatly accelerated and stabilized. Experiments on benchmark dataset with the state-of-the-art methods show that our network achieves 0.01 dB, 0.11 dB, 0.13 dB and 0.4 dB better PSNR results than that of RDN+, MDSR, D-DBPN and EDSR on 8 $$\times $$ enlargement, respectively, and also achieves favorable performance against the state-of-the-art methods on 2 $$\times $$ and 4 $$\times $$ enlargement.