Coupled image and kernel prior learning for high-generalized super-resolution

Abstract

Deep convolution neural networks (DCNNs) have demonstrated great success on single image super resolution, where most existing methods aim to construct a deep model in an fully-supervised way using large amount of synthetic image pairs under ideal degradation assumption. The super-resolution performance would be significantly degraded for the low-resolution images captured under uncontrolled imaging conditions with complicated degradation procedures. To handle the above limitations, this work proposes a high-generalized image super-resolution framework by synergistically learning the latent image and the degradation process to achieve the specific priors for an under-studying observation in an unsupervised manner. Specifically, we incorporate dual branches of networks to configure our framework, where one is structured with both convolution and transformer blocks to learn local-to-global prior to generate high-quality image while the other aims to learn the kernel prior with a simple convolution-based architecture. The kernel prior subnet is pre-trained to enhance the stability of the joint optimization for the overall unsupervised learning procedure. With the estimated degradation kernel and target image, we produce the approximated low-resolution version with a convolution-based degradation block to formulate the loss function for the specific learning. Moreover, to pursue visually plausible image generation, we further incorporate a perceptual loss by leveraging a pre-trained discriminator in Gan-based SR model. Extensive experiments on several benchmark datasets have demonstrated the effectiveness of our proposed high-generalized super-resolution model, manifesting superiority over both supervised and unsupervised state-of-the-art SR models.