Pixel-Level Kernel Estimation for Blind Super-Resolution

Abstract

Throughout the past several years, deep learning-based models have achieved success in super-resolution (SR). The majority of these works assume that low-resolution (LR) images are ‘uniformly’ degraded from their corresponding high-resolution (HR) images using predefined blur kernels — all regions of an image undergoing an identical degradation process. Furthermore, based on this assumption, there have been attempts to estimate the blur kernel of a given LR image, since correct kernel priors are known to be helpful in super-resolution. Although it has been known that blur kernels of real images are non-uniform (spatially varying), current kernel estimation algorithms are mostly done at image-level, estimating one kernel per image. These algorithms inevitably become sub-optimal in handling scenarios where an image is degraded non-uniformly. A divide-and-conquer form of approach, dividing an image into several patches for individual kernel estimation and SR can be a simple solution for this matter. Nevertheless, this approach fails in practice. In this paper, we address this issue by pixel-level kernel estimation. The three main components for training a SR framework based on pixel-level kernel estimation are as follows: Kernel Collage — a method for synthesizing non-uniformly degraded LR images, designed considering the coherency of kernels at neighboring regions while abruptly changing at times, the indirect loss — a novel loss for training the kernel estimator, based on the reconstruction loss, and an additional optimization — a scheme to robustify the SR network to minor errors in kernel estimations. Extensive experiments show the superiority of pixel-level kernel estimation in blind SR, surpassing state-of-the-art methods in terms of quantitative and qualitative results.