Reference guided image super-resolution via efficient dense warping and adaptive fusion

Abstract

Due to the limited improvement of single-image based super-resolution (SR) methods in recent years, the reference based image SR (RefSR) methods, which super-resolve the low-resolution (LR) input with the guidance of similar high-resolution (HR) reference images are emerging. There are two main challenges in RefSR, i.e. reference image warping and exploring the guidance information from the warped references. For reference warping, we propose an efficient dense warping method to deal with large displacements, which is much faster than traditional patch (or texture) matching strategy. For the SR process, since different reference images complement each other, and have different similarities with the LR image, we further propose a similarity based feature fusion strategy to take advantage of the most similar reference regions. The SR process is realized by an encoder–decoder network and trained with pixel-level reconstruction loss, degradation loss and feature-level perceptual loss. Extensive experiments on three benchmark datasets demonstrate that the proposed method outperforms state-of-the-art SR methods in both subjective and objective measurements.