RELIEF: Joint Low-Light Image Enhancement and Super-Resolution with Transformers

Abstract

The goal of Single-Image Super-Resolution (SISR) is to reconstruct a High-Resolution (HR) version of a degraded Low-Resolution (LR) image. Existing Super-Resolution (SR) methods mostly assume that the LR image is a result of blurring and downsampling the HR image, while in reality LR images are often degraded by additional factors such as low-light, low-contrast, noise, and color distortion. Due to this, current State-of-the-Art (SoTA) SR methods cannot reconstruct real low-light low-resolution images, and a straightforward strategy is, therefore, to first perform Low-Light Enhancement (LLE), followed by SR, using dedicated methods for each task. Unfortunately, this approach leads to poor performance, which motivates us to propose a method for joint LLE and SR. However, since LLE and SR are both ill-posed and ill-conditioned inverse problems, the joint reconstruction task becomes highly challenging, which calls for efficient ways to leverage as much as possible of the available information in the degraded image during reconstruction. In this paper, we propose REsolution and LIght Enhancement transFormer (RELIEF), a novel Transformer-based multi-scale hierarchical encoder-decoder network with efficient cross-shaped attention mechanisms that can extract informative features from large training patches due to its strong long-range dependency modeling capabilities. This in turn leads to significant improvements in reconstruction performance on real Low-Light Low-Resolution (LLLR) images. We evaluate our method on two publicly available datasets and present SoTA results on both.