Lightweight image super-resolution with feature enhancement residual network

Abstract

Recently, single image super-resolution (SR) methods based on deep convolutional neural network (CNN) have demonstrated remarkable progress. The essence of most CNN-based models is to learn the non-linear mapping between low-resolution patches and corresponding high-resolution ones. However, numerous convolutions are applied to implement this mapping, which directly contributes to large model sizes and huge graphics memory consumption. In this paper, we propose a lightweight feature enhancement residual network (FERN) to achieve prominent performance by incorporating lightweight non-local operations into the residual block. By taking advantage of utilizing this non-locally enhanced residual block, the proposed model can capture long-range dependencies. For further improving performance, we design the structure-aware channel attention layer that explicitly boosts feature maps with more structural and textural details. Extensive experiments suggest that the proposed approach performs favorably against the state-of-the-art SR algorithms in terms of visual quality and inference time.