SRUNet: Stacked Reversed U-Shape Network for Lightweight Single Image Super-Resolution

Abstract

In recent years, for relieving the heavy computational cost, lightweight models have been successfully applied to single image super-resolution (SISR) task. However, most lightweight models adopt low-resolution images as input and apply transposed convolution or subpixel convolution only in the tail of the model to reconstruct the super-resolution image which neglect to fully utilize the multi-scale features. To resolve this problem, we propose a stacked reversed U-shape network (SRUNet) to further extract and utilize the multi-scale features at different resolutions. In detail, SRUNet consists of shallow feature extraction (SFE), stacked reversed U-shape module (SRUM), multi-scale backward fusion module (MSBFM) and feature refinement module (FRM). Instead of upsampling the feature map at the tail of the model, we perform the upsampling and downsampling operation progressively and iteratively by the stacked reversed U-shape module to extract richer multi-scale features. Furthermore, for archiving better use of multi-scale features, a scale-wise dense connection with residual channel-wise attention and multi-scale backward fusion is added to the network. The fused super-resolved features are refined by the feature refinement module and reconstructed to the image. Extensive experiments demonstrate that our model can achieve competitive performance compared with the state-of-the-art methods. When scaling factor is 4, SRUNet achieves the highest SSIM performance in all benchmarks and it takes 29.2ms to process per image in Urban100 dataset.