Multi-Scale Fast Fourier Transform Based Attention Network for Remote-Sensing Image Super-Resolution

Abstract

Recently, with the rise and progress of convolutional neural networks (CNNs), CNN-based remote sensing image super-resolution (RSSR) methods have gained considerable advancement and showed great power for image reconstruction tasks. However, most of these methods cannot handle well the enormous number of objects with different scales contained in remote sensing images and thus limits super-resolution performance. To address these issues, we propose a Multi-Scale Fast Fourier Transform (FFT) based Attention Network (MSFFTAN) which employs a multi-input U-shape structure as backbone for accurate remote sensing image super-resolution. Specifically, we carefully design an FFT-based residual block consisting of an image domain branch and a Fourier domain branch to extract local details and global structures simultaneously. In addition, a Local-Global Channel Attention Block (LGCAB) is developed to further enhance the reconstruction ability of small targets. Finally, we present a Branch Gated Selective Block (BGSB) to adaptively explore and aggregate features from multiple scales and depths. Extensive experiments on two publicly datasets have demonstrated the superiority of MSFFTAN over the state-of-the-art (SOAT) approaches in aspects of both quantitative metrics and visual quality. The peak signal-to-noise ratio of our network is 1.5 dB higher than the SOAT method on the UCMerced LandUse with downscaling factor 2.