MCANet: A Multidimensional Channel Attention Residual Neural Network for Pansharpening

Abstract

In the remote sensing image fusion field, fusion methods based on deep learning are the latest techniques in panchromatic sharpening (pansharpening). However, existing pansharpening methods based on neural networks cannot adequately inject the spatial feature information of panchromatic (PAN) images into fusion images, and they do not exploit the feature relationships between spatial locations, such as rows and columns of feature maps. To solve these problems, a multidimensional channel attention residual neural network (MCANet) is proposed in this paper. To preserve the structural information in PAN images, a two-stream detail injection (TSDI) module is proposed, and the local skip connection operation is adopted to mine more spectral and structural information. A multidimensional channel attention (MCA) module is also designed to enable the network to learn the nonlinear mapping relationships between image spatial locations. In addition, a multiscale feature fusion (MSFF) module is designed to improve feature representation in the image fusion process, which is conducive to improving the pansharpening effect. The experimental results on the WorldView-2, GaoFen-2 and QuickBird datasets demonstrate that the proposed method outperforms state-of-the-art methods both visually and quantitatively.