Abstract
Inspired by the impressive achievements of convolutional neural networks (CNNs) in various computer vision tasks and the effective role of attention mechanisms, this paper proposes a two-branch fusion network based on attention feature fusion (AFF) called Attention FPNet to solve the pansharpening problem. We reconstruct the spatial information of an image in the high-pass filter domain and fully consider the spatial information in the multispectral (MS) and panchromatic (PAN) images. At the same time, the input PAN image and the upsampled MS image are directly transmitted to the reconstructed image through a long skip connection. The spectral information of the PAN and MS images is considered to improve the spectral resolution of the fused image. It also supplements the loss of spatial information that may be caused by network deepening. Moreover, an AFF method is used to replace the existing simple channel concatenation method commonly used in pansharpening, which fully considers the relationship between different feature maps and improves the fusion quality. Through experiments on image datasets acquired by the Pleiades, SPOT-6 and Gaofen-2 satellites, the results show that this method can effectively fuse PAN and MS images and generate a fused image and outperforms existing methods.
Highlights
W ITH the development of remote sensing technology, remote sensing applications have been widely used in agriculture and military fields
Combining the advantages of a convolutional neural networks (CNNs) and the effectiveness of the attention mechanism, a two-branch fusion network based on attention feature fusion (AFF) is proposed, called Attention FPNet, to solve the pansharpening problem of remote sensing images
Since the pansharpening task is to obtain MS images with high spatial resolution and high spectral resolution at the same time and existing methods often use certain feature extraction methods to extract the spectral information of MS images, such operations cause the loss of spectral information in the MS image and ignore the spectral information that may exist in the PAN image
Summary
W ITH the development of remote sensing technology, remote sensing applications have been widely used in agriculture and military fields. Combining the advantages of a CNN and the effectiveness of the attention mechanism, a two-branch fusion network based on attention feature fusion (AFF) is proposed, called Attention FPNet, to solve the pansharpening problem of remote sensing images. Different from the previous methods, to make full use of the spatial information in the MS and PAN images, we first fuse the spatial information of the MS and PAN images after highpass filtering and reconstruct the spatial information of the fused image, while using an AFF module to replace the commonly used concatenation operation, considering the relationship between different channels, thereby improving the quality of feature fusion. We use a long skip connection to directly propagate the input PAN image and the upsampled MS image to the fused image after spatial reconstruction, and the loss of spectral information is reduced.
Sign-in/Register to view highlights & summary 
Published Version
Talk to us
Join us for a 30 min session where you can share your feedback and ask us any queries you have
Schedule a call