Parallel interactive delayed attention network for pansharpening

Abstract

ABSTRACT As deep learning has gained wide attention in computer vision tasks, researchers have also started to explore the application of deep learning in panchromatic sharpening. In recent years, various convolutional neural network-based methods for panchromatic sharpening have been proposed, and these methods have achieved good results. However, shallower networks are unable to learn complex mapping relationships. Networks that are too deep are prone to overfitting, which leads to the degradation of the fusion effect. Moreover, since the convolutional operations are concentrated in local regions, it is difficult to obtain the association information with other regions even in deep networks. Therefore, more accurate feature selection and representation with limited network depth is needed. In this paper, a parallel interactive delayed attention network for panchromatic sharpening (PIDAN) is proposed. The network consists of multiple resolution subnetworks with parallel inputs to improve the representation of high-resolution feature maps by multiscale low-resolution features of the same depth and similar level. In addition, we propose a delayed channel attention module. This module obtains correlations between low-frequency and high-frequency information through adaptive learning, making the network more flexible in processing different types of information. The depth feature reweighting restriction combined with the residual unit can further avoid the overfitting representation of features as the network deepens. Experimental results on Gaofen-2 and WorldView-2 datasets show that the proposed method is competitive in both quality assessment and visual perception.