A sub-pixel convolution-based improved bidirectional feature pyramid network for pansharpening

Abstract

ABSTRACT Due to the dramatic scale changes in different ground object characteristics, effectively utilizing multi-scale features of remote sensing images represents a major challenge. This letter proposes a sub-pixel convolution-based improved bidirectional feature pyramid network (SCIBFPN) to address the problem. To reduce the information loss in the process of image pre-processing and multi-scale feature fusion, this letter introduces sub-pixel convolution instead of up-sampling. To obtain richer spatial and spectral information, an improved bidirectional feature pyramid network-based sub-pixel convolution is combined with a residual network (ResNet) for feature extraction. In addition, to further enhance the spatial structure of the fused images, a panchromatic (PAN) image is used as a guide to direct the injection of spatial information. Experimental results from two real datasets show that the proposed method outperforms state-of-the-art methods in terms of both objective metrics and subjective visual evaluation.