U-shaped pyramid dense residual network for pansharpening

Abstract

ABSTRACT With the widespread use of remote sensing images, pansharpening techniques have received increasing attention. Many pansharpening experiments have demonstrated that multiscale fusion effectively alleviates the problem of image detail loss, and the pyramidal structure is one of the manifestations of the multiscale spatial theory. This paper proposes a U-shaped network architecture based on the pyramid structure for pansharpening. We resample multispectral images and panchromatic images to generate two pyramid image sequences. The image pairs in the same sequence layer are double-merged in pixels and channels before extracting features. The pyramidal feature sequences undergo a U-shaped top-down downsampling fusion and bottom-up upsampling reconstruction to generate high-resolution multispectral images. In addition, this process also utilizes the dense network module to extract deep features and a lateral connection to shallow multiplex features. We conducted experimental tests on full-resolution and reduced-resolution datasets from WorldView-2, IKONOS, and GeoEye-1. The results confirm that the method in this paper can better trade off spectral and spatial features. The technique has better generalization in fusion results and advantages in objective and subjective aspects.