A generative adversarial network with structural enhancement and spectral supplement for pan-sharpening

Abstract

Pan-sharpening aims to obtain high-resolution multi-spectral images by fusing panchromatic images and low-resolution multi-spectral images though reasonable rules. This paper proposed a novel generative adversarial network for pan-sharpening, which utilizes the supplemented spectral information from low-resolution multi-spectral images and the enhanced structural information from panchromatic images to generate high-resolution multi-spectral images. Firstly, the forward differential operator is used to extract the spatial structural information of the panchromatic image both in the horizontal and vertical directions. Secondly, an architecture of generative adversarial network is designed. The enhanced structural information generated by the accumulation of the structural information of the two directions is added to the image fusion process in generator and the discriminating process in discriminator, and a new optimization objective is designed accordingly. What is more, the low-resolution multi-spectral image is added to the convolution process in the generator as a supplement to the spectral information. Finally, in order to obtain better image generation effect, a special objective function of the generator is designed, which adds a unique relationship to reduce the loss of spatial structural information and spectral information of fused images. Experiments on QuickBird and WorldView-3 satellites datasets show that the proposed method can generate high quality fused images and is better than most advanced methods in both objective indicators and intuitive observations.