RaGMAN: A relativistic average generative multi-adversarial network for pansharpening

Abstract

Pansharpening aims to reconstruct a high-resolution multi-spectral image (HRMS) from a low spatial resolution multi-spectral (MS) image and a high spatial resolution single band panchromatic (PAN) image acquired from the same satellite. To solve the spatial information and spectral information of the fused image that are not evenly preserved, this paper proposes a relativistic average generative multi-adversarial network (RaGMAN) for pansharpening. The RaGMAN consists of two parts. One builds a two-stream generator network to generate HRMS images by extracting features from PAN and MS images. The other is a dual discriminator to preserve the spectral and spatial information of the input when performing fusion. To make the generated image contain more spatial information, we propose multiple residual dense block in the generator. At the same time, in order to improve the overall quality of the fused images, two relativistic average discriminators are used in the network. Furthermore, a novel hybrid loss function is introduced to optimize training. Compared with seven state-of-the-art methods on GF-2, QB and WV-3 datasets, experimental results show that the proposed RaGMAN method can produce excellent pansharpening performance in terms of spatial and spectral fidelity.