Multispectral image fusion using super-resolution conditional generative adversarial networks

Abstract

In multispectral image fusion scenarios, deep learning has been widely applied. However, the fusion performance and image quality are still restricted by inflexible architecture and supervised learning mode. We proposed multispectral image fusion using super-resolution conditional generative adversarial networks (MS-cGANs) based on conditional cGANs, which produces the fused image through the flexible encode-and-decode procedure. In the proposed network, a least square model is extended to solve the gradients vanishing problem in cGANs. Then, to improve the fusion quality, the multiscale features are used to preserve the details. Furthermore, the image resolution is promoted by adding the perceptual loss in object function and injecting the super-resolution structure into a deconvolution procedure. In experimental results, MS-cGANs demonstrates a significant performance in fusing multispectral images and top-ranking image quality compared with the state-of-the-art methods.