Infrared and visible image fusion based on guided hybrid model and generative adversarial network

Abstract

To retain the detail information and enhance the visibility of the source images, a new and effective image fusion based on a feature decomposition guided hybrid model and GAN is proposed. Firstly, the source images are divided into low-frequency components and high-frequency components by latent low-rank decomposition. Secondly, a multi-scale image enhancement method is used to improve its detail information for the low-frequency component. The low-frequency components are fused to obtain the base layer image by an improved generative adversarial network method. WLS-based fusion rule is used to improve the contrast of an image and get the detail layer image. Finally, the fused image is reconstructed by the linear combination of final base and detail layer images. Experiments are conducted on public datasets and compared with the state-of-the-art image fusion methods. Our experiments show that the proposed method can achieve good scores based on multiple evaluation indices, such as spatial frequency (SF), peak signal-to-noise ratio (PSNR) and pixel feature mutual information (FMI_pixel).