MrFDDGAN: Multireceptive Field Feature Transfer and Dual Discriminator-Driven Generative Adversarial Network for Infrared and Color Visible Image Fusion

Abstract

Most of the previous infrared and visible image fusion methods based on deep learning were on the basis of grayscale images and used the single convolution kernel receptive field to extract deep features, which would inevitably cause information loss in the process of feature transfer. Accordingly, this paper proposes a new generative adversarial network based on multi-receptive field feature transfer and dual discriminators, which is named MrFDDGAN. It is applied to infrared and color visible image fusion. Firstly, three different receptive fields are used to extract multi-scale and multi-level deep features of multi-modal images on three channels, so as to obtain significant features of source images from multi-view and multi-perspective in a more comprehensive way. Secondly, a feature interaction module is introduced in the encoder to realize the information interaction and information pre-fusion among the three feature channels. Thirdly, the multi-level features fused in the encoder are cascaded with the deep features in the decoder to enhance feature transfer and feature reuse. Finally, this experiment adopts a dual-discriminator adversarial network structure to keep the balance of infrared image intensity and visible image texture preserved by fusion results. And qualitative and quantitative experimental analyses are carried out on two public datasets of grayscale infrared as well as visible images and one dataset of infrared and color visible images. The experimental results prove that the proposed MrFDDGAN algorithm has a better subjective visual effect and objective performance than the existing state-of-the-art fusion methods.