Research on fusion algorithm of infrared and visible image based on generative adversarial network and attention mechanism

Abstract

Generative adversarial network was proposed in 2014, and its core is a two-person zero-sum game, which improves the quality of generated images through the mutual game between the generator and the discriminator. In 2018, Ma et al. applied Generative Adversarial Networks to the field of infrared and visible image fusion. The generator generates an image that retains both the infrared intensity and visible light detail texture information. By inputting the visible image as "true data" into the discriminator, it retains more detailed texture parts in the visible image in the fusion result, and finally obtains a nice effect. However, since the model only uses one discriminator, it loses some infrared intensity information and detail information in the infrared image. The gradient of the loss function is not enough to describe the detailed texture of visible light, and information such as brightness and contrast is not considered, so the final fusion result has poor visual effect. This paper builds an infrared and visible image fusion network based on the LSGAN framework and uses dual discriminators, and introduces a convolutional attention module in the generator to make the fusion image pay more attention to infrared intensity information. It is proposed to use MS-SSIM loss function to constrain the generated image, so that the fused image has a higher structural similarity with the source image. Through qualitative and quantitative analysis, it is proved that this method retains the main infrared intensity information while retaining the detailed texture information of the visible image.