MDFN: Mask deep fusion network for visible and infrared image fusion without reference ground-truth

Abstract

A single infrared image or visible image cannot clearly present texture details and infrared information of the scene in poor illumination, bad weather, or other complex conditions. Thus, it is necessary to fuse the infrared and visible images into one image. In this paper, we propose a novel deep fusion architecture for fusing visible and infrared images without any reference ground-truth. Different from existing deep image fusion methods which directly output the fused images, a weight score corresponding to each pixel is estimated by our network to determine the contributions of two source images. This strategy transfers the valuable information in source images to the fused image. Considering the salient thermal radiation information in the infrared image, a mask of the infrared image is generated and used to preserve valuable contents in the infrared and visible images for the fused image. Furthermore, a hybrid loss is designed to make the fused image consistent with two source images. On account of the weight estimation, the mask strategy, and the hybrid loss, the images fused by our proposed method jointly maintain the thermal radiation and texture details, achieving state-of-the-art performance compared with existing fusion approaches. Our code is publicly available at https://github.com/NlCxg/MDFN.