An Information Retention and Feature Transmission Network for Infrared and Visible Image Fusion

Abstract

The aim of infrared and visible image fusion is to generate a composite image that contains the thermal radiation information in the infrared image and optical spectral information in visible image. In this paper, we proposed an end-to-end deep infrared and visible image fusion network which has the capability of information retention and feature transmission. In our proposed network, residual dense blocks (RDB) are introduced to ensure complete deep features extraction of source images. We also design intermediate feature transmission blocks to avoid information loss caused by convolution. In addition, we constrain the network by a comprehensive loss function based on image intensity, gradients, and structure similarity. The loss function ensures that the fused images retain rich details. We introduce weight blocks to produce adaptive weights to control the retention of similar information in two source images, which can reduce the intermediate information loss and play the role of information retention. Extensive experiments on both public <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">TNO</i> and <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">RoadScene</i> datasets are conduced to test the performances of the proposed method. Related ablation experiments are conducted to investigate the validation of the weight blocks and the feature transmission blocks. The experimental results demonstrate that the fusion results of the proposed network show more texture information and better visual quality than other state-of-the-art fusion methods. From both subjective and objective points, our method is competitive with or even outperform most of advanced fusion methods.