MSPIF: Multi-stage progressive visible and infrared image fusion with structures preservation

Abstract

Infrared and visible image fusion is one of the hottest research in computer vision to improve image quality. Traditional fusion methods suffer from detail loss, low resolution, and application scenarios limitation. To address these problems, this paper proposes a multi-stage progressive visible and infrared image fusion strategy (MSPIF), including the following key stages: Firstly, the visible image is enhanced using a weighted fusion visible image enhancement algorithm. Secondly, the infrared and enhanced visible images are both decomposed by a pre-trained network, namely Retinex_Net, to obtain their respective illumination and reflectance components. Thirdly, the reflectance components are decomposed by discrete wavelet transform (DWT). The low-frequency components are fused by a weighted information entropy fusion strategy while the high-frequency components are fused by a local energy fusion strategy. Such two fused parts contribute to the final fused reflectance component with inverse DWT (IDWT). Meanwhile, the illumination components are fused using a multi-scale fusion strategy based on the Laplace pyramid. Finally, the improved result is generated by the fused reflectance and illumination components based on Retinex theory. Quantitative and qualitative results of experiments on the TNO, KAIST, and ROAD datasets show that the proposed MSPIF is effective and achieves good results with structures preservation, even is superior to existing methods.