WaveFusionNet: Infrared and visible image fusion based on multi-scale feature encoder–decoder and discrete wavelet decomposition

Abstract

To merge complementary information from multimodal images, such as thermal saliency from infrared images and texture details from visible images, traditional multi-scale transform-based methods have been extensively studied, with deep learning-based methods gaining significant popularity in recent years. However, there has been limited research on optimally combining the advantages of these two categories in fusion. In this paper, we propose a novel infrared and visible image fusion (IVIF) framework, WaveFusionNet, which integrates precise frequency feature decomposition from the discrete wavelet transform (DWT) with the comprehensive feature extraction from the multi-scale encoder. Firstly, we train an encoder–decoder network for multi-scale feature extraction and image reconstruction. DWT is used for down-sampling with minimal information loss by decomposing extracted features into low and high-frequency sub-bands. Next, a dual-band feature fusion (DBFF) module is trained to merge these sub-bands by integrating a spatial feature transform-based sub-network for low-frequency fusion and a maximum absolute value selection strategy for fusing high-frequencies. Finally, all fused sub-bands are fed into the pre-trained decoder to reconstruct the final image. Experimental results on three benchmark datasets (TNO, Roadscene, and MSRS) demonstrate that the proposed fusion method outperforms recent IVIF methods in both quantitative assessment and visual perception while maintaining competitive time complexity.