Infrared and visible image fusion based on visibility enhancement and hybrid multiscale decomposition

Abstract

Infrared and visible image fusion technology aims to integrate the heat source information of infrared image into the visible image to generate a more informative image. Many fusion methods proposed in recent years have problems such as loss of detailed information and low contrast. In this paper, we propose a novel infrared and visible image fusion method based on visibility enhancement and hybrid multiscale decomposition. Firstly, we propose an effective image pre-processing method to increase the details and improve the quality of the source image. Then, the pre-processed images are decomposed by ℓ1−ℓ0 decomposition model to obtain the base and detail layers. Thirdly, for the base layer fusion part, we propose a meaningful method based on the weight of the visual saliency illumination map (VSIM), which not only preserves the contrast information but also does it guarantee the overall structure of the fusion result. For the detail layer fusion part, we take the advantage of convolutional neural network (CNN) to obtain the decision map of the fused detail layer. Next, we employ Laplacian and Gaussian pyramids to decompose the detail layers and decision map, respectively and then fuse them by a synthetic detail fusion strategy. Finally, we reconstruct the base and detail layers to generate the final fusion result. Experiment results demonstrate that our results are more in line with the human visual system (HVS) and outperform some state-of-the-art methods on quantitative metrics.