Multi-Sensor Fusion of Infrared and Visible Images Based on Modified Side Window Filter and Intensity Transformation

Abstract

For multi-sensor fusion of infrared and visible images, it is difficult to retain the thermal radiation information of the infrared image and the texture information of the visible image in the fused image. To overcome this problem, a novel infrared and visible image fusion method based on a modified side window filter (MSWF) and an intensity transformation is proposed. First, an MSWF with effective edge-preservation ability is developed by adding four additional kernels to better decompose the source images to obtain the base and detail layers. Furthermore, to extract the edge information of the source images, we propose to further decompose the base layers to obtain the low-frequency layers and high-frequency layers (edge information) through the non-subsampled shearlet transform (NSST). Then, an S-shape intensity transformation function (ITF) is proposed to enhance the saliency information and suppress the non-saliency information in the infrared image. In the fusion process, considering the characteristics of the decomposed components, different fusion rules are designed to obtain the fused detail layer and low- and high-frequency layers. Finally, these fused components are reconstructed to obtain the final fusion image. It is experimentally demonstrated that the proposed method is superior to state-of-the-art fusion methods both in terms of subjective evaluation and objective metrics.