Two-Level Consistency Metric for Infrared and Visible Image Fusion

Abstract

Infrared and visible image fusion plays an important role in various instrument-related applications. The preservation of vital information and the measurement of fusion performance are difficult but important for the task. In this work, the vital information preservation problem is analysed by investigating the existing fused image metrics. Since there is no ground-truth, most existing methods generate the fused result by carefully designing a loss function to constrain the distance between the fused image and the two types of source images. This single-level consistency metric generates a fused result that is close to the compromise of the source images and leads to important features affecting each other. Thus, the high-frequency information of the source images cannot be well preserved in the fused result. To address this, a novel image fusion enhancement loss function based on the fusion decomposition and the high-frequency attention is proposed in this paper. This provides a two-level metric for infrared and visible image fusion. Next, an image fusion performance measurement module is designed to decompose the fused image and the frequency attention to locate the important features and enhance the source images. Furthermore, a novel self-supervised network is proposed to preserve the vital features of the source images by narrowing the distance between the decomposition components and the enhanced source images. The effectiveness of the proposed method is evaluated through comprehensive experiments on two public datasets. The obtained results demonstrate that the proposed method outperforms the existing fusion methods in both subjective and objective evaluations. The code of our fusion method is available at https://github.com/xplin13/Two-level-Consistency-Metric-for-IVIF.