Feature Decomposition Transformers for Infrared and Visible Image Fusion

Abstract

Translate Article

We propose an infrared and visible image fusion algorithm using modality-shared and modality-specific feature decomposition transformers. First, the proposed algorithm extracts multiscale shallow features of infrared and visible images. Then, we develop modality-shared and modality-specific feature decomposition transformers that decompose the features into common and complementary components for each modality. For better decomposition, we develop a decomposition loss by constraining the common features to be correlated while the complementary features are uncorrelated. Finally, the reconstruction block generates the fused image by combining the common and complementary features. Experimental results show that the proposed algorithm significantly outperforms conventional algorithms on several datasets.