SBIT-Fuse: Infrared and visible image fusion based on Symmetrical Bilateral interaction and Transformer

Abstract

Infrared and visible image fusion, as a typical image enhancement task, aims to generate an image containing complementary information. Existing deep learning-based methods usually use a dual-stream model to extract features, which suffers from insufficient feature interaction in the feature extraction process. In order to effectively utilize information, this work proposes a Symmetrical Bilateral Interaction and Transformer fusion method (SBIT-Fuse) that is simple and efficient for constructing a dual-stream interaction network. Specifically, inspired by the problem of dead ReLU, we propose a Symmetrical Bilateral Interaction (SBI) module which consists of a certain number of cascaded cross domain activation interaction (CDAI) layers, where the deactivation information of the ReLU rectifier is transferred from one flow to another instead of being discarded. The proposed module exploits the drawbacks of ReLU to efficiently build the interaction of two streams and combine spatial and channel attention mechanisms, achieving the enhancement of feature extraction. Besides, our network is end-to-end and a transformer module is employed for extracting long-range semantic information. Moreover, combining a novel designed loss function, the proposed method can generate higher-quality fusion images. The comparative experiments with existing advanced methods and ablation studies on publicly datasets demonstrate the effectiveness of the proposed approach. Our code can be found at https://github.com/ljx111790/SBIT-Fuse.