NAF: Nest Axial Attention Fusion Network for Infrared and Visible Images

Abstract

In recent years, deep learning has been widely used in the field of infrared and visible image fusion. However, the existing methods based on deep learning have the problems of losing details and less consideration of long-range dependence. To address that, we propose a novel encoder-decoder fusion model based on nest connections and Axial-attention, named NAF. The network can extract more multi-scale information as possible and retain more long-range dependencies due to the Axial-attention in each convolution block. The method includes three parts: an encoder consists of convolutional blocks, a fusion strategy based on spatial attention and channel attention, and a decoder to process the fused features. Specifically, the source images are firstly fed into an encoder to extract multi-scale depth features. Then, a fusion strategy is employed to merge the depth features of each scale generated by the encoder. Finally, a decoder based on nested convolutional block is exploited to reconstruct the fused image. The experimental results on public data sets demonstrate that the proposed method has better fusion performance than other state-of-the-art methods in both subjective and objective evaluation.