Attention fusion network for multi-spectral semantic segmentation

Abstract

To improve the accuracy of multi-spectral semantic segmentation, an attention fusion network (AFNet) based on deep learning is proposed. Different from current methods, the AFNet uses a co-attention mechanism by designing an attention fusion module to calculate the spatial correlation between the red-green-blue (RGB) image and infrared (IR) image feature maps to guide the fusion of features from different spectra. This approach enhances the feature presentation and makes full use of the complementary characteristics of multi-spectral sources. The proposed network is tested on RGB-IR datasets and compared with relevant state-of-the-art networks. The experimental analyses prove that the proposed AFNet can improve multi-spectral semantic segmentation results with good visual definition and high accuracy in classification and localization.