Semantic attention-based heterogeneous feature aggregation network for image fusion

Abstract

Infrared and visible image fusion aims to generate a comprehensive image that retains both salient targets of the infrared image and texture details of the visible image. However, existing methods overlook the differences in attention to semantic information among different fused images. To address this issue, we propose a semantic attention-based heterogeneous feature aggregation network for image fusion. The key component of our network is the semantic attention-based fusion module, which leverages the weights derived from semantic feature maps to dynamically adjust the significance of various semantic objects within the fusion feature maps. By using semantic weights as guidance, our fusion process concentrates on regions with crucial semantics, resulting in a more focused fusion that preserves rich semantic information. Moreover, we propose an innovative component called the attentive dense block. This block effectively filters out irrelevant features during extraction, accentuates essential features to their maximum potential, and enhances the visual quality of the fused images. Importantly, our network demonstrates strong generalization capabilities. Extensive experiments validate the superiority of our proposed network over current state-of-the-art techniques in terms of both visual appeal and semantics-driven evaluation.