Unsupervised densely attention network for infrared and visible image fusion

Abstract

Integrating the information of infrared and visible images without human supervision is a long-standing problem. A key technical challenge in this domain is how to extract features from heterogeneous data-sources and fuse them appropriately. Prior deep learning works either extract the middle layers information or use costly training step to improve fusion performance, which limited their performances in cluttered scenes and real-time applications. In this paper, we introduce a novel and pragmatic unsupervised infrared and visible image fusion method based on a pre-trained deep network, which employs a densely connection structure and incorporates the attention mechanism to achieve high fusion performance. Furthermore, we propose to use the cross-dimensional weighting and aggregation to compute the attention map for infrared and visible image fusion. The attention map enables more efficient feature extraction and captures more structure information from source images. We evaluate our method and compare it with ten typical state-of-the-art fusion methods. Extensive experimental results demonstrate that our method achieves state-of-the-art fusion performance in both subjective and objective evaluation.