FPNFuse: A lightweight feature pyramid network for infrared and visible image fusion

Abstract

A novel deep learning structure of infrared and visible image fusion is proposed. In particular, feature pyramid networks are developed for enhanced feature extraction across multiple convolutional layers. Besides, a fusion strategy is improved based on the channel attention mechanism to highlight the relevant attributes in the fusion stage. The fusion method consists of four parts: encoder, feature pyramid networks, fusion strategy and decoder, respectively. First, the multi-scale deep features are extracted from the source images by encoder with embedded feature pyramid networks, realizing cross-layer interaction. Second, these features are fused by the improved fusion strategy with channel attention for each scale. Finally, the fused features are reconstructed by the designed decoder to produce the informative fused image. The experimental results show that the proposed fusion method achieves state-of-the-art results in both qualitative and quantitative evaluation with a lightweight architecture.