Dual Attention Mechanisms with Perceptual Loss Ensemble for Infrared and Visible Image Fusion

Abstract

Infrared and visible image fusion targets to combine the imaging advantages from different sensors into a single view, generating a more informative image to further serve for the subsequent computer vision tasks. Although various learning based infrared and visible image fusion methods have been proposed in recent years, the degeneration of intermediate feature maps in network is still hard to solve, leading to details loss and undesirable artifacts in the fused result. Additionally, the unsophisticated designed fusion rules also give rise to image distortion in the final result. In this paper, a novel deep learning architecture is proposed, by designing dual skip attention mechanisms and perceptual based fusion rule, together to solve afromentioned problems. Dual skip attention mechanisms make compensation for the information lost in feature extraction phase and reinforce different characteristics of the infrared and visible image. The perceptual based fusion rule could choose a more suitable way to fuse each channel from infrared and visible image. Extensive experimental results prove the superiority of the proposed method in comparison with the state-of-the-art methods quantitatively and qualitatively.