UNIFusion: A Lightweight Unified Image Fusion Network

Abstract

Advanced image fusion frameworks have focused on constructing a unified system to deliver general fusion solutions for all the potential subtasks, including visible and infrared image fusion, multi-focus image fusion, and multi-exposure image fusion. However, the fusion performance of these unified solutions cannot be guaranteed as the involved subtasks exhibit essential diversity. Besides, the huge volume of these models also sacrifices their flexibility in practical applications. To this end, we propose a lightweight unified fusion network to balance the multi-level information featured across different channels and different layers. In particular, We construct a novel network architecture based on the Ghost module endowed with a new loss function. The depth of the designed network is increased, while the number of parameters is reduced by an order of magnitude compared to existing learning-based fusion approaches. In principle, our network belongs to the autoencoder paradigm, comprising three parts: encoder, fusion layer, and decoder, respectively. Considering that the efficacy of existing autoencoder-based approaches is impeded by the single fusion strategy, we use the guided filtering approach to decompose the source image into a base layer and a detail layer to extend the diversity of the inputs. Drawing on this, we can design different fusion strategies in these two layers for adapting different image fusion tasks. In the fusion layer, besides the three basic fusion strategies, i.e., average, max, and spatial attention, an additional gradient perception strategy is proposed to handle the multi-focus image fusion problem, improving the corresponding fusion performance. Qualitative and quantitative experiments show that our UNIFusion achieves promising results in these image fusion tasks, compared to the state of the art.