Abstract
Infrared image retains typical thermal targets while visible image preserves rich texture details, image fusion aims to reconstruct a synthesized image containing prominent targets and abundant texture details. Most of deep learning-based methods mainly focus on convolution operation to extract the local features, but do not fully consider their multi-scale characteristics and global dependencies, which may cause loss of target regions and texture details in the fused image. Towards this goal, we present a unified multi-scale densely connected fusion network in this paper, named as UNFusion. We carefully design a multi-scale encoder-decoder architecture that can efficiently extract and reconstruct multi-scale deep features. Dense skip connections are employed in both encoder and decoder sub-networks to reuse all the intermediate features of different layers and scales for fusion tasks. In the fusion layer, <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$L_{p} $ </tex-math></inline-formula> normalized attention models, which include three kinds of different norms, are proposed to highlight and combine these deep features from spatial and channel dimensions, and the combined spatial and channel attention maps are used to reconstruct a final fused image. We conduct extensive experiments on the public TNO and Roadscene datasets, and the results demonstrate that our UNFusion can simultaneously preserve high brightness of typical thermal targets and abundant texture details to obtain superior scene representation and better visual perception. Besides, our UNFusion achieves better fusion performance and transcends other state-of-the-art methods in terms of qualitative and quantitative comparisons. Our code is available at <uri xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">https://github.com/Zhishe-Wang/UNFusion</uri> .
Talk to us
Join us for a 30 min session where you can share your feedback and ask us any queries you have
More From: IEEE Transactions on Circuits and Systems for Video Technology
Disclaimer: All third-party content on this website/platform is and will remain the property of their respective owners and is provided on "as is" basis without any warranties, express or implied. Use of third-party content does not indicate any affiliation, sponsorship with or endorsement by them. Any references to third-party content is to identify the corresponding services and shall be considered fair use under The CopyrightLaw.