Infrared and visible image fusion using multi-scale pyramid network

Abstract

Deep networks have been widely applied in infrared and visible image fusion. However, the current deep networks cannot well extract and fuse multi-scale information and high-frequency texture features of the source images. In this paper, a deep multi-scale pyramid network, termed MSPFNet, is proposed for infrared and visible image fusion by combining image Laplacian pyramid and deep network. Infrared and visible images are first decomposed into their Laplacian pyramids. For each source image, its Laplacian pyramid consists of a low-frequency component and a series of multi-scale high-frequency components containing texture details. Then, the Laplacian pyramid components of two source images in the same level are fused using convolutional neural networks (CNN). Finally, the final fused image is reconstructed on the fused Laplacian pyramid components using inverse Laplacian pyramid transform. The experimental results on publicly available datasets show that MSPFNet can efficiently extract and fuse the multi-scale detail information of source images, and the fused images of MSPFNet preserve more texture details of infrared and visible images than that of the previous state-of-the-art methods.