Long Range Imaging Using Multispectral Fusion of RGB and NIR Images

Abstract

Unlike RGB images, NIR images are robust to atmospheric environments such as Rayleigh scattering. In this paper, we propose long-range imaging using multispectral fusion of RGB and NIR images. The proposed network consists of feature extraction, feature fusion, and reconstruction. For feature extraction, we adopt pyramid feature selection to capture multi-scale information. For feature fusion, we present a learnable attention based fusion block (AFB) to refine and fuse latent features of RGB and NIR images and a multi-scale fusion block (MFB) to integrate the fused feature in multiple scales. For reconstruction, we use three convolutions corresponding to the encoder to reconstruct the fusion image. Besides, we perform image registration based on scale-invariant feature transform (SIFT) to align NIR image referring to RGB image. Experimental results show that the proposed network successfully recovers hidden textures lost in RGB images while keeping colors and outperforms state-of-the-art fusion methods in terms of both visual quality and quantitative measurements.