Scale-Aware Multispectral Fusion of RGB and NIR Images Based on Alternating Guidance

Kailong Zhou,Shengtao Yu,Cheolkon Jung

doi:10.1109/access.2020.3025154

Abstract

In low light condition, color (RGB) images captured by increasing the camera ISO contain much noise and detail loss. However, near infrared (NIR) images are robust to noise and have clear textures without color. In this paper, we propose scale-aware multispectral fusion of RGB and NIR images based on alternating guidance. Low light RGB images provide large-scale image structure and color information, while NIR images have fine details lost in RGB images. Since they are complementary, we adopt alternating guidance for the fusion of them using weighted least squares (WLS). First, we perform the first guidance to denoise the RGB image and obtain base layer. Then, we conduct the second guidance for scale-aware detail transfer of the NIR image and yield detail layer. Finally, we combine the base and detail layers to generate a fusion image. We maximize the multispectral advantage of RGB and NIR images for fusion based on alternating guidance. Experimental results show that the proposed method achieves good performance in noise reduction, detail transfer and color reproduction, and is superior to the state-of-the-art ones in terms of quantitative measurement and computational efficiency.

Highlights

With advances in the sensor technology, the image types are highly diversified
We focus on the multispectral fusion of RGB and near infrared (NIR) images in low light condition
CONTRIBUTIONS In this paper, we propose scale-aware fusion of RGB and NIR images in low light condition based on alternating guidance

Summary

Introduction

With advances in the sensor technology, the image types are highly diversified. In addition to the widely used visible RGB cameras, there exist depth cameras to record depth information, infrared (IR) and NIR cameras for invisible wavelength band imaging, and X-ray cameras for medical imaging. Due to the increasing demands for computer vision applications, the requirements for imaging quality are becoming higher and higher. To maximize the advantages of various sensors, image fusion uses multiple sensors to improve imaging quality and accuracy of vision applications [20], [28]. Recent image fusion methods include fusion of flash/ no-flash images [19], infrared/RGB images [17], near infrared (NIR)/RGB images [13], [21], [34]. We focus on the multispectral fusion of RGB and NIR images in low light condition

Methods

Results

Conclusion