Fusion of near-infrared and RGB images on a FPGA using high level synthesis tool

Abstract

Several applications utilizing a set of red green blue (RGB) and near infrared (NIR) images have been emerging over recent years. The present work proposes a technique of enhancing an image by combining color (RGB) and near infrared information (NIR). In order to fuse the two types of images, the NIR-channel is considered as a luminance counterpart to the visible image. International standard database (RGB-NIR Scene Dataset) is used in this work for image fusion. The objective of the paper is to present a simple and hardware efficient fusion method, where the original RGB image is converted into two different color spaces, namely, HSV and YCbCr. Later, the luminance channel of the RGB image is replaced with the near infrared channel, thereby obtaining a fused enhanced image. The above procedure is effectively implemented on FPGA using the Xilinx HLS tool. RGB-NIR dataset is used in the present work for testing the proposed image fusion algorithm, and the quality of the fused image is measured through peak signal to noise ratio (PSNR). The experimental results indicate that HSV color space is more efficient in image fusion compared to YCbCr color space based on the average PSNR values of approximately 29db for HSV and 25db for YCbCr for various images, respectively. Finally, this complete fusion algorithm is implemented on Xilinx Nexys4 FPGA board to be able to obtain real-time outputs in the form of vivid, contrasted images that are pleasing to the observers. The experimental results illustrate that the Xilinx FPGA utilizes only 50% of the available hardware resources and consumes approximately 5.3 Watts to implement the fusion process.