Single-Sensor RGB and NIR Image Acquisition: Toward Optimal Performance by Taking Account of CFA Pattern, Demosaicking, and Color Correction

Abstract

In recent years, many applications using a pair of RGB and near-infrared (NIR) images have been proposed in computer vision and image processing communities. Thanks to recent progress of image sensor technology, it is also becoming possible to manufacture an image sensor with a novel spectral filter array, which has RGB plus NIR pixels for one-shot acquisition of the RGB and the NIR images. In such a novel filter array, half of the G pixels in the standard Bayer color filter array (CFA) are typically replaced with the NIR pixels. However, its performance has not fully been investigated in the pipeline of single-sensor RGB and NIR image acquisition. In this paper, we present an imaging pipeline of the single-sensor RGB and NIR image acquisition and investigate its optimal performance by taking account of the filter array pattern, demosaicking and color correction. We also propose two types of filter array patterns and demosaicking algorithms for improving the quality of acquired RGB and NIR images. Based on the imaging pipeline we present, the performance of different filter array patterns and demosaicking algorithms is evaluated. In experimental results, we demonstrate that our proposed filter array patterns and demosaicking algorithms outperform the existing ones.