Multispectral demosaicking considering out-of-focus problem for red-green-blue-near-infrared image sensors

Abstract

A near-infrared (NIR) band provides information invisible to human eyes for discriminating and recognizing objects more clearly under low lighting conditions. To capture color and NIR images simultaneously, a multispectral filter array (MSFA) sensor is used. However, because lenses have different refractive indices for different wavelengths, lenses may fail to focus all rays to the same convergence. This is the reason an out-of-focus problem occurs and images are blurred. In this paper, a demosaicking algorithm that considers the out-of-focus problem is proposed. This algorithm is used by the MSFA of a red-green-blue-NIR image sensor to obtain color and NIR images. After the energies of the multispectral (MS) channels in the MSFA image are balanced to minimize aliasing, that image is filtered by the estimated low-pass kernel to generate a panchromatic (PAN) image. When an image is acquired, the out-of-focus problem and the formation process of the PAN image are modeled. The desired MS image is estimated by solving the least squares approach of the difference between the PAN and MS images based on the models. The experimental results show that the proposed algorithm performs well in estimating high-quality MS images and reduces the out-of-focus problem.