Spectral image recovery from Spectral Filter Array cameras using LMMSE

Abstract

A hyperspectral camera can record a cube of data with both spatial 2D and spectral 1D dimensions. Spectral Filter Arrays (SFAs) overlaid on a single sensor allows a snapshot version of a hyperspectral camera. But acquired image is subsampled both spatially and spectrally, and a recovery method should be applied. In this paper we present a linear model of spectral and spatial recovery based on Linear Minimum Mean Square Error (LMMSE) approach. The method learns a stable linear solution for which redundancy is controlled using spatial neighborhood. We evaluate results in simulation using gaussian shaped filter's sensitivities on SFA mosaics of upto 9 filters with sensitivities both in visible and Near-Infrared (NIR) wavelength. We show by experiment that by using big neighborhood sizes in our model we can accurately recover the spectra from the RAW images taken by such a camera. We also present results on recovered spectra of Macbeth color chart from a Bayer SFA having 3 filters.