Versatile, intelligent multispectral imaging camera made with off-the-shelf components

Abstract

Multispectral imagers collect a hypercube of data, where the spatial image is along two-dimensions and the spectral information is in the third. Two main technologies are used for multispectral imaging: sweeping, where the hypercube is built by scanning through different wavelengths or spatial positions and snapshot multispectral spectral imaging, where the 3D cube of images is taken in one shot. Sweeping imaging systems tend to have more lines and better spectral resolutions whilst snapshot cameras are often used for dynamic analysis of scenes. A common method to obtain the hypercube in snapshot imagers is by pixel level filtering on the sensor chip. Pixel level filtering, where the filter is placed directly on the pixels are intergrated into the wafer-level making processing making them difficult to customize. Therefore, these sensors tend to aim for equally spaced spectral lines in order to cover many applications. This results in an often in an unnecessarily large data cube when only a few spectral lines are needed, moreover the spectral lines are not adapted to the specific application. In this work we propose a multispectral camera based on plenoptic imaging, where the filtering is done in a front-end optics module. Our camera has the usual advantages of a snapshot imager, and the added advantage that the spectral lines can be both reduced and tailored to the specific application by customizing the filter. This procedure reduces the hypercube whilst keeping performance by selecting the relevant data. Moreover, the filter is interchangeable for different applications The camera presented here is built with off-the-shelf components, shows >40 spectral channels, image sizes are 260x260 pixels, with pixel limited spatial resolution. We demonstrate this technology by fruit quality control using machine learning algorithms.