Deep learning-based hyperspectral image reconstruction from emulated and real computed tomography imaging spectrometer data

Abstract

The computed tomography imaging spectrometer (CTIS) is a hyperspectral imaging (HSI) approach where spectral and spatial information of a scene is mixed during the imaging process onto a monochromatic sensor. This mixing is due to a diffractive optical element integrated into the underlying optics and creates a set of diffraction orders. To reconstruct a three-dimensional hyperspectral cube from the CTIS sensor image, iterative algorithms were applied. Unfortunately, such methods are highly sensitive to noise and require high computational time for reconstruction thus hindering their applicability in real-time and high frame-rate applications. To overcome such limitations, we propose a lightweight and efficient deep convolutional neural network for hyperspectral image reconstruction from CTIS sensor images. Compared with classical approaches our model delivers considerably better reconstruction results on synthetic as well as real CTIS images in under 0.17 s, which is over 60 times faster compared with the standard iterative approach. In addition, the reshaping method we have developed enables a lightweight network architecture with over 100 times fewer parameters than previously reported.