SHS-GAN: Synthetic Enhancement of a Natural Hyperspectral Database

Abstract

Deep Learning frameworks are gaining increased popularity in image processing tasks such as computational hyperspectral imaging. While these frameworks achieve state-of-the-art results in terms of reconstruction quality and run time, they often require massive databases of hyperspectral cubes for training the reconstruction algorithms. Unfortunately, such databases are usually hard to acquire due to complexity and cost considerations. To mitigate these challenges, we propose a method for generating a synthetic database of hyperspectral cubes in the visible range using a limited number of natural hyperspectral cubes, an unlimited number of RGB images, and a Generative Adversarial Network model. The suggested algorithm, dubbed SHS-GAN, is trained to get a query RGB image and to output a synthetic hyperspectral cube. While the spectral domain of the synthetic hyperspectral cube shares similar statistical properties as the natural hyperspectral cubes used in the training process, the SHS-GAN is trained to preserve the spatial characteristics of the query RGB image, whereas the R, G, B values provide an additional constraint along with the spectral domain. Our suggested framework was utilized for performing Snapshot Spectral Imaging (SSI) from a single monochromatic dispersed and diffused snapshot using the DD-Net reconstruction neural network. We demonstrate, by simulations and lab experiments, that enhancing the training database with synthetic data from the SHS-GAN improves the reconstruction quality of the hyperspectral cube. In addition, we share a new original database of more than 10,000 hyperspectral cubes of real objects of size 256x256x29 in the 420-700 nm visible range.