Evaluating the Performance of Different Cameras for Spectral Reconstruction

Abstract

Spectral Reconstruction (SR) algorithms seek to map RGB images to their hyperspectral image counterparts. Statistical methods such as regression, sparse coding, and deep neural networks are used to determine the SR mapping. All these algorithms are optimized ‘blindly’ and the provenance of the RGBs is not considered. In this paper, we benchmark the performance of SR methods—in order of increasing complexity: regression, sparse coding, and deep neural network—when different RGB camera spectral sensitivity functions are used. In effect, we ask: “Are some cameras better able to recover spectra from RGBs than others?”. In our experiments, RGB images are generated by numerical integration for a fixed set of hyperspectral images using 9 different camera response functions (each from a different camera manufacturer) plus the CIE 1964 color matching functions. Then, we train SR methods on the respective RGB image sets. Our experiments show three important results. First, different cameras <strong>do</strong> support slightly better or worse spectral reconstruction but, secondly, that changing the spectral sensitivities alone does not change the ranking of different algorithms. Finally, we show that sometimes switching the used camera for SR can give a greater performance boost than switching to use a more complex SR method.