Spectral Reflectance Recovery From a Single RGB Image

Abstract

The spectral reflectance is an intrinsic and discriminative characteristic of object materials and can be obtained by hyperspectral imaging. However, existing hyperspectral cameras are limited in low-spatial/temporal resolution while yet being complicated and expensive. In this paper, we present a nonnegative sparse representation based method to recover high-quality spectral reflectance from a single RGB image. Unlike previous methods, our approach learns multiple nonnegative sparse coding dictionaries from the training spectral dataset in terms of clustering results. Then, the spectral reflectance of the input RGB image is recovered based on nonnegative sparse representation, which also considers the spatial structured similarity and high correlation across spectra under the learned dictionaries. Furthermore, the illumination spectrum can be estimated with the recovered spectral reflectance under the known RGB camera spectral sensitivity. Experimental results show that the proposed method outperforms state-of-the-art spectral reflectance recovery methods in terms of both objective metrics and subjective visual quality. Besides, we show an application of our method to accurately relight scenes under the novel illumination.