Abstract
Spectral reconstruction algorithms recover spectra from RGB sensor responses. Recent methods—with the very best algorithms using deep learning—can already solve this problem with good spectral accuracy. However, the recovered spectra are physically incorrect in that they do not induce the RGBs from which they are recovered. Moreover, if the exposure of the RGB image changes then the recovery performance often degrades significantly—i.e., most contemporary methods only work for a fixed exposure. In this paper, we develop a physically accurate recovery method: the spectra we recover provably induce the same RGBs. Key to our approach is the idea that the set of spectra that integrate to the same RGB can be expressed as the sum of a unique fundamental metamer (spanned by the camera’s spectral sensitivities and linearly related to the RGB) and a linear combination of a vector space of metameric blacks (orthogonal to the spectral sensitivities). Physically plausible spectral recovery resorts to finding a spectrum that adheres to the fundamental metamer plus metameric black decomposition. To further ensure spectral recovery that is robust to changes in exposure, we incorporate exposure changes in the training stage of the developed method. In experiments we evaluate how well the methods recover spectra and predict the actual RGBs and RGBs under different viewing conditions (changing illuminations and/or cameras). The results show that our method generally improves the state-of-the-art spectral recovery (with more stabilized performance when exposure varies) and provides zero colorimetric error. Moreover, our method significantly improves the color fidelity under different viewing conditions, with up to a 60% reduction in some cases.
Highlights
Hyperspectral imaging devices are developed to capture scene radiance spectra at high spectral resolution
We present experiments which demonstrate that a physically plausible spectral recovery results in better cross-viewing-condition color prediction (Figure 4 shows an example of the cross-illumination color fidelity result when using our physically plausible approach)
Providing some motivation for the approach we develop in this paper, there were already studies that used the physics of image formation to improve spectral reconstruction
Summary
Hyperspectral imaging devices are developed to capture scene radiance spectra at high spectral resolution. For fast and less costly alternatives, using compressed sensing, the spatial and spectral information is jointly encoded in the captured 2D images and decoded by specialized algorithms [17,18,19,20,21,22,23]. Most of these approaches use learning algorithms to solve for the complex and ill-posed decompression. Maloney and Wandell [38] represented reflectances using a 3-dimensional linear model With respect to this model the spectra are related to RGBs by a simple 3 × 3 matrix transform. Long as the model has four or more degrees of freedom, we can always find (e.g., using “singlar value decomposition”, referring to pp. 382–391 in [60]), one or more axes in the spectral space that are orthogonal to the spectrum-to-RGB projection
Talk to us
Join us for a 30 min session where you can share your feedback and ask us any queries you have
Disclaimer: All third-party content on this website/platform is and will remain the property of their respective owners and is provided on "as is" basis without any warranties, express or implied. Use of third-party content does not indicate any affiliation, sponsorship with or endorsement by them. Any references to third-party content is to identify the corresponding services and shall be considered fair use under The CopyrightLaw.