Attention Network with Outdoor Illumination Variation Prior for Spectral Reconstruction from RGB Images

Abstract

Hyperspectral images (HSIs) are widely used to identify and characterize objects in scenes of interest, but they are associated with high acquisition costs and low spatial resolutions. With the development of deep learning, HSI reconstruction from low-cost and high-spatial-resolution RGB images has attracted widespread attention. It is an inexpensive way to obtain HSIs via the spectral reconstruction (SR) of RGB data. However, due to a lack of consideration of outdoor solar illumination variation in existing reconstruction methods, the accuracy of outdoor SR remains limited. In this paper, we present an attention neural network based on an adaptive weighted attention network (AWAN), which considers outdoor solar illumination variation by prior illumination information being introduced into the network through a basic 2D block. To verify our network, we conduct experiments on our Variational Illumination Hyperspectral (VIHS) dataset, which is composed of natural HSIs and corresponding RGB and illumination data. The raw HSIs are taken on a portable HS camera, and RGB images are resampled directly from the corresponding HSIs, which are not affected by illumination under CIE-1964 Standard Illuminant. Illumination data are acquired with an outdoor illumination measuring device (IMD). Compared to other methods and the reconstructed results not considering solar illumination variation, our reconstruction results have higher accuracy and perform well in similarity evaluations and classifications using supervised and unsupervised methods.