Spectral Super Resolution with DCT Decomposition and Deep Residual Learning

Abstract

AbstractIn this paper, we aim to reconstruct a hyperspectral image of multiple bands from an RGB image. This process of reconstruction of the hyperspectral image from RGB images is called Spectral Super Resolution (SSR). Using spectral super resolution, we can adapt the Dynamic Data Driven Applications System paradigm by using RGB cameras in surveillance instead of using hyperspectral cameras. This process is challenging because hyperspectral images have different information available in each band. There have been few works recently in SSR, most of them use a Convolutional Neural Network (CNN) to learn hyperspectral images from RGB image using a pixel wise loss function. The pixel wise loss function smooths the image which leads to loss of information in spectral bands. To overcome this in our work, we initially divide spectral bands into four subgroups and learn the hyperspectral image by learning the Discrete Coefficient Transform (DCT) coefficients of the hyperspectral image from RGB image using a residual dense network. Experiment results show our work using DCT based learning performs better than the state of the art HSCNN+ work [12].KeywordsSpectral super resolutionSuper resolutionDCT