Hyperspectral image super‐resolution under misaligned hybrid camera system

Abstract

Hyperspectral imaging has been widely used for agriculture, astronomy, surveillance, and so on. However, hyperspectral imaging usually suffers from low-spatial resolution, due to the limited photons in individual bands. Recently, more hyperspectral image super-resolution methods have been developed by fusing the low-resolution hyperspectral image and high-resolution RGB image, but most of them did not consider the misalignment between two input images. In this study, the authors present an effective method to restore a high-resolution hyperspectral image from the misaligned low-resolution hyperspectral image and high-resolution RGB image, which exploits spectral and spatial correlation in hyperspectral and RGB images. Specifically, they employ the spectral sparsity to restore the high-resolution hyperspectral image on the misaligned part, and then simultaneously employ spectral and spatial structure correlation to restore the high-resolution hyperspectral image on the aligned area, which can be fused to obtain the high-quality hyperspectral image restoration under a misaligned hybrid camera system. Experimental results show that the proposed method outperforms the state-of-the-art hyperspectral image super-resolution methods under a misaligned hybrid camera system in terms of both objective metric and subjective visual quality.