Hyperspectral Image Super-Resolution with Self-Supervised Spectral-Spatial Residual Network

Wenjing Chen,Xiangtao Zheng,Xiaoqiang Lu

doi:10.3390/rs13071260

Abstract

Recently, many convolutional networks have been built to fuse a low spatial resolution (LR) hyperspectral image (HSI) and a high spatial resolution (HR) multispectral image (MSI) to obtain HR HSIs. However, most deep learning-based methods are supervised methods, which require sufficient HR HSIs for supervised training. Collecting plenty of HR HSIs is laborious and time-consuming. In this paper, a self-supervised spectral-spatial residual network (SSRN) is proposed to alleviate dependence on a mass of HR HSIs. In SSRN, the fusion of HR MSIs and LR HSIs is considered a pixel-wise spectral mapping problem. Firstly, this paper assumes that the spectral mapping between HR MSIs and HR HSIs can be approximated by the spectral mapping between LR MSIs (derived from HR MSIs) and LR HSIs. Secondly, the spectral mapping between LR MSIs and LR HSIs is explored by SSRN. Finally, a self-supervised fine-tuning strategy is proposed to transfer the learned spectral mapping to generate HR HSIs. SSRN does not require HR HSIs as the supervised information in training. Simulated and real hyperspectral databases are utilized to verify the performance of SSRN.

Highlights

Hyperspectral imaging sensors collect hyperspectral images (HSIs) across many narrow spectral wavelengths, which contain rich physical properties of observed scenes [1].HSIs with high spectral resolution are beneficial for various tasks, e.g., classification [2] and detection [3]
This subsection directly uses peak signal-to-noise ratio (PSNR) and spectral angle mapper (SAM) to measure the quality of the reconstructed high spatial resolution (HR) HSI
coupled nonnegative matrix factorization (CNMF), generalization of the simultaneous orthogonal matching pursuit (GSOMP), HySure, and USDN usually rely on the assumption that the intrinsic spectral signatures of the same land-cover in HR multispectral image (MSI) and low spatial resolution (LR) HSIs are the same [28,36]

Summary

Introduction

Hyperspectral imaging sensors collect hyperspectral images (HSIs) across many narrow spectral wavelengths, which contain rich physical properties of observed scenes [1].HSIs with high spectral resolution are beneficial for various tasks, e.g., classification [2] and detection [3]. Many methods [8,9] have been proposed to reconstruct the desired HR HSI by fusing HR MSIs and LR HSIs, including sparse representation-based methods [10,11], Bayesian-based methods [12,13], spectral unmixing-based methods [1,14], and tensor factorization-based methods [15,16]. The desired HR HSI is reconstructed using the learned spectral bases and sparse codes. These methods usually treat the HR MSI and LR HSI as 2-D matrices, which result in the spatial structure information of HR MSIs and LR HSIs not being effectively exploited [15].

Methods

Results

Discussion

Conclusion