Self-supervised Hyperspectral and Multispectral Image Fusion in Deep Neural Network

Abstract

Hyperspectral and multispectral image fusion is to obtain high-resolution hyperspectral images from low-resolution hyperspectral images and high-resolution multispectral images. In recent years, many studies have applied deep learning methods to complete the fusion task. However, the function of deep learning-based methods is enslaved to the size and quality of training dataset, constraining the application of deep learning to the situation where training dataset is not available. In this paper, we introduce a new fusion algorithm, which operates in a self-supervised manner without training datasets. The proposed method obtains high-resolution hyperspectral images with the constraint of low-resolution hyperspectral image and a traditional fusion method. Several simulation and real-data experiments are conducted with remote sensing hyperspectral data under the condition where training datasets are unavailable. Quantitative and qualitative results indicates that the proposed method outperforms those traditional methods by a large extent.