General Hyperspectral Image Super-Resolution via Meta-Transfer Learning.

Abstract

Recent advances in deep learning-based methods have led to significant progress in the hyperspectral super-resolution (SR). However, the scarcity and the high dimension of data have hindered further development since deep models require sufficient data to learn stable patterns. Moreover, the huge domain differences between hyperspectral image (HSI) datasets pose a significant challenge in generalizability. To address these problems, we present a general hyperspectral SR framework via meta-transfer learning (MTL). We randomly sample various spectral ranges for SR tasks during MTL, allowing the model to accumulate diverse task experiences. Additionally, we implement a task schedule to gradually expand the number of bands, bridging the significant domain differences between datasets. By leveraging multiple datasets, we are able to achieve better performance and greater generalizability, making it applicable under various circumstances. Meanwhile, as a general framework, our scheme can be applied to existing methods to obtain performance improvements. In addition, we design an advanced network architecture based on the multifusion features to further improve the performance. Experiments demonstrate that our method not only achieves superior performance in both qualitative and quantitative terms but also can adapt robustly to a new and difficult sample, where few epochs can yield quite considerable results.