Deep Multiscale Feedback Network for Hyperspectral Image Fusion

Abstract

Hyperspectral imaging is useful in many remote sensing tasks. However, it is often challenging to obtain high-resolution images in both the spatial and spectral domains due to hardware limitations. Hyperspectral image fusion (HIF) solves this problem by fusing a low spatial resolution hyperspectral image (LR-HSI) and a high spatial resolution multispectral images (HR-MSI) to obtain a high spatial resolution hyperspectral image (HR-HSI). Many methods have been proposed for HIF, but few approaches have explored the multiscale mutual dependencies between LR-HSI, HR-MSI, and HR-HSI. This kind of mutual dependencies come from the fact that LR-HSI, HR-MSI, and HR-HSI capture the same scene with different spatial or spectral resolutions. To this end, we propose a deep multiscale feedback network (DMFBN) that iteratively learns image fusion and degeneration for HIF. We further equip the network with an error feedback mechanism coupled with multiscale feature learning. Both strategies help better learn the mutual dependencies. Extensive quantitative and qualitative evaluations on two public datasets show that the proposed method performs favorably against the state-of-the-art (SOTA) methods.