Structure–Color Preserving Network for Hyperspectral Image Super-Resolution

Abstract

Fusion-based hyperspectral super-resolution (HSR) algorithms usually utilize a low-resolution hyperspectral image (LR-HSI) and a high-resolution multispectral image (MSI) to generate a high-resolution hyperspectral image (HR-HSI), which have attracted increasing attention in recent years. However, how to deal with the abundant spectral information of hyperspectral images and complex structure characteristics of MSIs has always been the focus and difficulty of fusion-based HSR. In this article, we propose a new structure–color preserving network (SCPNet) for HSR, which is developed under the basis of the joint attention mechanism. The SCPNet mainly includes three modules: structure-preserving module (SPM), color-preserving module (CPM), and cross-fusion module. The SPM is constructed based on the spatial attention, which aims to capture and enhance the significant structure information from the high-resolution MSI. Meanwhile, the CPM is constructed based on the channel attention, where the spectral characteristics in the LR-HSI are preserved during the reconstruction process. Finally, we propose a cross attention-based cross-fusion strategy to integrate the features from the two branches and reconstruct the final HR-HSI. The major contribution of SCPNet is that the structure and color information is described and preserved via the joint attention mechanism. Experimental results indicate that the proposed SCPNet has presented advantages on three benchmark datasets when compared with some state-of-the-art HSR methods.