Combining global receptive field and spatial spectral information for single-image hyperspectral super-resolution

Abstract

Single-image hyperspectral super-resolution has poorer reconstruction performance in the spatial dimension than fused-image hyperspectral super-resolution due to the lack of auxiliary images. Some studies have attempted to use 3D convolution to explore hidden features between spatial spectra to enhance spatial details. However, either 2D or 3D convolution, the obtained receptive fields are limited, ignoring the effect of global spatial information on hyperspectral image reconstruction, and cannot be used for long-range dependent modeling. Therefore, we make the first attempt to combine Transformer with 3D convolution in single-image hyperspectral super-resolution and propose a 3D convolution and Transformer hyperspectral super-resolution (3D-THSR) network, which explores the hidden information between space and spectra while obtaining the global receptive field of space. Specifically, the Transformer module is used for feature extraction to enhance the learning ability of global spatial information and long-distance features. In addition, the 3D convolution module is embedded in the Transformer module to extract the information between different spectral bands by fusing the spectral and spatial dimensions. Finally, we design to train the network by using three loss functions to reduce the distortion spectrum and ensure the spectral band purity. Compared with other single-image hyperspectral methods by six hyperspectral evaluation metrics, spatial detail image, spectral error line map, and ablation study, it is proved that the proposed method achieves better hyperspectral super-resolution reconstruction performance.