Hyperspectral and Multispectral Data Fusion via Joint Local-Nonlocal Modeling and Truncation Operator

Abstract

Fusion technology has been the core strategy to obtain a high-spatial-spectral resolution hyperspectral image (HSI). In recent years, few fusion models focused on exploiting the underlying manifold structure in the spatial dimension of the high-resolution HSI. We assume that image patches of high-resolution multi-spectral image (HR-MSI) and high-resolution hyperspectral image (HR-HSI) share similar manifold structures. Through this bridge, a local-nonlocal low dimensional manifold defined from HR-MSI is built to favor the patch relationship in HR-HSI, which is further utilized to build a set of orthogonal bases. Subsequently, we introduce a truncation operation based on the adaptively constructed orthogonal bases, which can efficiently preserve the low-frequency information of the image and reduce the interference of noise. Finally, we combine the local-nonlocal manifold term and the truncation operation, coined as LNTM, into a variational super-resolution framework to regularize the latent HR-HSI, leading to a strongly convex function regarding the fusion problem. The cost function is solved by a carefully designed variant of alternating direction method of multipliers (ADMM) algorithm. Different experiments on three public benchmarks validate our algorithm outperforms the recent start-of-the-arts concerning both visual quality and numerical metrics.