NF-3DLogTNN: An effective hyperspectral and multispectral image fusion method based on nonlocal low-fibered-rank regularization

Abstract

In this paper, we develop a framelet representation of the three-directional log-based tensor nuclear norm (F-3DLogTNN) for hyperspectral and multispectral image fusion (HSI-MSI fusion). The three-directional log-based tensor nuclear norm, as the nonconvex relaxation of tensor fibered rank, is computed by mode-n tensor singular value decomposition (t-SVD) based on the mode-n tensor-tensor product (t-prod). The mode-n t-prod is based on the mode-n discrete Fourier transform. Hereafter, we suggest using the mode-n framelet transform to define the mode-n t-prod and subsequently the mode-n t-SVD. Due to the redundancy of the framelet basis, each fiber along mode-n of the tensor can be sparsely represented. When the tensor’s slices (including frontal, lateral, and horizontal slices) are correlated, respectively, the corresponding sum of the rank of framelet transformed slices should be small. Thereby, there has lower tensor fibered rank performance. With that, we propose a novel nonlocal low-fibered-rank regularization to depict the local spatial-spectral correlation and nonlocal self-similarity of high-spatial-resolution hyperspectral image. Since minimizing the fibered rank directly is an NP-hard problem, we suggest F-3DLogTNN as its nonconvex relaxation. Subsequently, nonlocal based F-3DLogTNN (NF-3DLogTNN) method is developed for HSI-MSI fusion. To deal with the proposed model, we design an algorithm based on the alternating direction multipliers method. Experimental results on three datasets prove the proposed method’s superiority over the related state-of-the-art HSI-MSI fusion methods.