Fusion of Hyperspectral-Multispectral images joining Spatial-Spectral Dual-Dictionary and structured sparse Low-rank representation

Abstract

High spatial resolution hyperspectral images (HR-HSIs) have shown considerable potential in urban green infrastructure monitoring. A prevalent scheme to overcome spatial resolution limitations in HSIs is by fusing low-resolution hyperspectral images (LR-HSIs) and high-resolution multispectral images (HR-MSIs). Existing methods considering the spectral dictionary or spatial dictionary can only reflect the unilateral characteristics of the HSI and cannot completely restore full information in the latent HSI. To overcome this issue, we propose a novel HSI-MSI fusion method, named DDSSLR, which joins spatial-spectral dual-dictionary and structured sparse low-rank representation. The spectral dictionary characterizing generalized spectra and the corresponding spectral sparse coefficients are extracted from LR-HSI and HR-MSI, while sparse low-rank priors of the local structure are imposed on the spectral pixels within the same superpixel in HR-MSI. Additionally, in the spatial domain, we exploit the remaining high-frequency components to learn the spatial dictionary and use the unitary transformation to factorize the spatial sparse coefficient into the sparse low-rank matrix in subspace, establishing the relationship between low-rank and sparse. We formulate the two fusion models as variational optimization problems, which are effectively solved by the alternating direction methods of multipliers (ADMM). Experiments on three HSI datasets show that DDSSLR achieves state-of-the-art performance.