An Unsupervised Laplacian Pyramid Network for Radiometrically Accurate Data Fusion of Hyperspectral and Multispectral Imagery

Abstract

Improving the spatial resolution of hyperspectral images (HSIs) has traditionally been an important topic in the field of remote sensing. Many approaches have been proposed based on various theories, including component substitution, multiresolution analysis, spectral unmixing, Bayesian probability, and tensor representation. However, these methods have some common disadvantages such that their performance degrades dramatically as the up-scale ratio increases, and they have little concern for the per-pixel radiometric accuracy of the sharpened image. Moreover, many learning-based methods have been proposed through decades of innovations, but most of them require a large set of training pairs, which is unpractical for many real problems. To solve these problems, we propose a stable hyperspectral sharpening method based on the Laplacian pyramid and the generative convolutional neural network (CNN), which achieves superior radiometric accuracy of the sharpened data in different up-scale ratios based on one single input pair. First, with a low-resolution HSI (LR-HSI) and high-resolution multispectral image (HR-MSI) pair, the preliminary high-resolution HSI (HR-HSI) is calculated via linear regression. Then, the high-frequency details of the preliminary HR-HSI are estimated via the subtraction between it and the CNN-generated-blurry version. By injecting the details to the output of the generative CNN with the LR-HSI as input, the final HR-HSI is obtained. Nine different state-of-the-art sharpening methods are chosen as our baselines, and three different datasets with different scene content are tested. Furthermore, the target detection method, the adaptive coherence estimator (ACE), is conducted on the reconstructed HR-HSI to evaluate the per-pixel radiometric accuracy. The results demonstrate that the proposed method has the best and the most stable performance in terms of spectral and spatial accuracies.