Coupled Convolutional Neural Network-Based Detail Injection Method for Hyperspectral and Multispectral Image Fusion

Xiaochen Lu,Fengde Jia,Dezheng Yang,Yifeng Zhao

doi:10.3390/app11010288

Xiaochen Lu, Fengde Jia + Show 2 more

Open Access

https://doi.org/10.3390/app11010288

Copy DOI

Abstract

In this paper, a detail-injection method based on a coupled convolutional neural network (CNN) is proposed for hyperspectral (HS) and multispectral (MS) image fusion with the goal of enhancing the spatial resolution of HS images. Owing to the excellent performance in spectral fidelity of the detail-injection model and the image spatial–spectral feature exploration ability of CNN, the proposed method utilizes a couple of CNN networks as the feature extraction method and learns details from the HS and MS images individually. By appending an additional convolutional layer, both the extracted features of two images are concatenated to predict the missing details of the anticipated HS image. Experiments on simulated and real HS and MS data show that compared with some state-of-the-art HS and MS image fusion methods, our proposed method achieves better fusion results, provides excellent spectrum preservation ability, and is easy to implement.

Highlights

Hyperspectral (HS) imagery presents plentiful spectral details and allows for accurate analyses of terrestrial features due to its high spectral resolution
In this paper, we propose to utilize convolutional neural network (CNN) to automatically learn the spatial details from HS and MS images themselves to promote the performance of image fusion
The experiments were conducted under different resolution ratios, i.e., the HS image was down-sampled by the facts of 2, 4, and 6 (i.e., s = 2, 4, and 6), respectively, as Section 3.1 mentioned

Summary

Introduction

Hyperspectral (HS) imagery presents plentiful spectral details and allows for accurate analyses of terrestrial features due to its high spectral resolution. By employing image fusion techniques, HS and high-resolution multispectral (MS) images have a possibility to produce enhanced HS data that would contribute to the accurate identification and classification of land-covers observed at a finer ground resolution. For this purpose, hyper-sharpening [1], i.e., fusion of HS and MS images, has attracted considerable concern over the past decade. In [5], the maximum a posteriori (MAP) estimation is presented for fusing HS data with an auxiliary high-resolution image, which can be an MS or panchromatic (PAN) image. A fast fusion algorithm for multi-band images is clarified in [8], applying a Sylvester equation

Methods

Results

Discussion

Conclusion