Cascaded Convolutional Neural Network-Based Hyperspectral Image Resolution Enhancement via an Auxiliary Panchromatic Image.

Abstract

Owing to the limits of incident energy and hardware system, hyperspectral (HS) images always suffer from low spatial resolution, compared with multispectral (MS) or panchromatic (PAN) images. Therefore, image fusion has emerged as a useful technology that is able to combine the characteristics of high spectral and spatial resolutions of HS and PAN/MS images. In this paper, a novel HS and PAN image fusion method based on convolutional neural network (CNN) is proposed. The proposed method incorporates the ideas of both hyper-sharpening and MS pan-sharpening techniques, thereby employing a two-stage cascaded CNN to reconstruct the anticipated high-resolution HS image. Technically, the proposed CNN architecture consists of two sub-networks, the detail injection sub-network and unmixing sub-network. The former aims at producing a latent high-resolution MS image, whereas the latter estimates the desired high-resolution abundance maps by exploring the spatial and spectral information of both HS and MS images. Moreover, two model-training fashions are presented in this paper for the sake of effectively training our network. Experiments on simulated and real remote sensing data demonstrate that the proposed method can improve the spatial resolution and spectral fidelity of HS image, and achieve better performance than some state-of-the-art HS pan-sharpening algorithms.