Abstract
Super-resolution (SR) is significant for hyperspectral image (HSI) applications. In single-frame HSI SR, how to reconstruct detailed image structures in high resolution (HR) HSI is challenging since there is no auxiliary image (e.g., HR multispectral image) providing structural information. Wavelet could capture image structures in different orientations, and emphasis on predicting high-frequency wavelet sub-bands is helpful for recovering the detailed structures in HSI SR. In this study, we propose a multi-scale wavelet 3D convolutional neural network (MW-3D-CNN) for HSI SR, which predicts the wavelet coefficients of HR HSI rather than directly reconstructing the HR HSI. To exploit the correlation in the spectral and spatial domains, the MW-3D-CNN is built with 3D convolutional layers. An embedding subnet and a predicting subnet constitute the MW-3D-CNN, the embedding subnet extracts deep spatial-spectral features from the low resolution (LR) HSI and represents the LR HSI as a set of feature cubes. The feature cubes are then fed to the predicting subnet. There are multiple output branches in the predicting subnet, each of which corresponds to one wavelet sub-band and predicts the wavelet coefficients of HR HSI. The HR HSI can be obtained by applying inverse wavelet transform to the predicted wavelet coefficients. In the training stage, we propose to train the MW-3D-CNN with L1 norm loss, which is more suitable than the conventional L2 norm loss for penalizing the errors in different wavelet sub-bands. Experiments on both simulated and real spaceborne HSI demonstrate that the proposed algorithm is competitive with other state-of-the-art HSI SR methods.
Highlights
Hyperspectral image (HSI) is collected in contiguous bands over a certain electromagnetic spectrum, and the spectral and spatial information in HSI is helpful for identifying and discriminating different materials in the scene
The authors in [30] proposed a spectral difference convolutional network (SDCNN) to learn the mapping of spectral differences between the low resolution (LR) and high resolution (HR) HSIs, and the SDCNN could be further integrated with a spatial error correction model to rectify the artifacts of HR HSI [31]. 3D convolution could exploit the spectral-spatial correlation in HSI, a 3D convolutional neural network (CNN) based HSI SR method was proposed in [32], where the mapping between the LR and HR HSIs was represented by 3D CNN
We propose a single-frame HSI SR method based on multi-scale wavelet 3D CNN (MW-3D-CNN) that predicts the wavelet package coefficients of the latent HR HSI, rather than directly inferring the HR HSI
Summary
Hyperspectral image (HSI) is collected in contiguous bands over a certain electromagnetic spectrum, and the spectral and spatial information in HSI is helpful for identifying and discriminating different materials in the scene. HR HSI can be reconstructed with the endmembers of LR HSI and the abundance of MSI Utilizing this idea, several unmixing based fusion methods have been proposed. 3D convolution could exploit the spectral-spatial correlation in HSI, a 3D CNN based HSI SR method was proposed in [32], where the mapping between the LR and HR HSIs was represented by 3D CNN. Training a deep learning network that predicts the wavelet coefficients, the high-frequency wavelet sub-bands, would encourage the network to produce more structural details in the image SR problem [33,34,35,36]. Unlike the previous deep learning models that reconstruct HR HSI directly [29,30,31,32], the proposed network predicts the wavelet coefficients of the latent HR HSI, which is beneficial for reconstructing detailed textures in HSI.
Sign-in/Register to view highlights & summary Sign-in/Register to access full text options 
Free) 
Talk to us
Join us for a 30 min session where you can share your feedback and ask us any queries you have
Schedule a call
