Deep Spatial–Spectral Representation Learning for Hyperspectral Image Denoising

Abstract

Deep learning has found successful applications in restoration of two-dimensional (2-D) images including denoising, dehazing, and superresolution. However, existing deep convolutional neural network (DCNN) architecture cannot fully exploit spatial–spectral correlations in three-dimensional (3-D) hyperspectral images (HSIs) (directly extending 2-D DCNN into 3-D will significantly increase computational complexity); meantime, unlike 2-D images, there is an obstacle caused by the shortage of training data for HSIs. To meet those challenges, we present a novel, deep-learning framework for 3-D HSI denoising with the following contributions. First, inspired by the success of U-net in low-dose current-transformer denoising, we propose a novel approach of encoding rich multi-scale information of HSIs by a modified 3-D U-net. Second, we present a computationally efficient implementation of 3-D U-net based on the strategy of separable filtering. By decomposing 3-D filtering into 2-D spatial filtering and 1-D spectral filtering, we can achieve substantial savings on the number of network parameters to keep the computational complexity low. Third, we have developed a transfer learning approach of synthetically generating HSIs from RGB images as supplementary training data. The synthesized HSIs are used for the initial training of the modified 3-D U-net denoising network, which will be fine-tuned on real HSI images. Experimental results have shown that the proposed 3-D U-net denoising method significantly outperforms existing model-based HSI denoising methods.