3D Separable Convolution based Super-Resolution of Hyperspectral Images using CNN

Abstract

Recently CNN based Super-Resolution methods have seen rise in popularity in solving the problem ill posed problem of Hyperspectral (HS) Image super resolution. One major factor for this popularity is that CNN based super-resolution methods have been developed to improve the performance and eliminate spectral distortion occurred due to sparse coding based methods which involve fusion of HS image and normal RGB image. Even though these Deep Learning based methods have outperformed the classic sparse coding methods for HS image super resolution, they lag behind in proper exploration of spectral features of the image and at the same time the number of training parameters used are very high, which eventually costs valuable time and computational resources. In order to eliminate these drawbacks, in this paper we propose a 3D Separable Convolution method to address two major tasks at hand. First is to reduce the number of parameters in the network and efficiently reuse the parameters for reducing the training time and resources, which is achieved by employing the 3D Separable Networks comprising of separable filters inside the Deep Feature Extraction subnetwork whose operational analysis is elaborately explained in the upcoming sections. Second, to efficiently use the strong correlation between spectral and spatial features and of the HS image and is performed by the 2D network which makes use of the feature maps obtained by the 3D Separable network to spatially convolve along the feature dimensions of the network extracted by the 3D filters. The experimental results show that the proposed model has achieved reduction in number of training parameters from 19lakhs to 17lakhs compared to the State-of-the-Art (SOTA) method. The proposed method has outperformed the SOTA methods with 5dB improvement in PSNR along with 0.9929 SSIM and 1.668°SAM.