Extending function mixture network for improved spectral super-resolution

Abstract

Hyperspectral Image reconstruction from RGB images is a low-cost and convenient alternative to acquiring hyperspectral images directly. The challenge in estimating the spectral response function and using it for generating the hyperspectral image data is addressed effectively by the use of convolutional neural networks for the task. The accuracy of the reconstruction techniques is improving and effective architectures are being adapted for the purpose. Specifically, in a recent work the authors propose the use of Function Mixture Network, to model the mapping from RGB to hyperspectral. This technique benefits from the use of an adaptive spatial receptive field, however, given that the goal is spectral superresolution additional use of adaptive spectral field is prudent. Accordingly, in this paper we propose an improved Function Mixture Network based reconstruction technique by additionally incorporating a variable spectral receptive field. The proposed updates have resulted in significant gains in performance as is demonstrated by the experiments conducted over multiple datasets. An additional update to the architecture is the use of non-convolutional branches in the function mixture network. The experiments of three standard datasets, clearly demonstrate the superiority of the proposed technique for hyperspectral image reconstruction based on RGB data.