Selection of Optimal Bands for Hyperspectral Local Feature Descriptor

Abstract

The use of hyperspectral images (HSI) and 3D data has proven to be an efficient combination for numerous applications. A common method of obtaining corresponding data is 3D reconstruction from HSI, for which the local feature descriptor is vital to the final accuracy. However, redundant bands may hamper the performance of the descriptor, which is similar to the so-called Hughes phenomenon. Band selection (BS) is effective in overcoming such problems. Existing BS methods fail to select the band in a differentiable manner and, thus, cannot be jointly optimized with downstream tasks. In this letter, we propose a novel end-to-end HSI local feature descriptor network (with joint optimal band selection) called HyperDesc. It implements a true band selection module by turning band selection into a CONCRETE random variable-based differentiable sampling operation. The discrete distribution of each selected band can be learned from input HSI with a non-local spectral-spatial attention network. Finally, the selected band is sent to a descriptor network to extract the local feature descriptor. The whole network is trained in an end-to-end manner. Experiments are conducted on multi-view close range HSI. The results show that spectral information provided by selected bands can boost the performance of the descriptor.