A Unified Multiview Spectral Feature Learning Framework for Hyperspectral Image Classification

Abstract

Recent progress in spectral classification is dominated by the use of deep learning models. While various learning architectures have been developed, they all extract spectral features from a single view input. In this paper, we investigate a different perspective and develop a unified multiview spectral feature learning framework, which extracts discriminative spectral features from multiple views of inputs. To our knowledge, this is the first reported multiview spectral feature learning method based on deep learning. In this framework, we introduce a multiview spectrum construction method by transforming the input spectral vector into multiple 3D image patches with different sizes, termed as multiview spectrum. This multiview spectrum is fed to a well-designed triple-stream architecture, where a global and two local spectral feature learning networks operate in parallel, capturing thus both global and local spectral contextual features simultaneously. Another important contribution of this work is a novel interactive attention mechanism to identify the most informative spectral contextual features. The model is trained in an end-to-end fashion from scratch with a joint loss. Experimental results on four data sets demonstrate excellent performance compared to the current state-of-the-art.