Multilevel Dual-Direction Modifying Variational Autoencoders for Hyperspectral Feature Extraction

Abstract

Hyperspectral images (HSIs) provide abundant high-quality spectral information through an immense number of spectral channels, which can be used to classify on-ground objects for Earth observation accurately. However, these highly correlated channels and complex informative features always limit the application of HSIs. In this letter, we propose a multilevel dual-direction modifying variational autoencoder (MD²MVAE) for hyperspectral feature extraction. Its architecture is inspired by the spectral–spatial coherence in HSIs. Our motivation is to modify spectral sequential features by spatial sequential features in a multilevel dual-direction network. The dual-direction strategy has two implications: 1) the spatial continuity is captured by flattening neighboring samples in dual classic directions and 2) the spectral–spatial continuities are captured in the forward and backward directions. This multilevel dual-direction network provides a feasible way to avoid the loss of spatial information when flattening samples in the spatial domain, without using any convolutional layers. Inspired by our previous work, a variational autoencoder (VAE)-based network is used to enhance the noise immunity of latent features. To preserve the consistency of spectral–spatial sequential features, a combined loss function based on the solid angle on a unit sphere is proposed for parameter optimization. Two typical datasets are selected as benchmarks to show the effectiveness of MD²MVAE, compared with the state-of-the-art methods.