A Unified Multiscale Learning Framework for Hyperspectral Image Classification

Abstract

The highly correlated spectral features and the limited training samples pose challenges in hyperspectral image classification. In this article, to tackle the issues of end-to-end feature learning and transfer learning with limited labeled samples, we propose a unified multiscale learning (UML) framework, which is based on a fully convolutional network. A multiscale spatial-channel attention mechanism and a multiscale shuffle block are proposed in the UML framework to improve the problem of land-cover map distortion. The contextual information and the spectral feature are enhanced before the last classification layer based on three strategies in this work: 1) the channel shuffle operation, which was employed to learn the more effective spectral characteristics by disordering the channels of the feature map; 2) multiscale block, which considered the contextual information in multiple ranges; and 3) spatiospectral attention, which enhanced the expression of the important characteristic among all pixels. Three hyperspectral datasets, including two airborne hyperspectral images and one spaceborne hyperspectral image, were used to demonstrate the performance of the UML framework in both classification and transfer learning. The experimental results confirmed that the proposed method outperforms most of the state-of-the-art hyperspectral image classification methods. The source code is released at <uri xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">https://github.com/Hyper-NN/UML</uri> .