Deep global-local transformer network combined with extended morphological profiles for hyperspectral image classification

Abstract

Recently, deep learning models based on convolutional neural networks (CNN) remain dominant in hyperspectral image (HSI) classification. However, there are some problems in CNN models, such as not good at modeling the long-distance dependencies and obtaining global context information. Different from the existing CNN-based models, an innovative classification method based on the transformer model is proposed to further improve the classification accuracy of HSI. Specifically, the proposed method first extracts the extended morphological profile (EMP) features of HSI to make full use of the spatial and spectral information while effectively reducing the number of bands. Next, a deep network model is constructed by introducing the transformer-iN-transformer (TNT) modules to carry out end-to-end classification. The outer and inner transformer models in the TNT module can extract the patch-level and pixel-level features, respectively, to make full use of the global and local information in the input EMP cubes. Experimental results on three public HSI data sets show that the proposed method can achieve better classification performance than the existing CNN-based models. In addition, using the transformer-based deep model without convolution to classify HSI provides a new idea for related research.