Convolutional Transformer Network for Hyperspectral Image Classification

Abstract

Convolutional neural networks (CNNs) have attained remarkable performance in hyperspectral image (HSI) classification. However, the existing CNNs are restricted by their limited receptive field in HSI classification. Recently, transformer networks have proved to be promising in many tasks thanks to the global receptive field, but they easily ignore some local information that is important for HSI classification. In this letter, we propose a novel method entitled convolutional transformer network (CTN) for HSI classification. In order to make full use of spectral information and spatial information, the method adopts center position encoding (CPE) to merge spectral features and pixel positions. Furthermore, the proposed method introduces convolutional transformer (CT) blocks. It effectively combines convolution and transformer structures together to capture local–global features of HSI patches, which is contributive for HSI classification. Experimental results on public datasets demonstrate the superiority of our method compared with several state-of-the-art classification methods. The codes of this work will be available at <uri xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">https://github.com/sky8791</uri> to facilitate reproducibility.