Abstract
ABSTRACT Hyperspectral image (HSI) contains hundreds of contiguous spectral bands compared with red green blue (RGB) image, making the precise identification of materials possible by capturing subtle spectral and spatial features. Owing to the special advantage in image processing, convolutional neural networks (CNNs) have been proven to be a successful architecture in HSI classification. However, due to the limitation of receptive field of fixed convolution kernel, CNNs can only extract local features of hyperspectral image. Besides, CNNs fail to mine and represent the sequence attributes of spectral bands because of the limitations of its inherent network backbone. With the emergence of vision transformer, the network can break through the limitation of receptive field and obtain the global correlation of the whole image, but it only focuses on the global information of the object and ignores the local information existing in the sequence and images. Moreover, the correlation of spectral information will be destroyed if traditional methods are directly used to convert hyperspectral images into sequence. To solve this issue, a novel convolution and vision transformer fusion network called CAVFN is devised which contains a new cube-embedding module that can reduce the loss of spectral information effectively by dividing the large HSI cube into several small cubes and encoding them into sequences. More significantly, this paper also combines 1D-CNN and 2D-CNN with vision transformer to extract the local features of sequences and patches, and combines them with global features to obtain better classification results. Finally, this paper evaluates the classification results of the proposed network on three HSI datasets by conducting extensive experiments, showing that our network outperforms other state-of-the-art methods.
Talk to us
Join us for a 30 min session where you can share your feedback and ask us any queries you have
Disclaimer: All third-party content on this website/platform is and will remain the property of their respective owners and is provided on "as is" basis without any warranties, express or implied. Use of third-party content does not indicate any affiliation, sponsorship with or endorsement by them. Any references to third-party content is to identify the corresponding services and shall be considered fair use under The CopyrightLaw.