Hyperspectral Image Classification Research Articles

E2TNet: Efficient enhancement Transformer network for hyperspectral image classification

Recently, Convolutional Transformer-based models have become popular in hyperspectral image (HSI) classification tasks and gained competitive classification performance. However, some Convolutional Transformer-based models fail to effectively mine the global correlations of coarse-grained and fine-grained features, which is adverse to recognizing the refined scale variation information of land-cover. The combination of convolution operations and multihead self-attention mechanisms also increases the computational cost, leading to low classification efficiency. In addition, shallow spectral–spatial features are directly input into the encoder, which inevitably incurs redundant spectral information. Therefore, this paper proposes an efficient enhancement Transformer network (E2TNet) for HSI classification. Specifically, this paper first designs a spectral–spatial feature fusion module to extract spectral and spatial features from HSI cubes and fuse them. Second, considering that redundant spectral information has a negative impact on classification performance, this paper designs a spectral–spatial feature weighted module to improve the feature representation of critical spectral information. Finally, to explore the global correlations of coarse-grained and fine-grained features and improve classification efficiency, an efficient multigranularity information fusion module is embedded in the encoder of E2TNet. The experiment is conducted on four benchmark hyperspectral datasets, and the experimental results demonstrate that the proposed E2TNet is better than several Convolutional Transformer-based classification models in terms of classification accuracy and classification efficiency.

Cnn-assisted multi-hop graph attention network for hyperspectral image classification

ABSTRACT Recently, the convolutional neural network (CNN) has gained widespread adoption in the hyperspectral image (HSI) classification owing to its remarkable feature extraction capability. However, the fixed acceptance domain of CNN restricts it to Euclidean image data only, making it difficult to capture complex information in hyperspectral data. To overcome this problem, much attention has been paid to the graph attention network (GAT), which can effectively model graph structure and capture complex dependencies between nodes. However, GAT usually acts on superpixel nodes, which may lead to the loss of pixel-level information. To better integrate the advantages of both, we propose a CNN-assisted multi-hop graph attention network (CMGAT) for HSI classification. Specifically, a parallel dual-branch architecture is first constructed to simultaneously capture spectral-spatial features from hyperspectral data at the superpixel and pixel levels using GAT and CNN, respectively. On this basis, the multi-hop and multi-scale mechanisms are further employed to construct a multi-hop GAT module and a multi-scale CNN module to capture diverse feature information. Secondly, an attention module is cascaded before the multi-scale CNN module to improve classification performance. Eventually, the output information from the two branches is weighted and fused to produce the classification result. We performed experiments on four benchmark HSI datasets, including Indian Pines (IP), University of Pavia (UP), Salinas Valley (SV) and WHU-Hi-LongKou (LK). The results demonstrate that the proposed method outperforms several deep learning methods, achieving overall accuracies of 95.67%, 99.04%, 99.55% and 99.51%, respectively, even with fewer training samples.

Local-global feature fusion network for hyperspectral image classification

ABSTRACT Hyperspectral images (HSIs) are rich in spatial and spectral information, making them crucial for accurate classification. While existing convolutional neural network (CNN) methods have shown promise in HSI classification, they primarily focus on local features and struggle with establishing long-range dependencies. To address these limitations, we propose a Local-Global Feature Fusion Network (LGFFN). LGFFN effectively learns joint local-global information from HSIs. Firstly, we employ a group-general convolution parallel module (GGCPM) to extract both local and global spectral features. Next, we utilize parallel branches: a CNN branch for capturing fine-grained spatial information and an improved transformer encoder (ITE) branch for capturing long-distance spatial-spectral features and exploiting semantic information. Finally, we elegantly integrate the diverse features obtained from these branches to obtain classification results. We validate our method on four datasets and compare it with six state-of-the-art hyperspectral classification methods. Our experimental results demonstrate that LGFFN achieves high classification accuracy, outperforming existing methods.

Interactive transformer and CNN network for fusion classification of hyperspectral and LiDAR data

ABSTRACT The Transformer has become pivotal for the integrated analysis of multi-source remote-sensing (RS) data in Earth observation, particularly in applications such as the fusion classification of hyperspectral images (HSI) and Light Detection and Ranging (LiDAR) data. However, Transformers are often employed as effective feature extractors by adopting similar processing blocks for different modalities from multi-source sensors, overlooking differences in imaging principles and data characteristics. Moreover, in the feature extraction process across different sensor data, there is a lack of necessary cross-modal information interaction, leading to insufficient utilization of complementary information between different sensors and resulting in suboptimal fusion outcomes. In this paper, we propose an interactive Transformer and CNN network for the fusion classification of HSI and LiDAR data. Specifically, a heterogeneous three-branch network architecture is designed for HSI and LiDAR data, where Transformers and CNNs encapsulate global contextual spatial and spectral information for HSI and capture geometric elevation patterns for LiDAR data, respectively. Elevation-Spatial Interaction (ESI) and Spectral-Spatial Interaction (SSI) modules are then introduced for multi-stage feature interaction. ESI enables the CNN-Transformer network to focus on essential local elevation details while simultaneously modelling global contextual spatial information. SSI facilitates the Transformer-Transformer network to cyclically intertwine spectral and spatial information for long-range spectral-spatial feature fusion. Finally, the interacted elevation, spatial, and spectral features undergo the Gated Fusion module to achieve hierarchical fusion adaptively, resulting in an elevation-spatial-spectral representation. Experiments conducted on three benchmark HSI-LiDAR datasets demonstrate the effectiveness of our proposed approach.

MBSFC: hyperspectral image classification based on multi-branch and spectral feature conversion

ABSTRACT Hyperspectral image classification (HSI) is a vital aspect of remote sensing technology. However, the high-dimensional nature of HSI and the continuous spectral bands significantly impact its classification performance. In this paper, we propose a classification method called MBSFC for HSI classification. MBSFC utilizes multi-branch and spectral feature conversion (SFC) techniques to eliminate redundant information and extract valuable features. Firstly, to preserve spectral-spatial information, the SFC block is introduced to eliminate redundant information through up-sampling. Subsequently, the feature maps from the SFC block and the spectral bands reduced by a factor of four are fed into three branches: the spectral branch, spatial-X branch, and spatial-Y branch. Each branch employs a 3D-CNN-based dense block and attention mechanism to extract useful features. Finally, the obtained features from the three branches are fused for HSI classification. To cater to different application scenarios, we divide MBSFC into three models with different network structures: MBSFC-s, MBSFC-m, and MBSFC-l. The MBSFC-l model, with a three-branch structure, achieves the best performance. The three models differ in the number of branches in the feature extraction. Experimental results on four publicly available hyperspectral datasets show that MBSFC achieves competitive results with small samples compared to other state-of-the-art methods. The code is available at https://github.com/TeresaTing/MBSFC.

Mamba-in-Mamba: Centralized Mamba-Cross-Scan in Tokenized Mamba Model for Hyperspectral image classification

Hyperspectral image (HSI) classification plays a crucial role in remote sensing (RS) applications, enabling the precise identification of materials and land cover based on spectral information. This supports tasks such as agricultural management and urban planning. While sequential neural models like Recurrent Neural Networks (RNNs) and Transformers have been adapted for this task, they present limitations: RNNs struggle with feature aggregation and are sensitive to noise from interfering pixels, whereas Transformers require extensive computational resources and tend to underperform when HSI datasets contain limited or unbalanced training samples. To address these challenges, Mamba architectures have emerged, offering a balance between RNNs and Transformers by leveraging lightweight, parallel scanning capabilities. Although models like Vision Mamba (ViM) and Visual Mamba (VMamba) have demonstrated improvements in visual tasks, their application to HSI classification remains underexplored, particularly in handling land-cover semantic tokens and multi-scale feature aggregation for patch-wise classifiers. In response, this study introduces the Mamba-in-Mamba (MiM) architecture for HSI classification, marking a pioneering effort in this domain. The MiM model features: (1) a novel centralized Mamba-Cross-Scan (MCS) mechanism for efficient image-to-sequence data transformation; (2) a Tokenized Mamba (T-Mamba) encoder that incorporates a Gaussian Decay Mask (GDM), Semantic Token Learner (STL), and Semantic Token Fuser (STF) for enhanced feature generation; and (3) a Weighted MCS Fusion (WMF) module with a Multi-Scale Loss Design for improved training efficiency. Experimental results on four public HSI datasets—Indian Pines, Pavia University, Houston2013, and WHU-Hi-Honghu—demonstrate that our method achieves an overall accuracy improvement of up to 3.3%, 2.7%, 1.5%, and 2.3% over state-of-the-art approaches (i.e., SSFTT, MAEST, etc.) under both fixed and disjoint training-testing settings.

Deep Ring-Block-Wise Network for Hyperspectral Image Classification.

Deep learning has achieved many successes in the field of the hyperspectral image (HSI) classification. Most of existing deep learning-based methods have no consideration of feature distribution, which may yield lowly separable and discriminative features. From the perspective of spatial geometry, one excellent feature distribution form requires to satisfy both properties, i.e., block and ring. The block means that in a feature space, the distance of intraclass samples is close and the one of interclass samples is far. The ring represents that all class samples are overall distributed in a ring topology. Accordingly, in this article, we propose a novel deep ring-block-wise network (DRN) for the HSI classification, which takes full consideration of feature distribution. To obtain the good distribution used for high classification performance, in this DRN, a ring-block perception (RBP) layer is built by integrating the self-representation and ring loss into a perception model. By such way, the exported features are imposed to follow the requirements of both block and ring, so as to be more separably and discriminatively distributed compared with traditional deep networks. Besides, we also design an optimization strategy with alternating update to obtain the solution of this RBP layer model. Extensive results on the Salinas, Pavia Centre, Indian Pines, and Houston datasets have demonstrated that the proposed DRN method achieves the better classification performance in contrast to the state-of-the-art approaches.

HKNAS: Classification of Hyperspectral Imagery Based on Hyper Kernel Neural Architecture Search.

Recent neural architecture search (NAS)-based approaches have made great progress in the hyperspectral image (HSI) classification tasks. However, the architectures are usually optimized independently of the network weights, increasing searching time, and restricting model performances. To tackle these issues, in this article, different from previous methods that extra define structural parameters, we propose to directly generate structural parameters by utilizing the specifically designed hyper kernels, ingeniously converting the original complex dual optimization problem into easily implemented one-tier optimizations, and greatly shrinking searching costs. Then, we develop a hierarchical multimodule search space whose candidate operations only contain convolutions, and these operations can be integrated into unified kernels. Using the above searching strategy and searching space, we obtain three kinds of networks to separately conduct pixel-level or image-level classifications with 1-D or 3-D convolutions. In addition, by combining the proposed hyper kernel searching scheme with the 3-D convolution decomposition mechanism, we obtain diverse architectures to simulate 3-D convolutions, greatly improving network flexibilities. A series of quantitative and qualitative experiments on six public datasets demonstrate that the proposed methods achieve state-of-the-art results compared with other advanced NAS-based HSI classification approaches.

An efficient hyperspectral image classification method using retentive network

In recent computer vision tasks, the vision transformer (ViT) has demonstrated competitive ability. However, ViT still has problems: the computational complexity of the self-attention layer leads to expensive and slow interference, and processing all tokens for high-resolution images may slow down due to the layer’s quadratic complexity. Recently, a retentive network with excellent performance, training parallelism, and an inexpensive inference cost was proposed. For hyperspectral image (HSI) classification, this paper proposesa retention-based network model called the HSI retentive network (HSIRN). The proposed model allows memory usage independent of the token’s sequence, facilitating the efficient processing of high-resolution images with low inference and computational costs. Although the retention encoder can extract global data, it pays limited attention to local data. A powerful tool for extracting local information is a convolutional neural network (CNN). The proposed HSIRN model uses a specific CNN-based block to extract local spectral-spatial information. To maintain degradation between successive vertical and horizontal positions with the depth dimension of the HSI, we propose a three-dimensional retention mechanism for the three-dimensional HSI dataset in the retention encoder. By efficiently using both local and global spectral-spatial information, the proposed method offers a potent tool for HSI classification. We evaluated the classification performance of the proposed HSIRN approach on four datasets through comprehensive examinations, and the results demonstrated its superiority over state-of-the-art methods. At https://github.com/RajatArya22/HSIRN, the source code will be available to the public.

Overcoming the Barrier of Incompleteness: A Hyperspectral Image Classification Full Model.

Deep learning-based methods have shown promising outcomes in many fields. However, the performance gain is always limited to a large extent in classifying hyperspectral image (HSI). We discover that the reason behind this phenomenon lies in the incomplete classification of HSI, i.e., existing works only focus on a certain stage that contributes to the classification, while ignoring other equally or even more significant phases. To address the above issue, we creatively put forward three elements needed for complete classification: the extensive exploration of available features, adequate reuse of representative features, and differential fusion of multidomain features. To the best of our knowledge, these three elements are being established for the first time, providing a fresh perspective on designing HSI-tailored models. On this basis, an HSI classification full model (HSIC-FM) is proposed to overcome the barrier of incompleteness. Specifically, a recurrent transformer corresponding to Element 1 is presented to comprehensively extract short-term details and long-term semantics for local-to-global geographical representation. Afterward, a feature reuse strategy matching Element 2 is designed to sufficiently recycle valuable information aimed at refined classification using few annotations. Eventually, a discriminant optimization is formulized in accordance with Element 3 to distinctly integrate multidomain features for the purpose of constraining the contribution of different domains. Numerous experiments on four datasets at small-, medium-, and large-scale demonstrate that the proposed method outperforms the state-of-the-art (SOTA) methods, such as convolutional neural network (CNN)-, fully convolutional network (FCN)-, recurrent neural network (RNN)-, graph convolutional network (GCN)-, and transformer-based models (e.g., accuracy improvement of more than 9% with only five training samples per class). The code will be available soon at https://github.com/jqyang22/ HSIC-FM.

Semi-Supervised Adaptive Pseudo-Label Feature Learning for Hyperspectral Image Classification in Internet of Things

Semi-Supervised Adaptive Pseudo-Label Feature Learning for Hyperspectral Image Classification in Internet of Things

Semantic Guided prototype learning for Cross-Domain Few-Shot hyperspectral image classification

Hyperspectral image(HSI) classification has found extensive application in the field of remote sensing. However, traditional HSI classification methods typically rely on a large number of labeled samples, incurring significant costs and requiring considerable time and effort for acquisition. To address this issue, this paper proposes a novel cross-domain few-shot learning method termed Semantic Guided Prototype Learning (SGPL), which aims to optimize prototype learning through the incorporation semantic information, thereby enhancing the classification accuracy of hyperspectral images in few-shot scenarios. The core idea of the SGPL method is to integrate semantic information into the prototype learning framework, thereby enhancing the expressive capacity of each category prototype. Specifically, HSI category labels are encoded via one-hot encoding, and the corresponding HSI features are concatenated with these encoded vectors to construct an augmented feature representation. These augmented feature vectors are subsequently fed into a prototype generator, which is designed to learn the generation of category prototypes from the concatenated features. Through the application of one-hot encoding and feature concatenation, SGPL effectively captures and leverages discriminative information between classes. Moreover, the proposed model exhibits superior performance on three different HSI datasets: Salinas, the University of Pavia(UP), and Indian Pines(IP) with overall classification accuracies of 91.27%, 85.84%, and 69.33%, respectively, outperforming other methods. The source code is available at https://github.com/AIYAU/SGPL.

Hyperspectral image classification using feature fusion fuzzy graph broad network

In recent years, graph convolutional networks (GCNs) have shown strong performance in hyperspectral image (HSI) classification. However, traditional GCN methods often use superpixel-based nodes to reduce computational complexity, which fails to capture pixel-level spectral-spatial features. Additionally, these methods typically focus on matching predicted labels with ground truth, neglecting the relationships between inter-class and intra-class distances, leading to less discriminative features. To address these issues, we propose a feature fusion fuzzy graph broad network (F3GBN) for HSI classification. Our method extracts pixel-level attribute contour features using attribute filters and fuses them with superpixel features through canonical correlation analysis. We employ a broad learning system (BLS) as the classifier, which fully utilizes spectral-spatial information via nonlinear transformations. Furthermore, we construct intra-class and inter-class graphs based on fuzzy set and manifold learning theories to ensure better clustering of samples within the same class and separation between different classes. A novel loss function is introduced in BLS to minimize intra-class distances and maximize inter-class distances, enhancing feature discriminability. The proposed F3GBN model achieved impressive overall accuracy on public datasets: 96.73% on Indian Pines, 98.29% on Pavia University, 98.69% on Salinas, and 99.43% on Kennedy Space Center, outperforming several classical and state-of-the-art methods, thereby demonstrating its effectiveness and feasibility.

Weighted Contrastive Prototype Network for Few-Shot Hyperspectral Image Classification with Noisy Labels

Few-shot hyperspectral image classification aims to develop the ability of classifying image pixels by using relatively few labeled pixels per class. However, due to the inaccuracy of the localization system and the bias of the ground survey, the potential noisy labels in the training data pose a very significant challenge to few-shot hyperspectral image classification. To solve this problem, this paper proposes a weighted contrastive prototype network (WCPN) for few-shot hyperspectral image classification with noisy labels. WCPN first utilizes a similarity metric to generate the weights of the samples from the same classes, and applies them to calibrate the class prototypes of support and query sets. Then the weighted prototype network will minimize the distance between features and prototypes to train the network. WCPN also incorporates a weighted contrastive regularization function that uses the sample weights as gates to filter the fake positive samples whose labels are incorrect to further improve the discriminative power of the prototypes. We conduct experiments on multiple hyperspectral image datasets with artificially generated noisy labels, and the results show that the WCPN has excellent performance that can sufficiently mitigate the impact of noisy labels.

PGNN-Net: Parallel Graph Neural Networks for Hyperspectral Image Classification Using Multiple Spatial-Spectral Features

Hyperspectral image (HSI) shows great potential for application in remote sensing due to its rich spectral information and fine spatial resolution. However, the high dimensionality, nonlinearity, and complex relationship between spectral and spatial features of HSI pose challenges to its accurate classification. Traditional convolutional neural network (CNN)-based methods suffer from detail loss in feature extraction; Transformer-based methods rely too much on the quantity and quality of HSI; and graph neural network (GNN)-based methods provide a new impetus for HSI classification by virtue of their excellent ability to handle irregular data. To address these challenges and take advantage of GNN, we propose a network of parallel GNNs called PGNN-Net. The network first extracts the key spatial-spectral features of HSI using principal component analysis, followed by preprocessing to obtain two primary features and a normalized adjacency matrix. Then, a parallel architecture is constructed using improved GCN and ChebNet to extract local and global spatial-spectral features, respectively. Finally, the discriminative features obtained through the fusion strategy are input into the classifier to obtain the classification results. In addition, to alleviate the over-fitting problem, the label smoothing technique is embedded in the cross-entropy loss function. The experimental results show that the average overall accuracy obtained by our method on Indian Pines, Kennedy Space Center, Pavia University Scene, and Botswana reaches 97.35%, 99.40%, 99.64%, and 98.46%, respectively, which are better compared to some state-of-the-art methods.

Class incremental learning with analytic learning for hyperspectral image classification

Hyperspectral image classification (HSIC) is crucial for applications in agriculture and environmental monitoring. As ground objects evolve and remote sensing technology progresses, there is a growing need for HSIC models that can adapt to new data classes without requiring to be retrained from scratch. Throughout the continual learning procedure, the model is expected to not only effectively extract spatial–spectral features from the hyperspectral image but also alleviate the issue of catastrophic forgetting, i.e., the model forgets the learned classes’ knowledge when accessing novel classes during the training process. In this paper, for the HSIC task, we propose a class incremental learning method (HSI-CIL) that is based on analytic learning, a technique that converts network training into linear problems. Specifically, The HSI-CIL model consists of a lightweight feature extractor, a Feature Processing Module (FPM) and an Analytic Linear Classifier (ALC). This model does not need data storage for old and new classes and has only one epoch in the incremental learning stage, so it has lower consumption of resources and training time than several attempts have been proposed for addressing catastrophic forgetting. We perform abundant experiments with the proposed HSI-CIL on three publicly available hyperspectral datasets including Indian Pines, Pavia University, and Salinas. The experiments demonstrate that our HSI-CIL exceeds the state-of-the-art class incremental learning (CIL) techniques applied in HSIC with a certain gap.

Hyperspectral Image Classification Using a Hybrid RNN-CNN with Enhanced Attention Mechanisms

Hyperspectral Image Classification Using a Hybrid RNN-CNN with Enhanced Attention Mechanisms

Investigation of the performances of Support Vector Machine, Random Forest, and 3D-2D Convolutional Neural Network for Hyperspectral Image Classification

Classification of the hyperspectral images (HSIs) is one of the most challenging tasks hyperspectral remote sensing. Various Machine Learning classification algorithms have been implemented to HSI classification. In recent years, several Convolutional Neural Network (CNN) architectures were developed for HSI classification. The aim of this study is to test the performance of CNN, and well-known Support Vector Machine and Random Forest algorithms using the HyRANK Loukia, Houston 2013, and Salinas Scene datasets. The findings indicate that the Modified HybridSN CNN outperformed other algorithms across all datasets, as demonstrated by various performance evaluation metrics.

SMALE: Hyperspectral Image Classification via Superpixels and Manifold Learning

As an extremely efficient preprocessing tool, superpixels have become more and more popular in various computer vision tasks. Nevertheless, there are still several drawbacks in the application of hyperspectral image (HSl) processing. Firstly, it is difficult to directly apply superpixels because of the high dimension of HSl information. Secondly, existing superpixel algorithms cannot accurately classify the HSl objects due to multi-scale feature categorization. For the processing of high-dimensional problems, we use the principle of PCA to extract three principal components from numerous bands to form three-channel images. In this paper, a novel superpixel algorithm called Seed Extend by Entropy Density (SEED) is proposed to alleviate the seed point redundancy caused by the diversified content of HSl. It also focuses on breaking the dilemma of manually setting the number of superpixels to overcome the difficulty of classification imprecision caused by multi-scale targets. Next, a space–spectrum constraint model, termed Hyperspectral Image Classification via superpixels and manifold learning (SMALE), is designed, which integrates the proposed SEED to generate a dimensionality reduction framework. By making full use of spatial context information in the process of unsupervised dimension reduction, it could effectively improve the performance of HSl classification. Experimental results show that the proposed SEED could effectively promote the classification accuracy of HSI. Meanwhile, the integrated SMALE model outperforms existing algorithms on public datasets in terms of several quantitative metrics.

Feature extraction via 3-D homogeneous attribute decomposition for hyperspectral imagery classification

ABSTRACT Feature extraction is a core aspect in hyperspectral image classification, which can extract key information closely related to ground cover from complex scene, thus improving classification accuracy. Therefore, designing an effective feature extraction network is a hotspot and a challenge in the current research. In this paper, a feature extraction framework based on 3-D homogeneous attribute decomposition (3D-HAD) is proposed for HSI classification, which consists of the following key technologies. First, the principal component analysis algorithm is applied to the raw HSI to extract the principal components (PCs), and the raw HSI is clustered into many 3D superpixel blocks according to the first three PCs-based over-segmentation strategy. Then, a superpixel intrinsic attribute decomposition (SIAD) is designed to capture reflectance feature and suppress shading feature. Meanwhile, a metric entropy is introduced into the decomposition process to overcome the spectral-spatial weak assumption among pixels. Next, superpixel-guided recursive filtering is employed to preserve global details of HSI to enhance accuracy in HSI classification. Finally, the support vector machine classifier is used to obtain classification results of HSI. Experiments performed on several real hyperspectral data sets with limited training samples indicate that the proposed 3D-HAD method outperforms the classic, advance, and deep learning classification methods.