Hyperspectral Image Classification Research Articles

A Joint Network of Edge-Aware and Spectral-Spatial Feature Learning for Hyperspectral Image Classification.

Hyperspectral image (HSI) classification is a vital part of the HSI application field. Since HSIs contain rich spectral information, it is a major challenge to effectively extract deep representation features. In existing methods, although edge data augmentation is used to strengthen the edge representation, a large amount of high-frequency noise is also introduced at the edges. In addition, the importance of different spectra for classification decisions has not been emphasized. Responding to the above challenges, we propose an edge-aware and spectral-spatial feature learning network (ESSN). ESSN contains an edge feature augment block and a spectral-spatial feature extraction block. Firstly, in the edge feature augment block, the edges of the image are sensed, and the edge features of different spectral bands are adaptively strengthened. Then, in the spectral-spatial feature extraction block, the weights of different spectra are adaptively adjusted, and more comprehensive depth representation features are extracted on this basis. Extensive experiments on three publicly available hyperspectral datasets have been conducted, and the experimental results indicate that the proposed method has higher accuracy and immunity to interference compared to state-of-the-art (SOTA) method.

Identification and sorting of impurities in tea using spectral vision

Removing impurities from Puer tea is an important step in its processing. However, existing methods are unable to accurately and efficiently sort impurities from Puer tea. This study proposes a novel method based on spectral imaging technology to identify impurities in Puer tea. Hyperspectral images of tea and impurities were acquired in the 400–1000 nm wavelengths. The spectra of each category of samples were extracted from the hyperspectral images and comprehensively analysed and processed, mainly including outlier detection and elimination, spectral preprocessing, and characteristic wavelengths selection. Support vector machine (SVM) models are built based on full-spectrum data and characteristic wavelengths data to achieve pixel-level classification of hyperspectral images. The results showed that the best model based on full spectra achieved an accuracy of 97.8% on the test dataset. The best models based on characteristic wavelengths selected by successive projections algorithm and genetic algorithm achieved accuracies of 95.9% and 94.9%, respectively. Finally, these pixel classification models were applied to the test samples to test the effectiveness of the models in identifying impurities. The results indicate that the three selected models can effectively identify impurities and even perform well on identifying unfamiliar categories of impurities.

Hyperspectral image classification based on adaptive spectral feature decoupling with global local feature fusion network

Hyperspectral image classification based on adaptive spectral feature decoupling with global local feature fusion network

HyperGCN – a multi-layer multi-exit graph neural network to enhance hyperspectral image classification

ABSTRACT Graph neural networks (GNNs) have recently garnered significant attention due to their exceptional performance across various applications, including hyperspectral (HS) image classification. However, most existing GNN-based models for HS image classification are limited depth models and often suffer from performance degradation as model depth increases. This study introduces HyperGCN, an exclusive GNN-based model designed with multiple graph convolutional layers to exploit the rich spectral information inherent in HS images, thereby enhancing classification performance. To address performance degradation, HyperGCN incorporates techniques resistant to oversmoothing into its architecture. Additionally, multiple-side exit branches are integrated into the intermediate layers of HyperGCN, enabling dynamic management of the complexity of HS images. Less complex HS images are processed by fewer layers, exiting early via attached branches, while more complex images traverse multiple layers until reaching the final output layer. Extensive experiments on four benchmark HS datasets (Indian Pines, Pavia University, Salinas, and Botswana) demonstrate HyperGCN’s superior performance over basic GNN-based models. Notably, HyperGCN outperforms or performs comparably to the CNN-GNN combined model in classifying HS images. Furthermore, the superior performance of multi-exit HyperGCN over its single-exit counterpart emphasizes the effectiveness of incorporating side exit branches in GNN-based HS image classification. Compared to state-of-the-art models, multi-exit HyperGCN demonstrates competitive performance, highlighting its effectiveness in handling complex spectral information in HS images while maintaining an acceptable balance between accuracy and computational efficiency.

Spectral-Spatial Mamba for Hyperspectral Image Classification

Recently, transformer has gradually attracted interest for its excellence in modeling the long-range dependencies of spatial-spectral features in HSI. However, transformer has the problem of the quadratic computational complexity due to the self-attention mechanism, which is heavier than other models and thus has limited adoption in HSI processing. Fortunately, the recently emerging state space model-based Mamba shows great computational efficiency while achieving the modeling power of transformers. Therefore, in this paper, we first proposed spectral-spatial Mamba (SS-Mamba) for HSI classification. Specifically, SS-Mamba mainly includes a spectral-spatial token generation module and several stacked spectral-spatial Mamba blocks. Firstly, the token generation module converts any given HSI cube to spatial and spectral tokens as sequences. And then these tokens are sent to stacked spectral-spatial mamba blocks (SS-MB). Each SS-MB includes two basic mamba blocks and a spectral-spatial feature enhancement module. The spatial and spectral tokens are processed separately by the two basic mamba blocks, correspondingly. Moreover, the feature enhancement module modulates spatial and spectral tokens using HSI sample’s center region information. Therefore, the spectral and spatial tokens cooperate with each other and achieve information fusion within each block. The experimental results conducted on widely used HSI datasets reveal that the proposed SS-Mamba requires less processing time compared with transformer. The Mamba-based method thus opens a new window for HSI classification.

Feature Consistency-Based Prototype Network for Open-Set Hyperspectral Image Classification.

Hyperspectral image (HSI) classification methods have made great progress in recent years. However, most of these methods are rooted in the closed-set assumption that the class distribution in the training and testing stages is consistent, which cannot handle the unknown class in open-world scenes. In this work, we propose a feature consistency-based prototype network (FCPN) for open-set HSI classification, which is composed of three steps. First, a three-layer convolutional network is designed to extract the discriminative features, where a contrastive clustering module is introduced to enhance the discrimination. Then, the extracted features are used to construct a scalable prototype set. Finally, a prototype-guided open-set module (POSM) is proposed to identify the known samples and unknown samples. Extensive experiments reveal that our method achieves remarkable classification performance over other state-of-the-art classification techniques.

Achieving Better Category Separability for Hyperspectral Image Classification: A Spatial-Spectral Approach.

The task of hyperspectral image (HSI) classification has attracted extensive attention. The rich spectral information in HSIs not only provides more detailed information but also brings a lot of redundant information. Redundant information makes spectral curves of different categories have similar trends, which leads to poor category separability. In this article, we achieve better category separability from the perspective of increasing the difference between categories and reducing the variation within category, thus improving the classification accuracy. Specifically, we propose the template spectrum-based processing module from spectral perspective, which can effectively expose the unique characteristics of different categories and reduce the difficulty of model mining key features. Second, we design an adaptive dual attention network from spatial perspective, where the target pixel can adaptively aggregate high-level features by evaluating the confidence of effective information in different receptive fields. Compared with the single adjacency scheme, the adaptive dual attention mechanism makes the ability of target pixel to combine spatial information to reduce variation more stable. Finally, we designed a dispersion loss from the classifier's perspective. By supervising the learnable parameters of the final classification layer, the loss makes the category standard eigenvectors learned by the model more dispersed, which improves the category separability and reduces the rate of misclassification. Experiments on three common datasets show that our proposed method is superior to the comparison method.

Superpixel‐guided locality preserving projection and spatial–spectral classification for hyperspectral image

Superpixel‐guided locality preserving projection and spatial–spectral classification for hyperspectral image

Composite Neighbor-Aware Convolutional Metric Networks for Hyperspectral Image Classification.

Supervised classification of hyperspectral image (HSI) is generally required to obtain better performance in spectral-spatial feature learning by fully using complex pixel- and superpixel-level interdependencies with small labeled samples. Limited by the local regular convolutions, convolutional neural networks (CNNs) can only exploit information from the short-range Euclidean neighbors of a target, hindering the effectiveness of feature representation. In contrast, graph convolutional networks (GCNs) can learn long-range dependencies between non-Euclidean neighbors but usually require the input of a full graph constructed from a whole HSI, making GCNs must be trained in a full-batch manner with tremendous computational consumption. In this work, we propose a composite neighbor-aware convolutional metric network (CNCMN), aiming to learn each target's representation from its composite neighbors (i.e., both Euclidean and non-Euclidean neighbors) in a batchwise manner. Specifically, for each target in an HSI, its Euclidean neighbors are the pixels in the local square region centered on itself, and its non-Euclidean neighbors are several related nodes selected from the constructed full graph. Correspondingly, a composite convolution (CoConv) is proposed by coupling an image convolution and a graph convolution, which can perform flexible convolutions on those composite neighbors and extract adaptively fused features from them. Besides, to further boost classification, we also propose a mini-batch metric classifier to dynamically optimize interclass and intraclass distances of samples batch by batch, which is then combined with the CoConv to form the mini-batch CNCMN. Extensive experiments on three real-world HSIs demonstrate the advantages of the proposed method over mini-batch deep learning algorithms and have obtained the state-of-the-art performance in these fields. The code is available at: https://github.com/qichaoliu/HSI-CNCMN.

DW-D3A: dynamic weighted dual-driven domain adaptation for cross-scene hyperspectral image classification

ABSTRACT Domain adaptation (DA) offers an effective way to align feature distributions of the source domain (SD) and the target domain (TD) without requiring any target label samples. As a method of DA, representation learning effectively realizes the alignment of feature distributions in different domains by transferring domain knowledge. However, existing representation learning methods often focus on unilateral representation transfer, which potentially results in transfer bias. Additionally, most methods ignore the connection between domain alignment and discrimination during the DA process, which easily causes negative transfer. This paper proposes a dynamic weighted dual-driven domain adaptation (DW-D 3 A) model that effectively addresses the aforementioned issues through bilateral feature transfer between domains and a dynamic weighted scheme. Technically, we first propose a dual-driven domain adaptation (D 3 A) model, which employs symmetrical structures to facilitate the knowledge transfer of bilateral representations between source and target domain samples, learning the subspaces of two domains and reducing distribution discrepancies between subspaces via joint distribution-driven alignment. This process mitigates transfer bias and goes beyond previous unilateral transfer methods. Then, to alleviate strong constraints on projecting SD and TD into the same subspace in existing approaches, we apply a relaxed subspace constraint to bring the projections of SD and TD closer. Furthermore, data reconstruction is incorporated to preserve discriminant information from the original data. Lastly, we expand (D 3 A) to DW-D 3 A using a dynamic weighted scheme, which adjusts the weights assigned to domain alignment and discrimination based on their significance to inhibit negative transfer. Extensive experimentation on three datasets indicates that DW-D 3 A outperforms seven other DA methods, showing its superior performance.

Physics-informed interactive network for hyperspectral image classification

In hyperspectral image (HSI) classification, convolutional neural networks (CNNs) have made great progress. In recent CNN-based HSI classification methods, spatial attention mechanism has been widely used to emphasize more salient features and suppress less useful ones. However, these methods tend to utilize plug-and-play attention modules which ignore the physical meaning of HSI pixels. To reflect the physical construction of HSI pixels in the attention mask, we propose a physics-informed interactive network (PI2Net) integrating a spatial guided spectral component stream (SGSC-stream) and a spectral constrained spatial attention stream (SCSA-stream). The SGSC-stream can obtain more expressive abundance features of the input HSI cubes through a component analysis block. The SCSA-stream completes the prediction of hyperspectral data in three-stage features extraction cells and a classification cell. To connect and interact between these two streams, a component-aware attention module (CAAM) and a class-wise endmember activation gate (CWEAG) are designed. The CAAM measures the similarity of abundance features to generate a component-aware attention mask, so as to learn the component-based relationships between pixels. Then, the multiscale attention masks are embedded into different stages of the SCSA-stream. The CWEAG is proposed to iteratively generate more accurate abundance features with the classification result of SCSA-stream. Experimental results based on three common hyperspectral data sets demonstrate that PI2Net outperforms several state-of-the-art methods in terms of qualitative and quantitative analysis.

Adaptive feature selection for hyperspectral image classification based on Improved Unsupervised Mayfly optimization Algorithm

Adaptive feature selection for hyperspectral image classification based on Improved Unsupervised Mayfly optimization Algorithm

Multiscale Feature Search-Based Graph Convolutional Network for Hyperspectral Image Classification

With the development of hyperspectral sensors, the availability of hyperspectral images (HSIs) has increased significantly, prompting advancements in deep learning-based hyperspectral image classification (HSIC) methods. Recently, graph convolutional networks (GCNs) have been proposed to process graph-structured data in non-Euclidean domains, and have been used for HSIC. The superpixel segmentation should be implemented first in the GCN-based methods, however, it is difficult to manually select the optimal superpixel segmentation sizes to obtain the useful information for classification. To solve this problem, we constructed a HSIC model based on a multiscale feature search-based graph convolutional network (MFSGCN) in this study. Firstly, pixel-level features of HSIs are extracted sequentially using 3D asymmetric decomposition convolution and 2D convolution. Then, superpixel-level features at different scales are extracted using multilayer GCNs. Finally, the neural architecture search (NAS) method is used to automatically assign different weights to different scales of superpixel features. Thus, a more discriminative feature map is obtained for classification. Compared with other GCN-based networks, the MFSGCN network can automatically capture features and obtain higher classification accuracy. The proposed MFSGCN model was implemented on three commonly used HSI datasets and compared to some state-of-the-art methods. The results confirm that MFSGCN effectively improves accuracy.

A Hybrid Firefly with Particle Swarm Optimization Based Hyperspectral Image Classification and Segmentation Using Structured Support Vector Machine

A Hybrid Firefly with Particle Swarm Optimization Based Hyperspectral Image Classification and Segmentation Using Structured Support Vector Machine

Fluorescence excitation-scanning hyperspectral imaging with scalable 2D–3D deep learning framework for colorectal cancer detection

Colorectal cancer is one of the top contributors to cancer-related deaths in the United States, with over 100,000 estimated cases in 2020 and over 50,000 deaths. The most common screening technique is minimally invasive colonoscopy using either reflected white light endoscopy or narrow-band imaging. However, current imaging modalities have only moderate sensitivity and specificity for lesion detection. We have developed a novel fluorescence excitation-scanning hyperspectral imaging (HSI) approach to sample image and spectroscopic data simultaneously on microscope and endoscope platforms for enhanced diagnostic potential. Unfortunately, fluorescence excitation-scanning HSI datasets pose major challenges for data processing, interpretability, and classification due to their high dimensionality. Here, we present an end-to-end scalable Artificial Intelligence (AI) framework built for classification of excitation-scanning HSI microscopy data that provides accurate image classification and interpretability of the AI decision-making process. The developed AI framework is able to perform real-time HSI classification with different speed/classification performance trade-offs by tailoring the dimensionality of the dataset, supporting different dimensions of deep learning models, and varying the architecture of deep learning models. We have also incorporated tools to visualize the exact location of the lesion detected by the AI decision-making process and to provide heatmap-based pixel-by-pixel interpretability. In addition, our deep learning framework provides wavelength-dependent impact as a heatmap, which allows visualization of the contributions of HSI wavelength bands during the AI decision-making process. This framework is well-suited for HSI microscope and endoscope platforms, where real-time analysis and visualization of classification results are required by clinicians.

Advanced Hyperspectral Image Analysis: Superpixelwise Multiscale Adaptive T-HOSVD for 3D Feature Extraction.

Hyperspectral images (HSIs) possess an inherent three-order structure, prompting increased interest in extracting 3D features. Tensor analysis and low-rank representations, notably truncated higher-order SVD (T-HOSVD), have gained prominence for this purpose. However, determining the optimal order and addressing sensitivity to changes in data distribution remain challenging. To tackle these issues, this paper introduces an unsupervised Superpixelwise Multiscale Adaptive T-HOSVD (SmaT-HOSVD) method. Leveraging superpixel segmentation, the algorithm identifies homogeneous regions, facilitating the extraction of local features to enhance spatial contextual information within the image. Subsequently, T-HOSVD is adaptively applied to the obtained superpixel blocks for feature extraction and fusion across different scales. SmaT-HOSVD harnesses superpixel blocks and low-rank representations to extract 3D features, effectively capturing both spectral and spatial information of HSIs. By integrating optimal-rank estimation and multiscale fusion strategies, it acquires more comprehensive low-rank information and mitigates sensitivity to data variations. Notably, when trained on subsets comprising 2%, 1%, and 1% of the Indian Pines, University of Pavia, and Salinas datasets, respectively, SmaT-HOSVD achieves impressive overall accuracies of 93.31%, 97.21%, and 99.25%, while maintaining excellent efficiency. Future research will explore SmaT-HOSVD's applicability in deep-sea HSI classification and pursue additional avenues for advancing the field.

An efficient Transformer with neighborhood contrastive tokenization for hyperspectral images classification

The success of vision Transformers (ViTs) relies heavily on the self-attention mechanism, which requires support from appropriate patch tokenization. However, hyperspectral image (HSI) often suffer from significant noise distortions and spectral uncertainty, which result in unstable attention patterns and overfitting due to equivocal tokenization. In this paper, we propose a neighborhood contrastive tokenization task (NeiCoT) to learn compact, semantically meaningful, and context-sensitive tokens for efficient Transformer encoding. Specifically, we employ a predictor on patch embedding to maximize the mutual information between local individuals and their global average anchor. This encourages neighboring tokens’ relevance and active participation in feature learning. Next, we revise a token-level contrastive loss to align predictions with local individuals and distinguish them from other samples in a mini-batch to enhance tokens rich in contextual semantics. Furthermore, we apply a Gaussian weighting to the tokens’ contrastive loss to balance the neighborhood contribution. Finally, we propose a sequence-specific MAE framework with NeiCoT to achieve HSI representation, and additionally validate NeiCoT on a supervised Transformer backbone. The results demonstrate that NeiCoT consistently enhances the robustness and generalization of the Transformer, achieving accurate object recognition and boundary localization even with limited training samples. Our code will be available at https://github.com/zoegnov07/NeiCoT.

Hybrid brain tumor classification of histopathology hyperspectral images by linear unmixing and an ensemble of deep neural networks.

Hyperspectral imaging has demonstrated its potential to provide correlated spatial and spectral information of a sample by a non-contact and non-invasive technology. In the medical field, especially in histopathology, HSI has been applied for the classification and identification of diseased tissue and for the characterization of its morphological properties. In this work, we propose a hybrid scheme to classify non-tumor and tumor histological brain samples by hyperspectral imaging. The proposed approach is based on the identification of characteristic components in a hyperspectral image by linear unmixing, as a features engineering step, and the subsequent classification by a deep learning approach. For this last step, an ensemble of deep neural networks is evaluated by a cross-validation scheme on an augmented dataset and a transfer learning scheme. The proposed method can classify histological brain samples with an average accuracy of 88%, and reduced variability, computational cost, and inference times, which presents an advantage over methods in the state-of-the-art. Hence, the work demonstrates the potential of hybrid classification methodologies to achieve robust and reliable results by combining linear unmixing for features extraction and deep learning forclassification.

Hyperspectral image classification based on deep separable residual attention network

Hyperspectral image have rich spatial and spectral information, and how to fully extract and utilize the features of these two dimensions is a research hotspot in hyperspectral classification methods. At present, the unique convolutional operation and deep feature extraction structure of convolutional neural network enable them to have stronger feature representation capabilities and achieve good results in hyperspectral image classification. However, CNN methods do not assign different weights based on the importance of features in the feature extraction process, making it difficult to effectively utilize key features, and most importantly, using fixed shaped convolution kernel can easily overlook the differences between hyperspectral image features. A hyperspectral image classification method based on deep separable residual attention network is proposed to address the above issues. Firstly, to reduce the correlation between hyperspectral image data and minimize the interference of redundant information, principal component analysis is used to reduce the dimensionality of hyperspectral image. Secondly, a shallow feature extraction module is constructed, which can dynamically adjust the size of the receptive field according to the actual situation of the image, adaptively extract shallow features, and reduce the loss of original image features. Then, a depthwise separable residual attention mechanism module is proposed, based on which features are extracted. Starting from global and local features, contextual information on image features in channel and spatial domains is extracted. Finally, use a multi-scale feature fusion module to fully integrate feature maps at different scales. Using Indian Pines, Pavia University and Botswana as experimental datasets, the overall classification accuracy of this paper's method is 98.47 %, 98.70 %, 98.83 % with only 50, 50, 30 training samples per class. The Kappa coefficient is 98.25 %, 98.27 %, and 98.73 %, respectively. Compared with advanced methods, this method not only has higher classification accuracy, but also fully utilizes key features at various network levels.

Hyperspectral Image Classification Based on Multi-Scale Convolutional Features and Multi-Attention Mechanisms

Convolutional neural network (CNN)-based and Transformer-based methods for hyperspectral image (HSI) classification have rapidly advanced due to their unique characterization capabilities. However, the fixed kernel sizes in convolutional layers limit the comprehensive utilization of multi-scale features in HSI land cover analysis, while the Transformer’s multi-head self-attention (MHSA) mechanism faces challenges in effectively encoding feature information across various dimensions. To tackle this issue, this article introduces an HSI classification method, based on multi-scale convolutional features and multi-attention mechanisms (i.e., MSCF-MAM). Firstly, the model employs a multi-scale convolutional module to capture features across different scales in HSIs. Secondly, to enhance the integration of local and global channel features and establish long-range dependencies, a feature enhancement module based on pyramid squeeze attention (PSA) is employed. Lastly, the model leverages a classical Transformer Encoder (TE) and linear layers to encode and classify the transformed spatial–spectral features. The proposed method is evaluated on three publicly available datasets—Salina Valley (SV), WHU-Hi-HanChuan (HC), and WHU-Hi-HongHu (HH). Extensive experimental results have demonstrated that the MSCF-MAM method outperforms several representative methods in terms of classification performance.