Hyperspectral Image Classification Research Articles

Unleashing the full potential of hyperspectral imaging: Decoupled image and frequency-domain spatial–spectral framework

Hyperspectral image classification (HSIC) is a rapidly developing field that utilizes deep learning methods. However, the reliance on convolutional neural networks (CNNs) for spectral–spatial feature extraction presents certain limitations. Specifically, the use of the fixed-position convolutional kernels in CNNs hinders their ability to capture fully the spectral information around the spatial central pixel, thereby overlooking critical differences between features. To address this issue, a decoupled image- and frequency-domain spectral–spatial framework for HSIC was developed in this study. This method incorporates image- and frequency-domain-based multiscale learnable convolutional attention to refine the differentiating features of the different feature distributions. Additionally, a novel frequency-domain information enhancement module was designed to extract the structural shape and texture details under the semantic constraints of the frequency phase, complementing the image domain to improve the extracted feature maps. Furthermore, a simple and efficient hierarchical feature representation module was introduced to extract both local and global information effectively from the fused features. The experimental results obtained using three open datasets and a practical hyperspectral image of the Gaofen-5 satellite demonstrate that the proposed method outperforms other state-of-the-art HSIC methods.

Multi-view graph representation learning for hyperspectral image classification with spectral–spatial graph neural networks

Multi-view graph representation learning for hyperspectral image classification with spectral–spatial graph neural networks

QTN: Quaternion Transformer Network for Hyperspectral Image Classification

Numerous state-of-the-art transformer-based techniques with self-attention mechanisms have recently been demonstrated to be quite effective in the classification of hyperspectral images (HSIs). However, traditional transformer-based methods severely suffer from the following problems when processing HSIs with three dimensions: (1) processing the HSIs using 1D sequences misses the 3D structure information; (2) too expensive numerous parameters for hyperspectral image classification tasks; (3) only capturing spatial information while lacking the spectral information. To solve these problems, we propose a novel Quaternion Transformer Network (QTN) for recovering self-adaptive and long-range correlations in HSIs. Specially, we first develop a band adaptive selection module (BASM) for producing Quaternion data from HSIs. And then, we propose a new and novel quaternion self-attention (QSA) mechanism to capture the local and global representations. Finally, we propose a new and novel transformer method, <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">i.e</i> ., QTN by stacking a series of QSA for hyperspectral classification. The proposed QTN could exploit computation using Quaternion algebra in hypercomplex spaces. Extensive experiments on three public datasets demonstrate that the QTN outperforms the state-of-the-art vision transformers and convolution neural networks.

Hyperspectral and SAR Image Classification via Multiscale Interactive Fusion Network.

Due to the limitations of single-source data, joint classification using multisource remote sensing data has received increasing attention. However, existing methods still have certain shortcomings when faced with feature extraction from single-source data and feature fusion between multisource data. In this article, a method based on multiscale interactive information extraction (MIFNet) for hyperspectral and synthetic aperture radar (SAR) image classification is proposed. First, a multiscale interactive information extraction (MIIE) block is designed to extract meaningful multiscale information. Compared with traditional multiscale models, it can not only obtain richer scale information but also reduce the model parameters and lower the network complexity. Furthermore, a global dependence fusion module (GDFM) is developed to fuse features from multisource data, which implements cross attention between multisource data from a global perspective and captures long-range dependence. Extensive experiments on the three datasets demonstrate the superiority of the proposed method and the necessity of each module for accuracy improvement.

Deep Network With Irregular Convolutional Kernels and Self-Expressive Property for Classification of Hyperspectral Images.

This article presents a novel deep network with irregular convolutional kernels and self-expressive property (DIKS) for the classification of hyperspectral images (HSIs). Specifically, we use the principal component analysis (PCA) and superpixel segmentation to obtain a series of irregular patches, which are regarded as convolutional kernels of our network. With such kernels, the feature maps of HSIs can be adaptively computed to well describe the characteristics of each object class. After multiple convolutional layers, features exported by all convolution operations are combined into a stacked form with both shallow and deep features. These stacked features are then clustered by introducing the self-expression theory to produce final features. Unlike most traditional deep learning approaches, the DIKS method has the advantage of self-adaptability to the given HSI due to building irregular kernels. In addition, this proposed method does not require any training operations for feature extraction. Because of using both shallow and deep features, the DIKS has the advantage of being multiscale. Due to introducing self-expression, the DIKS method can export more discriminative features for HSI classification. Extensive experimental results are provided to validate that our method achieves better classification performance compared with state-of-the-art algorithms.

Spectral unmixing-based Arctic plant species analysis using a spectral library and terrestrial hyperspectral Imagery: A case study in Adventdalen, Svalbard

Remote sensing is an invaluable tool for monitoring the rapid changes in Arctic vegetation distribution caused by global warming. Although hyperspectral data consisting of contiguous spectral bands enables the quantitative analysis of remote sensing data, mapping Arctic vegetation using hyperspectral remote sensing remains challenging due to the difficulty in acquiring data from the region. Additionally, the mixed pixel issue, which represents a mixture of more than one plant species due to low-spatial resolution, hinders accurate mapping of Arctic vegetation. To address these limitations, we collected hyperspectral information on the dominant plant species, such as shrubs and graminoids, in Adventdalen, Svalbard, and investigated effective methods for mapping Arctic vegetation using spectral unmixing. First, labeled datasets were constructed for Arctic plant species by extracting pixel data from terrestrial hyperspectral images. A spectral library was developed using the labeled datasets and used for spectral unmixing as endmembers. We employed three established classifiers, random forest, support vector machine, and one-dimensional convolutional neural network (1D-CNN) for hyperspectral image classification. Subsequently, we quantitatively and qualitatively compared the classification performances of these machine learning-based classifiers to determine the optimal classification method for validation purposes. The first derivative spectra of smoothed reflectance (RS,FD) and the 1D-CNN classifier were used to identify ground truth by classifying pixels of Arctic vegetation because they achieved the highest statistical accuracies of 0.9892 (Kappa = 0.9880) and 0.9352 (Kappa = 0.9280) for the two independent test sets and produced the most accurate vegetation maps. The spectral library using the RS,FD spectrum showed high spectral discriminability and potential for estimating the abundance of classes from simulated mixed pixels, resulting in good agreement with the ground truth. Accordingly, our findings provide valuable insights and references for large-scale mapping studies using remote sensing data and offer effective monitoring strategies for Arctic vegetation changes in response to climate change.

Multiple vision architectures-based hybrid network for hyperspectral image classification

Multiple vision architectures-based hybrid network for hyperspectral image classification

Hyperspectral Image Classification Promotion Using Dynamic Convolution Based on Structural Re-Parameterization

In a hyperspectral image classification (HSIC) task, manually labeling samples requires a lot of manpower and material resources. Therefore, it is of great significance to use small samples to achieve the HSIC task. Recently, convolutional neural networks (CNNs) have shown remarkable performance in HSIC, but they still have some areas for improvement. (1) Convolutional kernel weights are determined through initialization and cannot be adaptively adjusted based on the input data. Therefore, it is difficult to adaptively learn the structural features of the input data. (2) The convolutional kernel size is single per layer, which leads to the loss of local information for a large convolutional kernel or global information for a small convolutional kernel. In order to solve the above problems, we propose a plug-and-play method called dynamic convolution based on structural re-parameterization (DCSRP). The contributions of this method are as follows. Firstly, compared with traditional convolution, dynamic convolution is a non-linear function, so it has more representation power. In addition, it can adaptively capture the contextual information of input data. Secondly, the large convolutional kernel and the small convolutional kernel are integrated into a new large convolutional kernel. The large convolutional kernel shares the advantages of the two convolution kernels, which can capture global information and local information at the same time. The results in three publicly available HSIC datasets show the effectiveness of the DCSRP.

Adaptive Multi-Feature Fusion Graph Convolutional Network for Hyperspectral Image Classification

Graph convolutional networks (GCNs) are a promising approach for addressing the necessity for long-range information in hyperspectral image (HSI) classification. Researchers have attempted to develop classification methods that combine strong generalizations with effective classification. However, the current HSI classification methods based on GCN present two main challenges. First, they overlook the multi-view features inherent in HSIs, whereas multi-view information interacts with each other to facilitate classification tasks. Second, many algorithms perform a rudimentary fusion of extracted features, which can result in information redundancy and conflicts. To address these challenges and exploit the strengths of multiple features, this paper introduces an adaptive multi-feature fusion GCN (AMF-GCN) for HSI classification. Initially, the AMF-GCN algorithm extracts spectral and textural features from the HSIs and combines them to create fusion features. Subsequently, these three features are employed to construct separate images, which are then processed individually using multi-branch GCNs. The AMG-GCN aggregates node information and utilizes an attention-based feature fusion method to selectively incorporate valuable features. We evaluated the model on three widely used HSI datasets, i.e., Pavia University, Salinas, and Houston-2013, and achieved accuracies of 97.45%, 98.03%, and 93.02%, respectively. Extensive experimental results show that the classification performance of the AMF-GCN on benchmark HSI datasets is comparable to those of state-of-the-art methods.

A Dual-Attention Deep Discriminative Domain Generalization Model for Hyperspectral Image Classification

Recently, hyperspectral image classification has made great progress with the development of convolutional neural networks. However, due to the challenges of distribution shifts and data redundancies, the classification accuracy is low. Some existing domain adaptation methods try to mitigate the distribution shifts by training source samples and some labeled target samples. However, in practice, labeled target domain samples are difficult or even impossible to obtain. To solve the above challenges, we propose a novel dual-attention deep discriminative domain generalization framework (DAD3GM) for cross-scene hyperspectral image classification without training the labeled target samples. In DAD3GM, we mainly design two blocks: dual-attention feature learning (DAFL) and deep discriminative feature learning (DDFL). DAFL is designed to extract spatial features by multi-scale self-attention and extract spectral features by multi-head external attention. DDFL is further designed to extract deep discriminative features by contrastive regularization and class discrimination regularization. The combination of DAFL and DDFL can effectively reduce the computational time and improve the generalization performance of DAD3GM. The proposed model achieves 84.25%, 83.53%, and 80.63% overall accuracy on the public Houston, Pavia, and GID benchmarks, respectively. Compared with some classical and state-of-the-art methods, the proposed model achieves optimal results, which reveals its effectiveness and feasibility.

A comparative analysis of various activation functions and optimizers in a convolutional neural network for hyperspectral image classification

A comparative analysis of various activation functions and optimizers in a convolutional neural network for hyperspectral image classification

S2EFT: Spectral-Spatial-Elevation Fusion Transformer for hyperspectral image and LiDAR classification

With the innovation of sensors, communication, computer and other technologies, remote sensing (RS) imaging methods show a diversification trend. At present, relying on a variety of earth observation platforms, more multi-source RS data can be obtained, which overcomes the inadequacy and incompleteness of the new earth observation information of single source RS image. In this paper, a classification method of hyperspectral images (HSI) and LiDAR data based on spectral-spatial-elevation fusion Transformer (S2EFT) framework is proposed. The Transformer framework is introduced into the task of multi-source RS image classification. Two simple but effective modules, spatial information recognition module and sliding group spectral embedding module, are added to the proposed framework, and the Patch form commonly used in traditional convolutional neural networks (CNNs) is used as the input. It effectively solves the shortcomings of Transformer’s insufficient attention to local information, reduces redundant spatial information, enhances the transmission of information, and fully integrates multidimensional information of pixels in same position. Experiments on three real data sets are compared with existing methods, and the best results are obtained by the proposed methods. The source code can be downloaded from https://github.com/SYFYN0317/S2EFT.

Enhanced Hyperspectral Image Classification Through Pretrained CNN Model for Robust Spatial Feature Extraction

Enhanced Hyperspectral Image Classification Through Pretrained CNN Model for Robust Spatial Feature Extraction

TBTA-D2Net: a novel hyperspectral image classification method based on triple-branch ternary-attention mechanism and Dense2Net

ABSTRACTRecently, there has been a growing interest in the hyperspectral image (HSI) classification methods that employ deep learning techniques in small sample cases. To address issues with network degradation and enhance the extraction of discriminative HSI features, this article proposes a TBTA-D2Net network utilizing a triple-branch ternary-attention mechanism and Dense2Net. Furthermore, a new deep model optimizer named Adan is introduced to improve the training speed of the network model. This article takes spatial information as a two-dimensional vector, extracting spectral features as well as spatial-X and spatial-Y features separately in three branches. Each branch includes a Dense2Net bottleneck module and an attention module. Classification is achieved by fusing the features extracted from the three branches. Experimental results on four public datasets indicate that TBTA-D2Net can achieve competitive results over state-of-the-art methods. The code is available at https://github.com/TeresaTing/TBTA-D2Net.

Classification of hyperspectral image by preprocessing method based relation network

ABSTRACTHyperspectral image classification involves assigning a land cover class to each pixel of a hyperspectral image (HSI). Recently, several classification methods based on deep learning with spatial information from HSI have been proposed. These methods typically demonstrate strong classification capabilities. However, spatial information often comprises complex data, making it challenging to extract valuable information. To address this issue, this paper proposes a novel image preprocessing method for classification based on metric learning. This method aims to reduce the complexity of spatial information to facilitate network learning. First, the method assesses the similarity between the central pixel and other pixels in each image patch using a relation network. It replaces pixels that are dissimilar to the central pixel with pixels that are similar to the central pixel to simplify the image patch. This approach reduces interference information in the image patch, allowing the network to focus more on the central pixel features that represent category information during the classification task. This greatly reduces the complexity of the required network, the difficulty of the network learning, and the number of training samples needed. Experimental results on three popular hyperspectral image datasets demonstrate that the proposed preprocessing method significantly improves classification accuracy compared to the baseline network. The study also compares our method to the state-of-the-art attention mechanism CNN, demonstrating how our method excels at hyperspectral classification in data-smoothing and feature preservation.

TransHSI: A Hybrid CNN-Transformer Method for Disjoint Sample-Based Hyperspectral Image Classification

Hyperspectral images’ (HSIs) classification research has seen significant progress with the use of convolutional neural networks (CNNs) and Transformer blocks. However, these studies primarily incorporated Transformer blocks at the end of their network architectures. Due to significant differences between the spectral and spatial features in HSIs, the extraction of both global and local spectral–spatial features remains incomplete. To address this challenge, this paper introduces a novel method called TransHSI. This method incorporates a new spectral–spatial feature extraction module that leverages 3D CNNs to fuse Transformer to extract the local and global spectral features of HSIs, then combining 2D CNNs and Transformer to capture the local and global spatial features of HSIs comprehensively. Furthermore, a fusion module is proposed, which not only integrates the learned shallow and deep features of HSIs but also applies a semantic tokenizer to transform the fused features, enhancing the discriminative power of the features. This paper conducts experiments on three public datasets: Indian Pines, Pavia University, and Data Fusion Contest 2018. The training and test sets are selected based on a disjoint sampling strategy. We perform a comparative analysis with 11 traditional and advanced HSI classification algorithms. The experimental results demonstrate that the proposed method, TransHSI algorithm, achieves the highest overall accuracies and kappa coefficients, indicating a competitive performance.

Hyperspectral discrimination of ginseng variety and age from Changbai Mountain area

BackgroundThe efficacy and market value of Panax ginseng Meyer are significantly influenced by its diversity and age. Traditional identification methods are prone to subjective biases and necessitate the use of destructive sample processing, leading to the loss and wastage of ginseng. Consequently, non-destructive in-situ identification has emerged as a crucial subject of interest for both researchers and the ginseng industry. The advancement of technology and the expansion of research have introduced spectral technology and image processing technology as novel approaches and concepts for non-destructive in-situ identification. MethodsHyperspectral imaging (HSI) is a methodology that combines conventional spectroscopy and imaging to acquire comprehensive spectral and spatial data from various samples. In this study, we investigated the use of Support Vector Machine (SVM) and Spectral Angle Mapper (SAM) classifier algorithms, in conjunction with HSI classification technology, for quasi-Artificial Intelligence (quasi-AI) ginseng identification. To enhance the hyperspectral images prior to SVM classification, we compared the efficacy of Principal Component Analysis (PCA), Minimum Noise Fraction (MNF), and Independent Component Analysis (ICA). ResultsThe classification of ginseng based on age was accomplished through the utilization of Radial Basis Function (RBF) kernel SVM and SAM algorithm, which was trained on feature enhanced images. The classification of WMG, MCG, and GG is primarily based on age, with the endmember spectrum serving as the foundation for SAM and SVM. ConclusionThe “endmember spectrum set” derived from the classification outcomes can serve as the “mutation point” for identifying ginseng of different ages.

Domain-Invariant Feature and Generative Adversarial Network Boundary Enhancement for Multi-Source Unsupervised Hyperspectral Image Classification

Hyperspectral image (HIS) classification, a crucial component of remote sensing technology, is currently challenged by edge ambiguity and the complexities of multi-source domain data. An innovative multi-source unsupervised domain adaptive algorithm (MUDA) structure is proposed in this work to overcome these issues. Our approach incorporates a domain-invariant feature unfolding algorithm, which employs the Fourier transform and Maximum Mean Discrepancy (MMD) distance to maximize invariant feature dispersion. Furthermore, the proposed approach efficiently extracts intraclass and interclass invariant features. Additionally, a boundary-constrained adversarial network generates synthetic samples, reinforcing the source domain feature space boundary and enabling accurate target domain classification during the transfer process. Furthermore, comparative experiments on public benchmark datasets demonstrate the superior performance of our proposed methodology over existing techniques, offering an effective strategy for hyperspectral MUDA.

A multi-range spectral-spatial transformer for hyperspectral image classification

In recent years, the transformer-based approach becoming a hot research topic in hyperspectral image (HSI) classification tasks. However, most of these studies have focused on optimizing the model framework in pursuit of high-accuracy classification, with little attention to the composition of the input token sequence as an important factor affecting the performance of the transformer. Therefore, this paper further explores a novel token structure to strengthen the Transformer's performance for HSI classification tasks, based on which a Multi-Range Spectral-Spatial Transformer (MRSST) framework is developed. Specifically, a convolutional feature pre-encoder with two branches is designed to extract shallow features for each spectral channel separately. Then, a token generator is introduced to combine the shallow features with the raw spectral information to yield the token sequences with multi-range information. Finally, the tokens are input into the transformer encoders enhanced by a module that strengthens the information exchange between its mid-range and short-range semantic features. Experiments conducted on three well-known hyperspectral datasets demonstrate that the proposed multi-range composite token sequences and information exchange mechanisms significantly enhance the transformer's performance. Codes are released at: https://github.com/HyperSystemAndImageProc/Multi-Range-Spectral-Spatial-Transformer.

An efficient joint framework assisted by embedded feature smoother and sparse skip connection for hyperspectral image classification

Deep neural network (DNN) methods play an essential role in hyperspectral classification. However, the massive parameters and vast computing overhead of DNN needs to be reduced when facing the deployment with limited storage and computing resources for real-time response applications, especially considering the high dimensionality of hyperspectral image. So, applying dimension reduction (DR) methods is a crucial pre-processing method in various studies. Still, most of them ignore the feature restoration after the data transformation by DR. In neural networks, many works still involve sophisticated skip connections and dense feature reuse, which can lead to feature redundancy and increase computational complexity, especially when DR methods have been applied first. Motivated by these issues, an efficient joint framework assisted by embedded feature smoother (FS) and sparse skip connection (SSC) is proposed in this article. Instead of directly feeding DR data into the subsequent network, we embedded a computing-cheap FS based on isotropic total variation to restore and enhance the spatial features. Furthermore, we proposed a SSC 3D convolution neural network to complete spatial–spectral feature representation and classification. The SSC is embodied in the design of log2n-skip connection to concatenate feature maps instead of dense connection, pruning the number of channels and reducing the model parameters. Experimental results show that the embedded FS significantly improves classification accuracy and is superior to other processing methods. Our framework offers much superior to other state-of-the-art deep learning-based methods considering classification performance and lightweight aspects, especially when using few training samples. Moreover, considering detailed processing steps, our framework has a competitively cheaper time consumption.