Hyperspectral Image Classification Method Research Articles

Hyperspectral image classification based on superpixel merging and broad learning system

AbstractMost spectral–spatial classification methods for hyperspectral images (HSIs) can achieve satisfactory classification results. However, the common problem faced with these approaches is the need for a long training time and sufficient training samples. To address this issue, this study proposes an effective spectral–spatial HSI classification method based on superpixel merging, superpixel smoothing and broad learning system (SMS‐BLS). The newly introduced parameter‐free superpixel merging technique based on local modularity not only enhances the role of local spatial information in classification, but also maintains class boundary information as much as possible. In addition, the spectral and spatial information of HSIs is further fused during the superpixel smoothing process. As a result, with limited training samples, using merged and smoothed superpixels instead of pixels as input to the broad learning system significantly improves its classification performance. Moreover, the merged superpixels weaken the dependence of the classification results on the superpixel segmentation scale. The effectiveness of the proposed method was validated on three HSI benchmarks, namely Indian Pines, Pavia University and Salinas. Experimental and comparative results show the superiority of the method to other state‐of‐the‐art approaches in terms of overall accuracy and running time.

Factor annealing decoupling compositional training method for imbalanced hyperspectral image classification

Factor annealing decoupling compositional training method for imbalanced hyperspectral image classification

Cross-Scene Hyperspectral Image Classification Based on Graph Alignment and Distribution Alignment

A domain alignment-based hyperspectral image (HSI) classification method was designed to address the heterogeneity in resolution and band between the source domain and target domain datasets of cross-scene hyperspectral images, as well as the resulting reduction in common features. Firstly, after preliminary feature extraction, perform two domain alignment operations: image alignment and distribution alignment. Image alignment aims to align hyperspectral images of different bands or time points, ensuring that they are within the same spatial reference framework. Distribution alignment adjusts the distribution of features of samples of different categories in the feature space to reduce the distribution differences of the same type of features between two domains. Secondly, adjust the consistency of the two alignment methods to ensure that the features obtained through different alignment methods exhibit consistency in the feature space, thereby improving the comparability and reliability of the features. In addition, this method considers multiple losses in the model from different perspectives and makes comprehensive adjustments through a unified optimization process to more comprehensively capture and utilize the correlation information between data. Experimental results on Houston 2013 and Houston 2018 datasets can improve the hyperspectral prediction performance between datasets with different resolutions and bands, effectively solving the problems of high cost and limited training samples in HSI labeling and significantly improving cross-scene HSI classification performance.

Attention 3D central difference convolutional dense network for hyperspectral image classification.

Hyperspectral Images (HSI) classification is a challenging task due to a large number of spatial-spectral bands of images with high inter-similarity, extra variability classes, and complex region relationships, including overlapping and nested regions. Classification becomes a complex problem in remote sensing images like HSIs. Convolutional Neural Networks (CNNs) have gained popularity in addressing this challenge by focusing on HSI data classification. However, the performance of 2D-CNN methods heavily relies on spatial information, while 3D-CNN methods offer an alternative approach by considering both spectral and spatial information. Nonetheless, the computational complexity of 3D-CNN methods increases significantly due to the large capacity size and spectral dimensions. These methods also face difficulties in manipulating information from local intrinsic detailed patterns of feature maps and low-rank frequency feature tuning. To overcome these challenges and improve HSI classification performance, we propose an innovative approach called the Attention 3D Central Difference Convolutional Dense Network (3D-CDC Attention DenseNet). Our 3D-CDC method leverages the manipulation of local intrinsic detailed patterns in the spatial-spectral features maps, utilizing pixel-wise concatenation and spatial attention mechanism within a dense strategy to incorporate low-rank frequency features and guide the feature tuning. Experimental results on benchmark datasets such as Pavia University, Houston 2018, and Indian Pines demonstrate the superiority of our method compared to other HSI classification methods, including state-of-the-art techniques. The proposed method achieved 97.93% overall accuracy on the Houston-2018, 99.89% on Pavia University, and 99.38% on the Indian Pines dataset with the 25 × 25 window size.

Multi-dimensional deep dense residual networks and multiple kernel learning for hyperspectral image classification

To address the issues of inadequate feature expression capacity and poor adaptability of feature fusion in traditional hyperspectral image classification methods, a new approach to classification utilizing a multi-dimensional deep dense residual network and multiple kernel learning is suggested. First, we construct a deep dense residual structure to obtain the spectral, spatial and spectral-spatial features of the image, respectively. Then, the multiple kernel learning is employed to map the three extracted features into the high-dimensional space to achieve the adaptive fusion between features. Finally, multiple kernel support vector machine is adopted to accurately classify the multiple kernel fused features. By randomly selecting 10%, 1% and 1% of samples in each ground object category as training samples, our proposed method demonstrated an enhanced overall classification accuracy of 99.58%, 99.52% and 99.66% according to the experimental results conducted on three typical hyperspectral images. Comparing with some multiple kernel learning based methods, our method can obtain higher classification accuracy. So our method has better recognition ability for hyperspectral image features, and has strong adaptability and robustness.

Hyperspectral image classification method based on narrowing semantic gap convolutional neural network

ABSTRACT In recent years, numerous hyperspectral image classification methods based on convolutional neural network (CNN) have been proposed in recent years. However, the majority of CNN-based classification models fail to take into account the semantic gap between shallow and deep features, and instead, just sum up or splice these two types of features, which leads to subpar feature fusion. To address the above problems, the paper proposes a hyperspectral image classification method based on Narrowing Semantic Gap Convolutional Neural Network (NSGCNN). We developed the feature fusion sub-network, which consists of Feature Semantic Enhancement Module (FSEM), Spatial Detail Supplement Module (SDCM), and Cross-layer Feature Fusion Module (CFFM). FSEM extracts rich global semantic information based on multi-scale dilated convolution. Reverse attention mechanism is introduced in the SDCM to complement deep-layer features with fine spatial features like edges and textures. The fusing of hierarchical features is then achieved using CFFM. Additionally, we can regulate the model’s parameter count by regulating the number of output feature maps. Experimental results on three benchmark hyperspectral datasets demonstrate that NSGCNN outperforms other state-of-the-art methods in terms of classification performance and significantly outperforms them in terms of model complexity.

Spatial-Pooling-Based Graph Attention U-Net for Hyperspectral Image Classification

In recent years, graph convolutional networks (GCNs) have attracted increasing attention in hyperspectral image (HSI) classification owing to their exceptional representation capabilities. However, the high computational requirements of GCNs have led most existing GCN-based HSI classification methods to utilize superpixels as graph nodes, thereby limiting the spatial topology scale and neglecting pixel-level spectral–spatial features. To address these limitations, we propose a novel HSI classification network based on graph convolution called the spatial-pooling-based graph attention U-net (SPGAU). Specifically, unlike existing GCN models that rely on fixed graphs, our model involves a spatial pooling method that emulates the region-growing process of superpixels and constructs multi-level graphs by progressively merging adjacent graph nodes. Inspired by the CNN classification framework U-net, SPGAU’s model has a U-shaped structure, realizing multi-scale feature extraction from coarse to fine and gradually fusing features from different graph levels. Additionally, the proposed graph attention convolution method adaptively aggregates adjacency information, thereby further enhancing feature extraction efficiency. Moreover, a 1D-CNN is established to extract pixel-level features, striking an optimal balance between enhancing the feature quality and reducing the computational burden. Experimental results on three representative benchmark datasets demonstrate that the proposed SPGAU outperforms other mainstream models both qualitatively and quantitatively.

WFSS: weighted fusion of spectral transformer and spatial self-attention for robust hyperspectral image classification against adversarial attacks

The emergence of adversarial examples poses a significant challenge to hyperspectral image (HSI) classification, as they can attack deep neural network-based models. Recent adversarial defense research tends to establish global connections of spatial pixels to resist adversarial attacks. However, it cannot yield satisfactory results when only spatial pixel information is used. Starting from the premise that the spectral band is equally important for HSI classification, this paper explores the impact of spectral information on model robustness. We aim to discover potential relationships between different spectral bands and establish global connections to resist adversarial attacks. We design a spectral transformer based on the transformer structure to model long-distance dependency relationships among spectral bands. Additionally, we use a self-attention mechanism in the spatial domain to develop global relationships among spatial pixels. Based on the above framework, we further explore the influence of both spectral and spatial domains on the robustness of the model against adversarial attacks. Specifically, a weighted fusion of spectral transformer and spatial self-attention (WFSS) is designed to achieve the multi-scale fusion of spectral and spatial connections, which further improves the model’s robustness. Comprehensive experiments on three benchmarks show that the WFSS framework has superior defensive capabilities compared to state-of-the-art HSI classification methods.

Hyperspectral image classification based on local feature decoupling and hybrid attention SpectralFormer network

ABSTRACT Classification of hyperspectral image (HSI) is an essential step in various remote sensing tasks. Currently, transformer-based models have made significant contributions to HSI classification due to their remarkable ability to model the relation among long-range sequence data. However, HSI data exhibit strong nonlinear coupling and strong correlation, leading to a decrease in the classification performance of transformer-based models. The SpectralFormer network has achieved excellent performance in HSI classification, but its cross-layer adaptive fusion mechanism cannot effectively highlight critical information, affecting the representation ability of spectral-spatial features. To address the above issues, this paper proposes an HSI classification method based on local feature decoupling and SpectralFormer with a hybrid attention module (HAMSF) network. Specifically, this paper first uses the histogram of oriented gradient algorithm to preprocess HSI coupled data and achieve preliminary nonlinear decoupling of HSI data. Secondly, the hybrid attention module is embedded in a cross-layer adaptive fusion mechanism, generating refined neighbouring information before each information fusion to improve the spectral-spatial feature representation ability. Finally, the processed data are input into the HAMSF network for the final decoupling and classification of HSI. The experiment is carried out on three benchmark hyperspectral datasets, and the experimental results confirm that the proposed method has better classification performance compared to several transformer-based and classical HSI classification methods.

A hyperspectral image classification method based on feature enhancement and a hybrid deformable convolution network

ABSTRACT In recent years, some hyperspectral image (HSI) classification methods based on deep models have shown excellent performance. Most deep models receive three-dimensional (3D) block structures as input to extract spectral-spatial features from HSI data. However, the existence of background pixel information in 3D blocks affects the spectral-spatial feature representation ability. In addition, directly performing overall dimension reduction on high-dimensional spectral information lacks analysis of the original data, which weakens the constraint relationship between spectral data and increases the difficulty of feature extraction. Therefore, an HSI classification method based on feature enhancement and hybrid deformable convolution network is proposed. Firstly, a novel 3D block is constructed to enhance spectral-spatial feature representation ability by replacing background pixel information with non-background pixel information. Secondly, based on the analysis results of the original spectral data, this paper proposes a regional dimension reduction mechanism to enhance the constraint relationship between HSI data. Finally, the proposed method is applied to hybrid deformable convolution network for experimental verification. The experimental results of three standard datasets show that the classification effect of the proposed method is better than that of several classification methods, which proves the effectiveness of the proposed method.

Hyperspectral image classification using a double-branch hierarchical partial convolution network

ABSTRACT Recently, convolutional neural network (CNN) and Transformer have been successfully applied in hyperspectral image (HSI) classification. Despite their success, some challenges persist. Popular HSI classification methods typically employ only a single patch as the network input, which may not provide robust information for various datasets or within different scenarios of a specific dataset. Large-scale patches are usually conducive to classifying large-scale scenes, while small-scale patches excel at classifying boundary locations. In addition, the majority of methods do not distinguish between the central pixel and its adjacent pixels, although the information from the central position is crucial. In this paper, a novel double-branch hierarchical partial convolution network (DBHP-Net) is proposed to address the aforementioned challenges. Concretely, DBHP-Net employs a new convolution, named TConv, to emphasize mining the spatial-channel features of the central position. Then, the TConvFormer Block is built by adopting the advanced engineering techniques of Transformer. Furthermore, we construct our model in a hierarchical style. The double-branch structure with large-scale and small-scale patches provides complementary information from the dataset. Experimental results based on randomly selected samples and spatially disjoint samples of two benchmark datasets demonstrate that our method outperforms other state-of-the-art (SOTA) methods.

Relaxation linear regression with spectral–spatial constrained locality adaptive regularization for hyperspectral image classification

AbstractRecently, relaxation linear regression has received increasing attention in image analysis. However, the current relaxation linear regression methods fail to consider the local geometrical structure and spatial information while they are applied for hyperspectral image (HSI) classification. To address the above problems, this letter proposes a novel relaxation linear regression with spectral–spatial constrained locality adaptive regularization (SSLA‐RLR) method for HSI classification. The SSLA‐RLR method not only integrates the locality adaptive graph with relaxation linear regression to adaptively exploit the local geometrical structure for relieving the side effects of noise corruptions, but also takes the spatial correlations between each data point and its neighbours into consideration via spectral–spatial constraint. Finally, extensive experiments are conducted on three benchmark HSI datasets to demonstrate the effectiveness of the proposed method.

Effective hyperspectral image classification based on segmented PCA and 3D-2D CNN leveraging multibranch feature fusion

ABSTRACT We present an innovative hyperspectral image (HSI) classification method addressing challenges posed by closely spaced wavelength bands. Our approach combines 3D-2D convolutional neural networks (CNNs) with multi-branch feature fusion for improved spectral-spatial feature extraction. Using segmented principal component analysis (Seg-PCA), we reduce HSIs’ spectral dimensions into global and local intrinsic characteristics. The integration of 3D and 2D CNNs captures joint spectral-spatial features, while a multi-branch network extracts and merges diverse local features along the spectral dimension. Our method outperforms existing approaches, achieving remarkable accuracy of 99.27%, 100%, and 99.99% on Indian Pines, Salinas Scene, and University of Pavia datasets, respectively.

Hyperspectral Image Classification on Large-Scale Agricultural Crops: The Heilongjiang Benchmark Dataset, Validation Procedure, and Baseline Results

Over the past few decades, researchers have shown sustained and robust investment in exploring methods for hyperspectral image classification (HSIC). The utilization of hyperspectral imagery (HSI) for crop classification in agricultural areas has been widely demonstrated for its feasibility, flexibility, and cost-effectiveness. However, numerous coexisting issues in agricultural scenarios, such as limited annotated samples, uneven distribution of crops, and mixed cropping, could not be explored insightfully in the mainstream datasets. The limitations within these impractical datasets have severely restricted the widespread application of HSIC methods in agricultural scenarios. A benchmark dataset named Heilongjiang (HLJ) for HSIC is introduced in this paper, which is designed for large-scale crop classification. For practical applications, the HLJ dataset covers a wide range of genuine agricultural regions in Heilongjiang Province; it provides rich spectral diversity enriched through two images from diverse time periods and vast geographical areas with intercropped multiple crops. Simultaneously, considering the urgent demand of deep learning models, the two images in the HLJ dataset have 319,685 and 318,942 annotated samples, along with 151 and 149 spectral bands, respectively. To validate the suitability of the HLJ dataset as a baseline dataset for HSIC, we employed eight classical classification models in fundamental experiments on the HLJ dataset. Most of the methods achieved an overall accuracy of more than 80% with 10% of the labeled samples used for training. Furthermore, the advantages of the HLJ dataset and the impact of real-world factors on experimental results are comprehensively elucidated. The comprehensive baseline experimental evaluation and analysis affirm the research potential of the HLJ dataset as a large-scale crop classification dataset.

SSC-SFN: spectral-spatial non-local segment federated network for hyperspectral image classification with limited labeled samples

ABSTRACT Hyperspectral image (HSI) classification methods based on deep learning (DL) have performed well in numerous investigations. Although many modified superpixel-wise neural networks are utilized to enhance spatial information, their ability to mine spectral information in graph structures is insufficient. Moreover, single classifier approaches are unable to extract adequate spatial and spectral information simultaneously. For the classification of large-scale research areas, many works have relied on the use of a large number of labeled samples, leading to low efficiency and weak generalization. To address these issues, an effective spectral-spatial HSI classification approach based on spectral-spatial non-local segment federated network (SSC-SFN) was developed in this study. In this framework, deconvolution is employed to recover the data size, while the lost spatial information is replaced by up-pooling. The spectral dimensional features are updated through the generation of non-Euclidean graph structures and the non-local segment smoothing technique. The convolutional neural network and graph convolutional network techniques are coupled to exploit the available spectral and spatial structure information fully. Extensive experimental results obtained using four public benchmark datasets show that the classification accuracy of SSC-SFN can exceed 90% for large-scale HSIs with limited samples.

Superpixel-based multi-scale multi-instance learning for hyperspectral image classification

Superpixels can define meaningful local regions within a hyperspectral image (HSI) and have become the building blocks of various HSI classification methods. The superpixels in HSIs consist of multiple spectral pixels, sharing a similar structure with the data in multi-instance learning (MIL). However, the potential of MIL methods in the field of HSI classification has been rarely explored. In this paper, we propose the superpixel-based multi-scale multi-instance learning (MSMIL) framework, enhancing the superpixel representation with MIL for the first time. Segmenting the HSIs with superpixels, MSMIL converts the HSI classification into MIL problems and extracts superpixel representations via the MIL method, namely multi-instance factor analysis (MIFA). Compared with the existing methods focusing exclusively on the local information, MIFA utilizes the deviations from an overall generative model to describe the superpixels, retaining both the local and the global information. Moreover, MSMIL introduces multi-scale superpixels and a spectral–spatial decision fusion strategy for further refinement, where the results of multi-scale superpixel maps are weighted according to prediction certainty and spatial consistency. The proposed method is evaluated on four benchmark datasets and achieves competitive results. For example, MSMIL outperforms the comparison methods with a margin of 5% overall accuracy on the Indian Pines dataset, when 2% pixels are selected as the training set.

Conventional to Deep Ensemble Methods for Hyperspectral Image Classification: A Comprehensive Survey

Conventional to Deep Ensemble Methods for Hyperspectral Image Classification: A Comprehensive Survey

Feature Consistency-Based Prototype Network for Open-Set Hyperspectral Image Classification.

Hyperspectral image (HSI) classification methods have made great progress in recent years. However, most of these methods are rooted in the closed-set assumption that the class distribution in the training and testing stages is consistent, which cannot handle the unknown class in open-world scenes. In this work, we propose a feature consistency-based prototype network (FCPN) for open-set HSI classification, which is composed of three steps. First, a three-layer convolutional network is designed to extract the discriminative features, where a contrastive clustering module is introduced to enhance the discrimination. Then, the extracted features are used to construct a scalable prototype set. Finally, a prototype-guided open-set module (POSM) is proposed to identify the known samples and unknown samples. Extensive experiments reveal that our method achieves remarkable classification performance over other state-of-the-art classification techniques.

A Deep Stochastic Adaptive Fourier Decomposition Network for Hyperspectral Image Classification.

Deep learning-based hyperspectral image (HSI) classification methods have recently shown excellent performance, however, there are two shortcomings that need to be addressed. One is that deep network training requires a large number of labeled images, and the other is that deep network needs to learn a large number of parameters. They are also general problems of deep networks, especially in applications that require professional techniques to acquire and label images, such as HSI and medical images. In this paper, we propose a deep network architecture (SAFDNet) based on the stochastic adaptive Fourier decomposition (SAFD) theory. SAFD has powerful unsupervised feature extraction capabilities, so the entire deep network only requires a small number of annotated images to train the classifier. In addition, we use fewer convolution kernels in the entire deep network, which greatly reduces the number of deep network parameters. SAFD is a newly developed signal processing tool with solid mathematical foundation, which is used to construct the unsupervised deep feature extraction mechanism of SAFDNet. Experimental results on three popular HSI classification datasets show that our proposed SAFDNet outperforms other compared state-of-the-art deep learning methods in HSI classification.

A Hyperspectral Image Classification Method Based on Pyramid Feature Extraction With Deformable–Dilated Convolution

A Hyperspectral Image Classification Method Based on Pyramid Feature Extraction With Deformable–Dilated Convolution