Hyperspectral Image Classification Method Research Articles

Diverse-Region Hyperspectral Image Classification via Superpixelwise Graph Convolution Technique

In this paper, a diverse-region hyperspectral image classification (DRHy) method is proposed by considering both irregularly local pixels and globally contextual connections between pixels. Specifically, the proposed method is operated on non-Euclidean graphs, which are constructed by superpixel segmentation methods for diverse regions to cluster irregularly local-region pixels. In addition, the dimensionality reduction method is employed to alleviate the curse of dimensionality problem with a lower computational burden, generating more representative data with the input graph features. In this context, it then constructs a superpixelwise Chebyshev polynomial graph convolution network (ChebyNet) to aggregate global-region superpixels. Benefiting from different superpixel numbers of segmentations, we construct different graph structures, and multiple classification results are obtained, which brings more opportunities to represent the hyperspectral data correctly. Then, all the diverse-region results are further fused by a majority voting technique to improve the final performance. Finally, numerical experiments on two benchmark datasets are provided to demonstrate the superiority of the proposed DRHy-ChebyNet method to the other state-of-the-art methods.

Multi-feature fusion: Graph neural network and CNN combining for hyperspectral image classification

Due to its impressive representation power, the graph convolutional network (GCN) has attracted increasing attention in the hyperspectral image (HSI) classification. However, the most of available GCN-based methods for HSI classification utilize superpixels as graph nodes, which ignore the pixel-wise spectral-spatial features. To overcome the issues, we propose a novel multi-feature fusion network (MFGCN), where two different convolutional networks, i.e., multi-scale GCN and multi-scale convolutional neural network (CNN), are utilized in two branches, separately. The multi-scale superpixel-based GCN can reduce the computing power requirements, deal with the problem of labeled deficiency, and refine the multi-scale spatial features from HSI. The multi-scale CNN can extract the multi-scale pixel-wise local features for HSI classification. Furthermore, we introduced a 1D CNN to extract the spectral features for superpixels (nodes), which is different from most existing methods. Finally, a concatenate operation is employed to fuse the complementary multi-scale features. In comparison with the state-of-the-art models on three datasets, the proposed method achieves superior experimental results and outperforms competitive methods.

Hyperspectral Image Classification with a Multiscale Fusion-Evolution Graph Convolutional Network Based on a Feature-Spatial Attention Mechanism

Convolutional neural network (CNN) has achieved excellent performance in the classification of hyperspectral images (HSI) due to its ability to extract spectral and spatial feature information. However, the conventional CNN model does not perform well in regions with irregular geometric appearances. The recently proposed graph convolutional network (GCN) has been successfully applied to the analysis of non-Euclidean data and is suitable for irregular image regions. However, conventional GCN has problems such as very high computational cost on HSI data and cannot make full use of information in the image spatial domain. To this end, this paper proposes a multi-scale fusion-evolution graph convolutional network based on the feature-spatial attention mechanism (MFEGCN-FSAM). Our model enables the graph to be automatically evolved during the graph convolution process to produce more accurate embedding features. We have established multiple local and global input graphs to utilize the multiscale spectral and spatial information of the image. In addition, this paper designs a feature-spatial attention module to extract important features and structural information from the graph. The experimental results on four typical datasets show that the MFEGCN-FSAM proposed in this paper has better performance than most existing HSI classification methods.

Hyperspectral Image Classification Based on Class-Incremental Learning with Knowledge Distillation

By virtue of its large-covered spatial information and high-resolution spectral information, hyperspectral images make lots of mapping-based fine-grained remote sensing applications possible. However, due to the inconsistency of land-cover types between different images, most hyperspectral image classification methods keep their effectiveness by training on every image and saving all classification models and training samples, which limits the promotion of related remote sensing tasks. To deal with the aforementioned issues, this paper proposes a hyperspectral image classification method based on class-incremental learning to learn new land-cover types without forgetting the old ones, which enables the classification method to classify all land-cover types with one final model. Specially, when learning new classes, a knowledge distillation strategy is designed to recall the information of old classes by transferring knowledge to the newly trained network, and a linear correction layer is proposed to relax the heavy bias towards the new class by reapportioning information between different classes. Additionally, the proposed method introduces a channel attention mechanism to effectively utilize spatial–spectral information by a recalibration strategy. Experimental results on the three widely used hyperspectral images demonstrate that the proposed method can identify both new and old land-cover types with high accuracy, which proves the proposed method is more practical in large-coverage remote sensing tasks.

LWIR hyperspectral image classification based on a temperature-emissivity residual network and conditional random field model

ABSTRACT Long-wave infrared (LWIR) hyperspectral images can provide temperature and emissivity information, and have unique advantages in classification tasks, especially in the fields of minerals, vegetation, and man-made materials. However, LWIR image acquisition is greatly affected by the atmosphere, which results in obvious spectral noise. Meanwhile, field-based and aerial thermal infrared hyperspectral images are faced with problems such as spatial heterogeneity and salt-and-pepper noise, which results in the traditional spectral-based and object-oriented classification methods performing poorly in the classification task. In this paper, using the emissivity spectra and land surface temperature inversed from LWIR hyperspectral images, a classification framework based on a temperature-emissivity residual network and conditional random field model (TERN-CRF) is proposed. In this method, the residual network is designed to extract and fuse in-depth spectral and local spatial features, and the conditional random field (CRF) model incorporates the spatial-contextual information by the use of the temperature information, to improve the problem of voids and isolated regions in the classification map. To validate the performance of the proposed method, three LWIR datasets acquired by the Hyper-Cam sensor were used in the experiments: an aerial dataset and two field-based datasets of minerals and leaves. The experimental classification results obtained using the LWIR datasets confirm the performance of the proposed TERN-CRF classification method, which is shown to be superior to the traditional hyperspectral image classification methods and a CNN-based method. Furthermore, the CRF post-processing part can effectively alleviate the isolated regions in the classification map.

DEEP FEATURE EXTRACTION BASED ON DYNAMIC GRAPH CONVOLUTIONAL NETWORKS FOR ACCELERATED HYPERSPECTRAL IMAGE CLASSIFICATION

Abstract. Deep learning has achieved impressive results on hyperspectral images (HSIs) classification. Among them, supervised learning convolutional neural networks (CNNs) and semi-supervised learning graph neural networks (GNNs) are the two main network frameworks. However, 1) the supervised learning CNN faces the problem of high model time complexity as the number of network layers deepens; 2) the semi-supervised learning GNN faces the problem of high spatial complexity due to the computation of adjacency relations. In this paper, a novel dynamic graph convolutional HSI classification method is proposed, which is called dynamic graph convolutional networks (DGCNet). We first obtain two classification features by implementing flattening and global average pooling operation on the results of the convolution layer, which fully exploits the spatial-spectral information contained in the hyperspectral data. Then the dynamic graph convolution module is applied to extract the intrinsic structural information of each patch. Finally, HSI is classified based on spatial, spectral and structural features. DGCNet uses three branches to process multiple features of HSI in parallel and is trained in a supervised learning manner. In addition, DropBlock and label smoothing regularization techniques are applied to further improve the generalization capability of the model. Comparative experiments show that our proposed algorithm is comparable with the state-of-the art supervised learning models in terms of accuracy while also significantly outperforming in terms of time.

Integrating Hybrid Pyramid Feature Fusion and Coordinate Attention for Effective Small Sample Hyperspectral Image Classification

In recent years, hyperspectral image (HSI) classification (HSIC) methods that use deep learning have proved to be effective. In particular, the utilization of convolutional neural networks (CNNs) has proved to be highly effective. However, some key issues need to be addressed when classifying hyperspectral images (HSIs), such as small samples, which can influence the generalization ability of the CNNs and the HSIC results. To address this problem, we present a new network that integrates hybrid pyramid feature fusion and coordinate attention for enhancing small sample HSI classification results. The innovative nature of this paper lies in three main areas. Firstly, a baseline network is designed. This is a simple hybrid 3D-2D CNN. Using this baseline network, more robust spectral-spatial feature information can be obtained from the HSI. Secondly, a hybrid pyramid feature fusion mechanism is used, meaning that the feature maps of different levels and scales can be effectively fused to enhance the feature extracted by the model. Finally, coordinate attention mechanisms are utilized in the network, which can not only adaptively capture the information of the spectral dimension, but also include the direction-aware and position sensitive information. By doing this, the proposed CNN structure can extract more useful HSI features and effectively be generalized to test samples. The proposed method was shown to obtain better results than several existing methods by experimenting on three public HSI datasets.

Classification of hyperspectral image based on dual-branch feature interaction network

ABSTRACT Hyperspectral image (HSI) classification method based on convolutional neural network (CNN) has been widely used and has achieved remarkable results. However, the convolution operation is good at extracting local features, it is difficult to capture the global representation. Recently, Vision Transformer (ViT) shines with cascading self-attention modules that can aggregate global representation between patch embeddings. The self-attention module cascaded in ViT can capture long-distance feature dependencies, but it is prone to ignore the local feature details. In order to address those series of problems, this article proposes a dual-branch feature interaction (DBFI) network based on CNN and ViT, which allows global information and local feature to fully interact to enhance feature representation capabilities. To resolve the problem of the semantic gap between the two branches, a feature interaction unit (FIU) based on the attention mechanism is designed as an information interactive bridge to connect the two branches. The features are branched from ViT to CNN through the upsampling stream to enhance the global perception of the network. Similarly, the down-sampling stream is used as a coupling to connect the CNN branch to the Transformer branch to supplement the local detailed features of the network. In addition, the channel attention and spectral attention in FIU can further emphasize the distinguishability of HSI features and improve the performance of network. Experiments are conducted on three benchmark HSI datasets demonstrate that the DBFI network is better than other classification methods.

Convolutional Neural Network-Based Cross-Media Semantic Matching and User Adaptive Satisfaction Analysis Model.

In this paper, an in-depth study of cross-media semantic matching and user adaptive satisfaction analysis model is carried out based on the convolutional neural network. Based on the existing convolutional neural network, this paper uses rich information. The spatial correlation of cross-media semantic matching further improves the classification accuracy of hyperspectral images and reduces the classification time under user adaptive satisfaction complexity. Aiming at the problem that it is difficult for the current hyperspectral image classification method based on convolutional neural network to capture the spatial pose characteristics of objects, the problem is that principal component analysis ignores some vital information when retaining a few components. This paper proposes a polymorphism based on extension Attribute Profile Feature (EMAP) Stereo Capsule Network Model for Hyperspectral Image Classification. To ensure the model has good generalization performance, a new remote sensing image Pan sharpening algorithm based on convolutional neural network is proposed, which increases the model's width to extract the feature information of the image and uses dilated instead of traditional convolution. The experimental results show that the algorithm has good generalization while ensuring self-adaptive satisfaction.

Hyperspectral image classification method based on M-3DCNN-Attention

Hyperspectral image (HSI) classification methods based on three-dimensional convolutional neural network (3DCNN) have problems of overfitting the in-sample training process and difficulty in highlighting the role of discriminant features, which reduce the classification accuracy. To solve the above problems, an HSI classification method based on M-3DCNN-Attention is proposed. First, the Mixup algorithm is used to construct HSI virtual samples to expand the original data set. The sample size of the expanded data set is twice that of the original data set, which greatly alleviates the overfitting phenomenon caused by the small sample of HSI. Second, the structure of 3DCNN is improved. A convolutional block attention module (CBAM) is added between each 3D convolutional layer and ReLU layer, and a total of three CBAMs are used so as to highlight the discriminant features in spectral and spatial dimensions of HSI and suppress the nondiscriminant features. Finally, the spectral–spatial features are transferred to the Softmax classifier to obtain the final classification results. The comparative experiments are conducted on three hyperspectral data sets (Indian Pines, University of PaviaU, and Salinas), and the overall accuracy of M-3DCNN-Attention is 99.90%, 99.93%, and 99.36%, respectively, which is better than the comparative methods.

Optimized kernel Nonparametric Weighted Feature Extraction for Hyperspectral Image Classification

Hyperspectral image (HSI) classification is an essential means of the analysis of remotely sensed images. Remote sensing of natural resources, astronomy, medicine, agriculture, food health, and many other applications are examples of possible applications of this technique. Since hyperspectral images contain redundant measurements, it is crucial to identify a subset of efficient features for modeling the classes. Kernel-based methods are widely used in this field. In this paper, we introduce a new kernel-based method that defines Hyperplane more optimally than previous methods. The presence of noise data in many kernel-based HSI classification methods causes changes in boundary samples and, as a result, incorrect class hyperplane training. We propose the optimized kernel non-parametric weighted feature extraction for hyperspectral image classification. KNWFE is a kernel-based feature extraction method, which has promising results in classifying remotely-sensed image data. However, it does not take the closeness or distance of the data to the target classes. Solving the problem, we propose optimized KNWFE, which results in better classification performance. Our extensive experiments show that the proposed method improves the accuracy of HSI classification and is superior to the state-of-the-art HIS classifiers.

Pyramidal and conditional convolution attention network for hyperspectral image classification using limited training samples

ABSTRACT Hyperspectral image classification methods based on convolutional neural network (CNN) and attention mechanism have obtained satisfactory results. However, the currently applied convolution operations treat all inputs equally, and the multiscale information extraction of hyperspectral data is not sufficient. The used attention mechanism is also one-dimensional, two-dimensional, or a simple combination of them. Alongside these, the high-precision classification models rely on many training samples. Based on these, we propose a pyramidal and conditional convolution attention network (PCAN), which can fully and effectively capture the information of hyperspectral input data through convolution kernels with different weights and different scales. And for the first time, the attention of three-dimensional weights is applied to hyperspectral data, and satisfactory results are achieved on minimal training sets. Specifically, PCAN is composed of spectral branch and spatial branch. The spectral branch uses conditionally parameterized convolutions (CondConv) to customize the convolution kernels according to the input features. It directly generates three-dimensional weights to quantify the importance of the entire feature maps. The spatial branch uses the pyramidal convolution (PyConv), which includes convolution kernels of different spatial sizes and depths and can capture different levels of detailed information in hyperspectral images. At the same time, competition and cooperation between different channels are captured through gating adaptation and channel normalization. Experiments on three well-known hyperspectral data sets show that our model achieves higher accuracy and less training time than other advanced methods, especially with fewer training samples.

Hyperspectral Data Classification Algorithm considering Spatial Texture Features

As a cutting-edge technology, hyperspectral remote sensing has been widely applied in many fields, including agricultural production, mineral identification, target detection, disaster warning, military reconnaissance, and urban planning. The collected hyperspectral data have high spectral resolution and spatial resolution and are characterized by a large amount of information, redundancy, and high dimension. At the same time, there is a strong correlation between the bands. Therefore, hyperspectral data not only provides rich information but also brings great challenges for subsequent processing. Hyperspectral image classification is a hot issue in remote sensing information processing. Traditional hyperspectral remote sensing image classification methods only use the spectral features of the image without considering the spatial features of each pixel in the hyperspectral remote sensing image. In this paper, a hyperspectral image classification method is proposed not only considering spectral features but also considering texture features. This method jointly considers both these features. Firstly, six texture features contributing a lot to each pixel of hyperspectral remote sensing image are extracted by using a gray level cooccurrence matrix, and then, the spectral features of each pixel in neighbor are combined to form the texture-spectral features. Finally, the classification experiment of the Indian Pines and Pavia University scene is carried out based on a support vector machine and extreme random tree algorithm, and the obtained results show that the proposed method achieves higher classification performance than the traditional method.

Mixed Structure with 3D Multi-Shortcut-Link Networks for Hyperspectral Image Classification

A hyperspectral image classification method based on a mixed structure with a 3D multi-shortcut-link network (MSLN) was proposed for the features of few labeled samples, excess noise, and heterogeneous homogeneity of features in hyperspectral images. First, the spatial–spectral joint features of hyperspectral cube data were extracted through 3D convolution operation; then, the deep network was constructed and the 3D MSLN mixed structure was used to fuse shallow representational features and deep abstract features, while the hybrid activation function was utilized to ensure the integrity of nonlinear data. Finally, the global self-adaptive average pooling and L-softmax classifier were introduced to implement the terrain classification of hyperspectral images. The mixed structure proposed in this study could extract multi-channel features with a vast receptive field and reduce the continuous decay of shallow features while improving the utilization of representational features and enhancing the expressiveness of the deep network. The use of the dropout mechanism and L-softmax classifier endowed the learned features with a better generalization property and intraclass cohesion and interclass separation properties. Through experimental comparative analysis of six groups of datasets, the results showed that this method, compared with the existing deep-learning-based hyperspectral image classification methods, could satisfactorily address the issues of degeneration of the deep network and “the same object with distinct spectra, and distinct objects with the same spectrum.” It could also effectively improve the terrain classification accuracy of hyperspectral images, as evinced by the overall classification accuracies of all classes of terrain objects in the six groups of datasets: 97.698%, 98.851%, 99.54%, 97.961%, 97.698%, and 99.138%.

A novel global-local block spatial-spectral fusion attention model for hyperspectral image classification

ABSTRACT Deep learning brought a new method for hyperspectral image (HSI) classification, in which images are usually pre-processed by reducing their dimensions before being packaged into pieces to be input to the deep network for feature extraction. However, the learning capability of convolutional kernels of fixed dimensions is usually limited, and thus they are inclined to cause losses of feature details. In this paper, a new global-local block spatial-spectral fusion attention (GBSFA) model is proposed. An improved Inception structure is designed to extract the feature information of the global block, and the self-attention mechanism and spatial pyramid pooling (SPP) are applied to focus on the interclass edge feature information of the local block. Combined with long-short term memory (LSTM) networks, the effective information of the spectral dimension is extracted. Finally, the features extracted from the spatial dimension and the spectral dimension are conveyed in the full connection layer for classification training. Experimental results show that the classification accuracy of the proposed approach is higher than that of other comparative methods using small training sets.

Multiscale Feature Fusion Network Incorporating 3D Self-Attention for Hyperspectral Image Classification

In recent years, the deep learning-based hyperspectral image (HSI) classification method has achieved great success, and the convolutional neural network (CNN) method has achieved good classification performance in the HSI classification task. However, the convolutional operation only works with local neighborhoods, and is effective in extracting local features. It is difficult to capture interactive features over long distances, which affects the accuracy of classification to some extent. At the same time, the data from HSI have the characteristics of three-dimensionality, redundancy, and noise. To solve these problems, we propose a 3D self-attention multiscale feature fusion network (3DSA-MFN) that integrates 3D multi-head self-attention. 3DSA-MFN first uses different sized convolution kernels to extract multiscale features, samples the different granularities of the feature map, and effectively fuses the spatial and spectral features of the feature map. Then, we propose an improved 3D multi-head self-attention mechanism that provides local feature details for the self-attention branch, and fully exploits the context of the input matrix. To verify the performance of the proposed method, we compare it with six current methods on three public datasets. The experimental results show that the proposed 3DSA-MFN achieves competitive classification and highlights the HSI classification task.

A novel spectral-spatial multi-scale network for hyperspectral image classification with the Res2Net block

ABSTRACT Traditional hyperspectral image (HSI) classification methods mainly include machine learning and convolutional neural network. However, they extremely depend on the large training samples. To obtain high accuracy on limited training samples, we propose a novel end-to-end spectral-spatial multi-scale network (SSMSN) for HSI classification. The SSMSN uses the multi-scale spectral module and the multi-scale spatial module to extract discriminative multi-scale spectral and multi-scale spatial features separately. In the multi-scale spectral module and spatial module, the multi-scale Res2Net block structure can learn multi-scale features at a granular level and increase the range of receptive fields by constructing the hierarchical residual-like connection within one single residual block. To alleviate the overfitting problem and further improve the classification accuracy on limited training samples, we adopt a simple but effective hinge cross-entropy loss function to train the SSMSN at the dynamic learning rate. A large number of experimental results demonstrate that on the Indiana Pines, University of Pavia, Kennedy Space Center, and Salinas Scene data sets, the proposed SSMSN achieves higher classification accuracy than state-of-the-art methods on limited training samples. Meanwhile, our SSMSN obtains less training and testing time than the popular AUSSC method.

Multilayer Spectral-Spatial Graphs for Label Noisy Robust Hyperspectral Image Classification.

In hyperspectral image (HSI) analysis, label information is a scarce resource and it is unavoidably affected by human and nonhuman factors, resulting in a large amount of label noise. Although most of the recent supervised HSI classification methods have achieved good classification results, their performance drastically decreases when the training samples contain label noise. To address this issue, we propose a label noise cleansing method based on spectral-spatial graphs (SSGs). In particular, an affinity graph is constructed based on spectral and spatial similarity, in which pixels in a superpixel segmentation-based homogeneous region are connected, and their similarities are measured by spectral feature vectors. Then, we use the constructed affinity graph to regularize the process of label noise cleansing. In this manner, we transform label noise cleansing to an optimization problem with a graph constraint. To fully utilize spatial information, we further develop multiscale segmentation-based multilayer SSGs (MSSGs). It can efficiently merge the complementary information of multilayer graphs and thus provides richer spatial information compared with any single-layer graph obtained from isolation segmentation. Experimental results show that MSSG reduces the level of label noise. Compared with the state of the art, the proposed MSSG method exhibits significantly enhanced classification accuracy toward the training data with noisy labels. The significant advantages of the proposed method over four major classifiers are also demonstrated. The source code is available at https://github.com/junjun-jiang/MSSG.

A Review of Hyperspectral Image Classification Based on Joint Spatial-spectral Features

Hyperspectral image classification technology is a basic work in the application of hyperspectral images. In recent years, with the innovation and development of hyperspectral image classification technology, the method and performance of hyperspectral image classification based on joint spatial-spectral features have made breakthroughs, and have gradually become the focus of researchers. In order to further promote the development of the spatial-spectral feature union class method and improve the classification accuracy of hyperspectral images, this paper summarizes the commonly used spatial-spectral union algorithms. Firstly, the introduction briefly outlines the background and research status of this field. Some common problems in the process of hyperspectral image classification are listed. Finally, some current hyperspectral image classification methods based on joint spatial-spectral features are introduced. The main roles and existing problems of spatial-spectral joint features in the field of hyperspectral image classification are summarized in detail, and the future research directions are prospected.

Multiple Superpixel Graphs Learning Based on Adaptive Multiscale Segmentation for Hyperspectral Image Classification

Hyperspectral image classification (HSIC) methods usually require more training samples for better classification performance. However, a large number of labeled samples are difficult to obtain because it is cost- and time-consuming to label an HSI in a pixel-wise way. Therefore, how to overcome the problem of insufficient accuracy and stability under the condition of small labeled training sample size (SLTSS) is still a challenge for HSIC. In this paper, we proposed a novel multiple superpixel graphs learning method based on adaptive multiscale segmentation (MSGLAMS) for HSI classification to address this problem. First, the multiscale-superpixel-based framework can reduce the adverse effect of improper selection of a superpixel segmentation scale on the classification accuracy while saving the cost to manually seek a suitable segmentation scale. To make full use of the superpixel-level spatial information of different segmentation scales, a novel two-steps multiscale selection strategy is designed to adaptively select a group of complementary scales (multiscale). To fix the bias and instability of a single model, multiple superpixel-based graphical models obatined by constructing superpixel contracted graph of fusion scales are developed to jointly predict the final results via a pixel-level fusion strategy. Experimental results show that the proposed MSGLAMS has better performance when compared with other state-of-the-art algorithms. Specifically, its overall accuracy achieves 94.312%, 99.217%, 98.373% and 92.693% on Indian Pines, Salinas and University of Pavia, and the more challenging dataset Houston2013, respectively.