Spectral-spatial Hyperspectral Image Research Articles

SSC-SFN: spectral-spatial non-local segment federated network for hyperspectral image classification with limited labeled samples

ABSTRACT Hyperspectral image (HSI) classification methods based on deep learning (DL) have performed well in numerous investigations. Although many modified superpixel-wise neural networks are utilized to enhance spatial information, their ability to mine spectral information in graph structures is insufficient. Moreover, single classifier approaches are unable to extract adequate spatial and spectral information simultaneously. For the classification of large-scale research areas, many works have relied on the use of a large number of labeled samples, leading to low efficiency and weak generalization. To address these issues, an effective spectral-spatial HSI classification approach based on spectral-spatial non-local segment federated network (SSC-SFN) was developed in this study. In this framework, deconvolution is employed to recover the data size, while the lost spatial information is replaced by up-pooling. The spectral dimensional features are updated through the generation of non-Euclidean graph structures and the non-local segment smoothing technique. The convolutional neural network and graph convolutional network techniques are coupled to exploit the available spectral and spatial structure information fully. Extensive experimental results obtained using four public benchmark datasets show that the classification accuracy of SSC-SFN can exceed 90% for large-scale HSIs with limited samples.

Weighted residual self-attention graph-based transformer for spectral–spatial hyperspectral image classification

ABSTRACT Recently, deep learning for hyperspectral image classification has been successfully applied, and some convolutional neural network (CNN)-based models already achieved attractive classification results. Since hyperspectral data is a spectral-spatial cube data that can generally be considered as sequential data along with the spectral dimension, CNN models perform poorly on such a sequential data. Unlike convolutional neural networks (CNNs) that mainly concern with local relationship models in images, transformer has been shown to be a powerful structure for qualifying sequential data. In the SA (self-attention) module of ViT, each token is updated through aggregating all token’s features based on the self-attention graph. Through this, tokens can exchange information sufficiently among each other which provides a powerful representation capability. However, as the layers become deeper, the transformer model suffers from network degradation. Therefore, in order to improve the layer-to-layer information exchange and alleviate the network degradation problem, we propose a Weighted Residual Self-attention Graph-based Transformer (RSAGformer) model for hyperspectral image classification with respect to the self-attention mechanism. It effectively solves the network degradation problem of deep transformer model by fusing the self-attention information between adjacent layers and extracts the information of data effectively. Extensive experiment evaluation with six public hyperspectral datasets shows that the RSAGformer yields competitive results for classification.

Machine Learning and Deep Learning Techniques for Spectral Spatial Classification of Hyperspectral Images: A Comprehensive Survey

The growth of Hyperspectral Image (HSI) analysis is due to technology advancements that enable cameras to collect hundreds of continuous spectral information of each pixel in an image. HSI classification is challenging due to the large number of redundant spectral bands, limited training samples and non-linear relationship between the collected spatial position and the spectral bands. Our survey highlights recent research in HSI classification using traditional Machine Learning techniques like kernel-based learning, Support Vector Machines, Dimension Reduction and Transform-based techniques. Our study also digs into Deep Learning (DL) techniques that involve the usage of Autoencoders, 1D, 2D and 3D-Convolutional Neural Networks to classify HSI. From the comparison, it is observed that DL-based classification techniques outperform ML-based techniques. It has also been observed that spectral-spatial HSI classification outperforms pixel-by-pixel classification because it incorporates spectral signatures and spatial domain information. The performance of ML and DL-based classification techniques has been reviewed on commonly used land cover datasets like Indian Pines, Salinas valley and Pavia University.

Cascaded Convolution-Based Transformer With Densely Connected Mechanism for Spectral–Spatial Hyperspectral Image Classification

Hyperspectral image (HSI) classification attempts to classify each pixel, which is an important means of obtaining land–cover knowledge. Hyperspectral images are cubic data with spectral–spatial knowledge and can generally be considered as sequential data alongside spectral dimension. Unlike convolutional neural networks (CNNs), which mainly focus on local relationship models in images, transformers have been shown to be a powerful structure for qualifying sequence data. However, it lacks the excellent ability of CNNs in establishing local relationships in images and cannot perform good generalization in case of insufficient data. In addition, the gradient disappearance problem hinders the convergence stability of deep learning networks as the layers get deeper. To address these problems, we propose a Cascaded Convolution-based Transformer with Densely Connected Mechanism (CDCformer) for hyperspectral image classification. First, we propose a cascaded convolution feature tokenization to extract spectral–spatial information, which will introduce some inductive bias properties of CNN into the transformer. In addition, we design a simple and effective densely connected transformer to enhance feature propagation and transfer memorable information from shallow to deep layers. It efficiently improves the performance of the transformer and extracts more discriminative spectral–spatial features from the HSI. Extensive experimental evaluation of three public hyperspectral data sets shows that CDCformer achieves competitive classification results.

Spectral–Spatial Hyperspectral Image Classification Based on Multiple Views and Multigraphs Fusion With Few Labeled Samples

The issue of limited labeled samples is still grave in hyperspectral image classification. Semisupervised learning (SSL) utilizing both labeled and unlabeled samples promotes a solution to this issue. However, it has been found that the performance of single SSL is frail when the labeled samples is limited. To tackle this problem, we propose 3D-Gabor and multiple graphs semisupervised framework (3DG-MGSF). The whole framework is tripartite, including multi-views generation and selection, multiple graphs-based label propagation, and double layers classification fusion process. Specifically, a number of 3D-Gabor filters with various directions and frequencies are employed to generate multiple views. Afterwards, a double multi-views selection procedure is applied to ensure the sufficiency and diversity of multiple views. Subsequently, it is time for the multiple graphs-based label propagation to put on its pump. Moreover, spatial and spectral classifications are combined based on the weighted re-fusion algorithm to obtain the final classification. Experimental results illustrate numerically and visually the significantly superior performance of proposed algorithm compared with four state-of-the-art algorithms with few labeled samples.

Proxy-Based Deep Learning Framework for Spectral–Spatial Hyperspectral Image Classification: Efficient and Robust

Deep convolutional networks have been extensively deployed in hyperspectral image (HSI) classification. Reaching for high accuracy, the existing deep-learning-based methods commonly deepen or widen their networks for better performance, which brings higher computational complexity and the risk of overfitting. Although the introduction of the residual module and batch-normalization reduces the generalization degradation in complex networks, the mainstream methods still suffer from low robustness to the noise. To tackle these issues, a compact proxy-based deep learning framework is proposed to perform highly accurate HSI classification with superb efficiency and robustness. In this article: 1) novel deep proxies are integrated to replace the dense classifier layers in conventional networks, which represents specific classes in deep embedding space and enables fast and reliable convergence; 2) the proxy-based feature embedding is studied in distance metric and similarity metric, and compatible dual-metric loss functions are designed for further optimized embedding distribution, which leads to more robust generalization; and 3) state-of-the-art performance and robustness are demonstrated by the proposed framework on mainstream HSI data sets with the minimal network scale and time complexity.

Generative Adversarial Minority Oversampling for Spectral–Spatial Hyperspectral Image Classification

Recently, convolutional neural networks (CNNs) have exhibited commendable performance for hyperspectral image (HSI) classification. Generally, an important number of samples are needed for each class to properly train CNNs. However, existing HSI data sets suffer from a significant class imbalance problem, where many classes do not have enough samples to characterize the spectral information. The performance of existing CNN models is biased toward the majority classes, which possess more samples for the training. This article addresses this issue of imbalanced data in HSI classification. In particular, a new <monospace xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">3D-HyperGAMO</monospace> model is proposed, which uses generative adversarial minority oversampling. The proposed <monospace xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">3D-HyperGAMO</monospace> automatically generates more samples for minority classes at training time, using the existing samples of that class. The samples are generated in the form of a 3-D hyperspectral patch. A different classifier from the generator and the discriminator is used in the <monospace xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">3D-HyperGAMO</monospace> model, which is trained using both original and generated samples to determine the classes of newly generated samples to which they actually belong. The generated data are combined classwise with the original training data set to learn the network parameters of the class. Finally, the trained 3-D classifier network validates the performance of the model using the test set. Four benchmark HSI data sets, namely, Indian Pines (IP), Kennedy Space Center (KSC), University of Pavia (UP), and Botswana (BW), have been considered in our experiments. The proposed model shows outstanding data generation ability during the training, which significantly improves the classification performance over the considered data sets. The source code is available publicly at <uri xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">https://github.com/mhaut/3D-HyperGAMO</uri> .

Hierarchical broad learning system for hyperspectral image classification

A new spectral-spatial hyperspectral image (HSI) classification method called hierarchical broad learning system (HBLS) has been proposed in this paper. Specifically, it combines wavelet, broad learning system (BLS) and Gabor filters into a hierarchical structure. First of all, wavelet is used to reduce the observation noise of HSIs. Then BLS is adopted to acquire a set of pixelwise probability maps from the input data, and Gabor filters are used to explore spatial information by refining these probability maps. These two operations (BLS and Gabor filtering) are alternated to form a hierarchical architecture. And the discriminative spectral-spatial features can be extracted at each layer of the hierarchical architecture. Finally, the spectral-spatial features are fed into the standard BLS for classification. Experimental results on three widely used HSIs reveal that HBLS outperforms some state-of-the-art methods in terms of classification accuracy and sample complexity.

Iterative Training Sampling Coupled With Active Learning for Semisupervised Spectral–Spatial Hyperspectral Image Classification

Training sample selection is a great challenge for hyperspectral image classification (HSIC), specifically when only a very limited number of labeled data samples are available for training. Two recently developed concepts for training sample selection are of particular interest. One is active learning (AL), which augments labeled training samples by including unlabeled data samples as new training samples. The other is iterative training sampling (ITS), which expands data cubes by including additional spatial classification information into the training samples. This article combines AL and ITS simultaneously to derive a joint ITS–AL spectral–spatial (SS) classification approach, to be called ITS augmentation by AL spectral–spatial classification, referred to as ITSA-AL-SS, which can improve SS classification using either AL or ITS alone. The novel idea of ITSA-AL-SS is to take advantage of ITS to expand data cubes by incorporating additional spatial classification information iteratively into the AL-augmented unlabeled data samples to update the current training samples iteration by iteration in one-shot operation. As expected, ITSA-AL-SS is benefited from both ITS and AL to not only further improve classification accuracy but also reduce classification inconsistency.

Hyperspectral Image Classification Based on Sparse Superpixel Graph

Hyperspectral image (HSI) classification is one of the major problems in the field of remote sensing. Particularly, graph-based HSI classification is a promising topic and has received increasing attention in recent years. However, graphs with pixels as nodes generate large size graphs, thus increasing the computational burden. Moreover, satisfactory classification results are often not obtained without considering spatial information in constructing graph. To address these issues, this study proposes an efficient and effective semi-supervised spectral-spatial HSI classification method based on sparse superpixel graph (SSG). In the constructed sparse superpixels graph, each vertex represents a superpixel instead of a pixel, which greatly reduces the size of graph. Meanwhile, both spectral information and spatial structure are considered by using superpixel, local spatial connection and global spectral connection. To verify the effectiveness of the proposed method, three real hyperspectral images, Indian Pines, Pavia University and Salinas, are chosen to test the performance of our proposal. Experimental results show that the proposed method has good classification completion on the three benchmarks. Compared with several competitive superpixel-based HSI classification approaches, the method has the advantages of high classification accuracy (&gt;97.85%) and rapid implementation (&lt;10 s). This clearly favors the application of the proposed method in practice.

Flexible 3-D Gabor features fusion for hyperspectral imagery classification

In recent years, Gabor filtering has been successfully applied in spectral–spatial hyperspectral image (HSI) classification tasks due to its strong power to characterize surface materials. Generally, the standard Gabor filter involves the real and imaginary parts. However, the traditional usage to integrate both the two parts, i.e., the Gabor magnitude feature, might weaken some unique characteristics of each part, therefore leading to possible information loss. To solve this problem, we proposed a flexible 3-D Gabor features fusion (F3DGF) approach to make better use of two parts of Gabor features, based on the phase-induced 3-D Gabor feature, which is rarely exploited before. As the term suggests, the phase-induced feature is guided by a phase parameter P, which can flexibly combine the information from two parts of Gabor components. The proposed F3DGF scheme explores all the phase-induced 3-D Gabor features by means of a decision-level fusion strategy, where the obtained probability outputs are directly gathered to generate the decision map. Experimental results on three real HSIs demonstrate that our approach exhibits good improvements, as compared to the traditional real part and the magnitude Gabor features-based classification methods. The results show great potential to introduce phase-induced 3-D Gabor features for classification tasks.

Hyperspectral image spectral-spatial classification via weighted Laplacian smoothing constraint-based sparse representation.

As a powerful tool in hyperspectral image (HSI) classification, sparse representation has gained much attention in recent years owing to its detailed representation of features. In particular, the results of the joint use of spatial and spectral information has been widely applied to HSI classification. However, dealing with the spatial relationship between pixels is a nontrivial task. This paper proposes a new spatial-spectral combined classification method that considers the boundaries of adjacent features in the HSI. Based on the proposed method, a smoothing-constraint Laplacian vector is constructed, which consists of the interest pixel and its four nearest neighbors through their weighting factor. Then, a novel large-block sparse dictionary is developed for simultaneous orthogonal matching pursuit. Our proposed method can obtain a better accuracy of HSI classification on three real HSI datasets than the existing spectral-spatial HSI classifiers. Finally, the experimental results are presented to verify the effectiveness and superiority of the proposed method.

Hyperspectral Image Classification Based on Superpixel Pooling Convolutional Neural Network with Transfer Learning

Deep learning-based hyperspectral image (HSI) classification has attracted more and more attention because of its excellent classification ability. Generally, the outstanding performance of these methods mainly depends on a large number of labeled samples. Therefore, it still remains an ongoing challenge how to integrate spatial structure information into these frameworks to classify the HSI with limited training samples. In this study, an effective spectral-spatial HSI classification scheme is proposed based on superpixel pooling convolutional neural network with transfer learning (SP-CNN). The suggested method includes three stages. The first part consists of convolution and pooling operation, which is a down-sampling process to extract the main spectral features of an HSI. The second part is composed of up-sampling and superpixel (homogeneous regions with adaptive shape and size) pooling to explore the spatial structure information of an HSI. Finally, the hyperspectral data with each superpixel as a basic input rather than a pixel are fed to fully connected neural network. In this method, the spectral and spatial information is effectively fused by using superpixel pooling technique. The use of popular transfer learning technology in the proposed classification framework significantly improves the training efficiency of SP-CNN. To evaluate the effectiveness of the SP-CNN, extensive experiments were conducted on three common real HSI datasets acquired from different sensors. With 30 labeled pixels per class, the overall classification accuracy provided by this method on three benchmarks all exceeded 93%, which was at least 4.55% higher than that of several state-of-the-art approaches. Experimental and comparative results prove that the proposed algorithm can effectively classify the HSI with limited training labels.

Multi-Direction Networks With Attentional Spectral Prior for Hyperspectral Image Classification

Convolutional neural networks (CNNs) have achieved prominent progress in recent years and demonstrated remarkable properties in spectral–spatial hyperspectral image (HSI) classification. However, conventional spatial-context-based CNNs commonly adopt the single patchwise scheme to represent the to-be-classified samples, which often fails to completely investigate the wealthy spectral–spatial information in complicated situations. For instance, it has great probability to cause misclassifications on the irregular or inhomogeneous areas, especially for the borders across different classes. To counteract this deficiency, we propose a unified multi-direction network (MDN) for HSI Classification (HSIC), which can exhaustively explore the abundant spectral and detailed spatial-context information through multi-direction samples. Additionally, considering the image-spectrum merged structure of the HSI, 3-D Squeeze-and-Excitation residual (3DSERes) blocks are devised in each stream of the framework to consecutively learn the spectral and spatial from low-level to high-level features. Specifically, 3DSERes can not only facilitate fluent gradient in backpropagation through skip connections, but also emphasize the significant spectral–spatial features and constrain the futile ones. This characteristic is beneficial to enhance the model’s generalization capability even with limited training samples. Furthermore, for properly aggregating the multi-direction deep features, we exploit the simple, yet effective attentional spectral prior (ASP) creatively through leveraging the original spectral correlations. Extensive experimental results on three benchmark data sets indicate that the proposed MDN-ASP can achieve promising classification performance compared to the state-of-the-art methods.

Broad Learning System with Locality Sensitive Discriminant Analysis for Hyperspectral Image Classification

In this paper, we propose a new method for hyperspectral images (HSI) classification, aiming to take advantage of both manifold learning-based feature extraction and neural networks by stacking layers applying locality sensitive discriminant analysis (LSDA) to broad learning system (BLS). BLS has been proven to be a successful model for various machine learning tasks due to its high feature representative capacity introduced by numerous randomly mapped features. However, it also produces redundancy, which is indiscriminate and finally lowers its performance and causes heavy computing demand, especially in cases of the input data bearing high dimensionality. In our work, a manifold learning method is integrated into the BLS by inserting two LSDA layers before the input layer and output layer separate, so the spectral-spatial HSI features are fully utilized to acquire the state-of-the-art classification accuracy. The extensive experiments have shown our method’s superiority.

Adaptive MultiScale Segmentations for Hyperspectral Image Classification

The number of superpixels (i.e., segmentation scale) is crucial for spectral–spatial hyperspectral image (HSI) classification. Existing methods always set the segmentation scale through a manually experimental strategy, which is time-consuming and unsuitable for various complicated practical applications. The information fusion of complementary multiple scales is proven to be more effective than the single scale for HSI classification, but the scale level is still set manually. In this article, we propose a novel adaptive multiscale segmentations (AMSs) method that can automatically provide a set of suitable scales that are adapted to different hyperspectral data. Specifically, based on the assumption that the segmentation scale of HSI is related to the image complexity itself, the texture ratio and the number of land cover classes are used to examine a candidate scale pool. A good scale means that it contains a small spectral difference between pixels within the same superpixel (intrasuperpixel discrimination index) and a large discrepancy between neighboring superpixels (intersuperpixel discrimination index). Thus, an intra–interscale discrimination index is defined and applied to depict the characteristics of the scale. Then, the scale with the best intra–inter discrimination index, which usually has satisfactory performance, is treated as the initially selected scale. The remaining suitable scales are iteratively compared with the selected ones and then added to the target scale pool, until the newly added scale can no longer provide significantly complementary information. Extensive experimental results on three HSI data sets have demonstrated the effectiveness of the proposed AMS when compared with state-of-the-art methods.

Spectral-Spatial Hyperspectral Image Classification with Superpixel Pattern and Extreme Learning Machine

Spectral-spatial classification of hyperspectral images (HSIs) has recently attracted great attention in the research domain of remote sensing. It is well-known that, in remote sensing applications, spectral features are the fundamental information and spatial patterns provide the complementary information. With both spectral features and spatial patterns, hyperspectral image (HSI) applications can be fully explored and the classification performance can be greatly improved. In reality, spatial patterns can be extracted to represent a line, a clustering of points or image texture, which denote the local or global spatial characteristic of HSIs. In this paper, we propose a spectral-spatial HSI classification model based on superpixel pattern (SP) and kernel based extreme learning machine (KELM), called SP-KELM, to identify the land covers of pixels in HSIs. In the proposed SP-KELM model, superpixel pattern features are extracted by an advanced principal component analysis (PCA), which is based on superpixel segmentation in HSIs and used to denote spatial information. The KELM method is then employed to be a classifier in the proposed spectral-spatial model with both the original spectral features and the extracted spatial pattern features. Experimental results on three publicly available HSI datasets verify the effectiveness of the proposed SP-KELM model, with the performance improvement of 10% over the spectral approaches.

A Novel Hyperspectral Image Classification Pattern Using Random Patches Convolution and Local Covariance

Today, more and more deep learning frameworks are being applied to hyperspectral image classification tasks and have achieved great results. However, such approaches are still hampered by long training times. Traditional spectral–spatial hyperspectral image classification only utilizes spectral features at the pixel level, without considering the correlation between local spectral signatures. Our article has tested a novel hyperspectral image classification pattern, using random-patches convolution and local covariance (RPCC). The RPCC is an effective two-branch method that, on the one hand, obtains a specified number of convolution kernels from the image space through a random strategy and, on the other hand, constructs a covariance matrix between different spectral bands by clustering local neighboring pixels. In our method, the spatial features come from multi-scale and multi-level convolutional layers. The spectral features represent the correlations between different bands. We use the support vector machine as well as spectral and spatial fusion matrices to obtain classification results. Through experiments, RPCC is tested with five excellent methods on three public data-sets. Quantitative and qualitative evaluation indicators indicate that the accuracy of our RPCC method can match or exceed the current state-of-the-art methods.

Multiple deep-belief-network-based spectral-spatial classification of hyperspectral images

A deep-learning-based feature extraction has recently been proposed for HyperSpectral Images (HSI) classification. A Deep Belief Network (DBN), as part of deep learning, has been used in HSI classification for deep and abstract feature extraction. However, DBN has to simultaneously deal with hundreds of features from the HSI hyper-cube, which results into complexity and leads to limited feature abstraction and performance in the presence of limited training data. Moreover, a dimensional-reduction-based solution to this issue results in the loss of valuable spectral information, thereby affecting classification performance. To address the issue, this paper presents a Spectral-Adaptive Segmented DBN (SAS-DBN) for spectral-spatial HSI classification that exploits the deep abstract features by segmenting the original spectral bands into small sets/groups of related spectral bands and processing each group separately by using local DBNs. Furthermore, spatial features are also incorporated by first applying hyper-segmentation on the HSI. These results improved data abstraction with reduced complexity and enhanced the performance of HSI classification. Local application of DBN-based feature extraction to each group of bands reduces the computational complexity and results in better feature extraction improving classification accuracy. In general, exploiting spectral features effectively through a segmented-DBN process and spatial features through hyper-segmentation and integration of spectral and spatial features for HSI classification has a major effect on the performance of HSI classification. Experimental evaluation of the proposed technique on well-known HSI standard data sets with different contexts and resolutions establishes the efficacy of the proposed techniques, wherein the results are comparable to several recently proposed HSI classification techniques.

Hyperspectral image classification with SVM and guided filter

Hyperspectral image (HSI) classification has been long envisioned in the remote sensing community. Many methods have been proposed for HSI classification. Among them, the method of fusing spatial features has been widely used and achieved good performance. Aiming at the problem of spatial feature extraction in spectral-spatial HSI classification, we proposed a guided filter-based method. We attempted two fusion methods for spectral and spatial features. In order to optimize the classification results, we also adopted a guided filter to obtain better results. We apply the support vector machine (SVM) to classify the HSI. Experiments show that our proposed methods can obtain very competitive results than compared methods on all the three popular datasets. More importantly, our methods are fast and easy to implement.