Spatial Hyperspectral Image Classification Research Articles

Weighted residual self-attention graph-based transformer for spectral–spatial hyperspectral image classification

ABSTRACT Recently, deep learning for hyperspectral image classification has been successfully applied, and some convolutional neural network (CNN)-based models already achieved attractive classification results. Since hyperspectral data is a spectral-spatial cube data that can generally be considered as sequential data along with the spectral dimension, CNN models perform poorly on such a sequential data. Unlike convolutional neural networks (CNNs) that mainly concern with local relationship models in images, transformer has been shown to be a powerful structure for qualifying sequential data. In the SA (self-attention) module of ViT, each token is updated through aggregating all token’s features based on the self-attention graph. Through this, tokens can exchange information sufficiently among each other which provides a powerful representation capability. However, as the layers become deeper, the transformer model suffers from network degradation. Therefore, in order to improve the layer-to-layer information exchange and alleviate the network degradation problem, we propose a Weighted Residual Self-attention Graph-based Transformer (RSAGformer) model for hyperspectral image classification with respect to the self-attention mechanism. It effectively solves the network degradation problem of deep transformer model by fusing the self-attention information between adjacent layers and extracts the information of data effectively. Extensive experiment evaluation with six public hyperspectral datasets shows that the RSAGformer yields competitive results for classification.

Machine Learning and Deep Learning Techniques for Spectral Spatial Classification of Hyperspectral Images: A Comprehensive Survey

The growth of Hyperspectral Image (HSI) analysis is due to technology advancements that enable cameras to collect hundreds of continuous spectral information of each pixel in an image. HSI classification is challenging due to the large number of redundant spectral bands, limited training samples and non-linear relationship between the collected spatial position and the spectral bands. Our survey highlights recent research in HSI classification using traditional Machine Learning techniques like kernel-based learning, Support Vector Machines, Dimension Reduction and Transform-based techniques. Our study also digs into Deep Learning (DL) techniques that involve the usage of Autoencoders, 1D, 2D and 3D-Convolutional Neural Networks to classify HSI. From the comparison, it is observed that DL-based classification techniques outperform ML-based techniques. It has also been observed that spectral-spatial HSI classification outperforms pixel-by-pixel classification because it incorporates spectral signatures and spatial domain information. The performance of ML and DL-based classification techniques has been reviewed on commonly used land cover datasets like Indian Pines, Salinas valley and Pavia University.

Cascaded Convolution-Based Transformer With Densely Connected Mechanism for Spectral–Spatial Hyperspectral Image Classification

Hyperspectral image (HSI) classification attempts to classify each pixel, which is an important means of obtaining land–cover knowledge. Hyperspectral images are cubic data with spectral–spatial knowledge and can generally be considered as sequential data alongside spectral dimension. Unlike convolutional neural networks (CNNs), which mainly focus on local relationship models in images, transformers have been shown to be a powerful structure for qualifying sequence data. However, it lacks the excellent ability of CNNs in establishing local relationships in images and cannot perform good generalization in case of insufficient data. In addition, the gradient disappearance problem hinders the convergence stability of deep learning networks as the layers get deeper. To address these problems, we propose a Cascaded Convolution-based Transformer with Densely Connected Mechanism (CDCformer) for hyperspectral image classification. First, we propose a cascaded convolution feature tokenization to extract spectral–spatial information, which will introduce some inductive bias properties of CNN into the transformer. In addition, we design a simple and effective densely connected transformer to enhance feature propagation and transfer memorable information from shallow to deep layers. It efficiently improves the performance of the transformer and extracts more discriminative spectral–spatial features from the HSI. Extensive experimental evaluation of three public hyperspectral data sets shows that CDCformer achieves competitive classification results.

Spectral–Spatial Hyperspectral Image Classification Based on Multiple Views and Multigraphs Fusion With Few Labeled Samples

The issue of limited labeled samples is still grave in hyperspectral image classification. Semisupervised learning (SSL) utilizing both labeled and unlabeled samples promotes a solution to this issue. However, it has been found that the performance of single SSL is frail when the labeled samples is limited. To tackle this problem, we propose 3D-Gabor and multiple graphs semisupervised framework (3DG-MGSF). The whole framework is tripartite, including multi-views generation and selection, multiple graphs-based label propagation, and double layers classification fusion process. Specifically, a number of 3D-Gabor filters with various directions and frequencies are employed to generate multiple views. Afterwards, a double multi-views selection procedure is applied to ensure the sufficiency and diversity of multiple views. Subsequently, it is time for the multiple graphs-based label propagation to put on its pump. Moreover, spatial and spectral classifications are combined based on the weighted re-fusion algorithm to obtain the final classification. Experimental results illustrate numerically and visually the significantly superior performance of proposed algorithm compared with four state-of-the-art algorithms with few labeled samples.

Proxy-Based Deep Learning Framework for Spectral–Spatial Hyperspectral Image Classification: Efficient and Robust

Deep convolutional networks have been extensively deployed in hyperspectral image (HSI) classification. Reaching for high accuracy, the existing deep-learning-based methods commonly deepen or widen their networks for better performance, which brings higher computational complexity and the risk of overfitting. Although the introduction of the residual module and batch-normalization reduces the generalization degradation in complex networks, the mainstream methods still suffer from low robustness to the noise. To tackle these issues, a compact proxy-based deep learning framework is proposed to perform highly accurate HSI classification with superb efficiency and robustness. In this article: 1) novel deep proxies are integrated to replace the dense classifier layers in conventional networks, which represents specific classes in deep embedding space and enables fast and reliable convergence; 2) the proxy-based feature embedding is studied in distance metric and similarity metric, and compatible dual-metric loss functions are designed for further optimized embedding distribution, which leads to more robust generalization; and 3) state-of-the-art performance and robustness are demonstrated by the proposed framework on mainstream HSI data sets with the minimal network scale and time complexity.

Generative Adversarial Minority Oversampling for Spectral–Spatial Hyperspectral Image Classification

Recently, convolutional neural networks (CNNs) have exhibited commendable performance for hyperspectral image (HSI) classification. Generally, an important number of samples are needed for each class to properly train CNNs. However, existing HSI data sets suffer from a significant class imbalance problem, where many classes do not have enough samples to characterize the spectral information. The performance of existing CNN models is biased toward the majority classes, which possess more samples for the training. This article addresses this issue of imbalanced data in HSI classification. In particular, a new <monospace xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">3D-HyperGAMO</monospace> model is proposed, which uses generative adversarial minority oversampling. The proposed <monospace xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">3D-HyperGAMO</monospace> automatically generates more samples for minority classes at training time, using the existing samples of that class. The samples are generated in the form of a 3-D hyperspectral patch. A different classifier from the generator and the discriminator is used in the <monospace xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">3D-HyperGAMO</monospace> model, which is trained using both original and generated samples to determine the classes of newly generated samples to which they actually belong. The generated data are combined classwise with the original training data set to learn the network parameters of the class. Finally, the trained 3-D classifier network validates the performance of the model using the test set. Four benchmark HSI data sets, namely, Indian Pines (IP), Kennedy Space Center (KSC), University of Pavia (UP), and Botswana (BW), have been considered in our experiments. The proposed model shows outstanding data generation ability during the training, which significantly improves the classification performance over the considered data sets. The source code is available publicly at <uri xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">https://github.com/mhaut/3D-HyperGAMO</uri> .

Iterative Training Sampling Coupled With Active Learning for Semisupervised Spectral–Spatial Hyperspectral Image Classification

Training sample selection is a great challenge for hyperspectral image classification (HSIC), specifically when only a very limited number of labeled data samples are available for training. Two recently developed concepts for training sample selection are of particular interest. One is active learning (AL), which augments labeled training samples by including unlabeled data samples as new training samples. The other is iterative training sampling (ITS), which expands data cubes by including additional spatial classification information into the training samples. This article combines AL and ITS simultaneously to derive a joint ITS–AL spectral–spatial (SS) classification approach, to be called ITS augmentation by AL spectral–spatial classification, referred to as ITSA-AL-SS, which can improve SS classification using either AL or ITS alone. The novel idea of ITSA-AL-SS is to take advantage of ITS to expand data cubes by incorporating additional spatial classification information iteratively into the AL-augmented unlabeled data samples to update the current training samples iteration by iteration in one-shot operation. As expected, ITSA-AL-SS is benefited from both ITS and AL to not only further improve classification accuracy but also reduce classification inconsistency.

An SVM-Based Nested Sliding Window Approach for Spectral–Spatial Classification of Hyperspectral Images

This paper proposes a Nested Sliding Window (NSW) method based on the correlation between pixel vectors, which can extract spatial information from the hyperspectral image (HSI) and reconstruct the original data. In the NSW method, the neighbourhood window constructed with the target pixel as the centre contains relevant pixels that are spatially adjacent to the target pixel. In the neighbourhood window, a nested sliding sub-window contains the target pixel and a part of the relevant pixels. The optimal sub-window position is determined according to the average value of the Pearson correlation coefficients of the target pixel and the relevant pixels, and the target pixel can be reconstructed by using the pixels and the corresponding correlation coefficients in the optimal sub-window. By combining NSW with Principal Component Analysis (PCA) and Support Vector Machine (SVM), a classification model, namely NSW-PCA-SVM, is obtained. This paper conducts experiments on three public datasets, and verifies the effectiveness of the proposed model by comparing with two basic models, i.e., SVM and PCA-SVM, and six state-of-the-art models, i.e., CDCT-WF-SVM, CDCT-2DCT-SVM, SDWT-2DWT-SVM, SDWT-WF-SVM, SDWT-2DCT-SVM and Two-Stage. The proposed approach has the following advantages in overall accuracy (OA)—take the experimental results on the Indian Pines dataset as an example: (1) Compared with SVM (OA = 53.29%) and PCA-SVM (OA = 58.44%), NSW-PCA-SVM (OA = 91.40%) effectively utilizes the spatial information of HSI and improves the classification accuracy. (2) The performance of the proposed model is mainly determined by two parameters, i.e., the window size in NSW and the number of principal components in PCA. The two parameters can be adjusted independently, making parameter adjustment more convenient. (3) When the sample size of the training set is small (20 samples per class), the proposed NSW-PCA-SVM approach achieves 2.38–18.40% advantages in OA over the six state-of-the-art models.

Weighted Sparse Graph Regularization for Spectral–Spatial Classification of Hyperspectral Images

In this letter, we propose a novel weighted sparse graph regularization (SGR) (WSGR) model by incorporating the spatial information both in pre and postprocessing stages. In the preprocessing stage, we generate adaptive patch features to represent the spatial information. Traditional patch feature extraction methods rarely consider the quality of the neighboring pixels within a patch since the central pixel and its neighbors may belong to different classes. To solve this issue, a Gaussian-kernel-based weighting scheme that can adaptively model the neighborhood information is introduced into the original SGR model. Experimental results, conducted with two popular hyperspectral data sets, indicate that the proposed method outperforms other related methods with an overall accuracy higher than 90% when only 20 labeled samples per class are used for training.

A two-stage method for spectral–spatial classification of hyperspectral images

We propose a novel two-stage method for the classification of hyperspectral images. Pixel-wise classifiers, such as the classical support vector machine (SVM), consider spectral information only. As spatial information is not utilized, the classification results are not optimal and the classified image may appear noisy. Many existing methods, such as morphological profiles, superpixel segmentation, and composite kernels, exploit the spatial information. In this paper, we propose a two-stage approach inspired by image denoising and segmentation to incorporate the spatial information. In the first stage, SVMs are used to estimate the class probability for each pixel. In the second stage, a convex variant of the Mumford–Shah model is applied to each probability map to denoise and segment the image into different classes. Our proposed method effectively utilizes both spectral and spatial information of the data sets and is fast as only convex minimization is needed in addition to the SVMs. Experimental results on three widely utilized real hyperspectral data sets indicate that our method is very competitive in accuracy, timing, and the number of parameters when compared with current state-of-the-art methods, especially when the inter-class spectra are similar or the percentage of training pixels is reasonably high.

An Adaptive Multiview Active Learning Approach for Spectral–Spatial Classification of Hyperspectral Images

Combining spectral and spatial features in hyperspectral image classification is a common practice due to the improvements in classification accuracy that can be obtained by extracting information from neighboring pixels. However, the resulting high dimensionality of the input data and the typically limited number of labeled samples are two key challenges that affect the overall performance of supervised classification methods. To alleviate these two issues, we propose an adaptive multiview (MV)-based active learning (AL) approach that is different from the existing MV AL methods in two main ways: 1) to improve the view sufficiency, a spectral–spatial view generation approach is proposed by incorporating spatial features derived from the segmentation maps into each view and 2) to increase the diversity across views, a dynamic view is generated at each AL iteration by selecting important features from the predefined views. The performance of each view is further improved by applying the proposed AL algorithm in conjunction with an ensemble approach as back-end classifier, a scenario less explored in the remote sensing community than single classifier-based AL methodologies. The proposed approach is applied to three widely analyzed hyperspectral data sets [i.e., Kennedy Space Center (KSC), Indian Pine, and University of Houston (UH)], and the results demonstrate the efficacy of the proposed method compared with other state-of-the-art AL classification methods.

Improved neural network classification of hyperspectral imagery using weighted genetic algorithm and hierarchical segmentation

This study proposes a modified spectral–spatial classification approach for improving the spectral–spatial classification of hyperspectral images. The spatial information is obtained by an enhanced marker-based hierarchical segmentation (MHS) algorithm. The weighted genetic (WG) algorithm is first employed to obtain the subspace of hyperspectral data. The obtained features are then fed into the multi-layer perceptron (MLP) neural network classification algorithm. Afterwards, the MHS algorithm is applied in order to increase the accuracy of less-accurately classified land-cover types. In the proposed approach, namely MLP-MHS, the markers are extracted from the classification maps obtained by MLP and support vector machine classifiers. Experiments on two benchmark hyperspectral datasets, Pavia University and Berlin, validate the soundness of the proposed approach compared to the MLP and the original MHS algorithms.

A Novel Hyperspectral Image Classification Pattern Using Random Patches Convolution and Local Covariance

Today, more and more deep learning frameworks are being applied to hyperspectral image classification tasks and have achieved great results. However, such approaches are still hampered by long training times. Traditional spectral–spatial hyperspectral image classification only utilizes spectral features at the pixel level, without considering the correlation between local spectral signatures. Our article has tested a novel hyperspectral image classification pattern, using random-patches convolution and local covariance (RPCC). The RPCC is an effective two-branch method that, on the one hand, obtains a specified number of convolution kernels from the image space through a random strategy and, on the other hand, constructs a covariance matrix between different spectral bands by clustering local neighboring pixels. In our method, the spatial features come from multi-scale and multi-level convolutional layers. The spectral features represent the correlations between different bands. We use the support vector machine as well as spectral and spatial fusion matrices to obtain classification results. Through experiments, RPCC is tested with five excellent methods on three public data-sets. Quantitative and qualitative evaluation indicators indicate that the accuracy of our RPCC method can match or exceed the current state-of-the-art methods.

Convolutional neural network for spectral–spatial classification of hyperspectral images

Hyperspectral images (HSIs) have great potential in military reconnaissance, land use, marine monitoring and many other fields. In recent years, the convolutional neural network (CNN) has been successfully used to classify hyperspectral data and achieved remarkable performance. However, the limited labeled samples of HSI lead to the small sample size problem, which remains the major challenge for CNN-based HSI classification. Besides, most CNN models have large number of parameters needed to be learned, which cause high computational cost. To address the aforementioned two issues, a novel CNN-based HSI classification method is proposed. The proposed classification method has several distinguishing characteristics. First, the proposed method can robustly extract spectral and spatial features of the HSI simultaneously. Second, in the proposed CNN architecture, all convolution layers are 1 × 1 convolution layer except the first one, which can greatly reduce the number of network parameters, thus accelerating the training and testing process. Third, a small convolution and feature reuse (SC-FR) module is developed. The SC-FR module is composed of two composite layers and each composite layer consists of two cascaded 1 × 1 convolution layers. Through cross-layer connecting, the input and output features of each composite layer are concatenated and passed to the next convolution layer, thus achieving feature reuse mechanism. Cross-layer connection increases information flow and the utilization rate of middle-level features, which enhances the generalization performance of CNN effectively. Experimental results on three benchmark HSIs demonstrate the competitive superiority of the proposed method over several state-of-the-art HSI classification methods, especially when training samples are limited.

Spectral–spatial classification of hyperspectral images by algebraic multigrid based multiscale information fusion

ABSTRACT In this work, we present a novel spectral-spatial classification framework of hyperspectral images (HSIs) by integrating the techniques of algebraic multigrid (AMG), hierarchical segmentation (HSEG) and Markov random field (MRF). The proposed framework manifests two main contributions. First, an effective HSI segmentation method is developed by combining the AMG-based marker selection approach and the conventional HSEG algorithm to construct a set of unsupervised segmentation maps in multiple scales. To improve the computational efficiency, the fast Fish Markov selector (FMS) algorithm is exploited for feature selection before image segmentation. Second, an improved MRF energy function is proposed for multiscale information fusion (MIF) by considering both spatial and inter-scale contextual information. Experiments were performed using two airborne HSIs to evaluate the performance of the proposed framework in comparison with several popular classification methods. The experimental results demonstrated that the proposed framework can provide superior performance in terms of both qualitative and quantitative analysis.

Spectral and Spatial Classification of Hyperspectral Images Based on Random Multi-Graphs

Hyperspectral image classification has been acknowledged as the fundamental and challenging task of hyperspectral data processing. The abundance of spectral and spatial information has provided great opportunities to effectively characterize and identify ground materials. In this paper, we propose a spectral and spatial classification framework for hyperspectral images based on Random Multi-Graphs (RMGs). The RMG is a graph-based ensemble learning method, which is rarely considered in hyperspectral image classification. It is empirically verified that the semi-supervised RMG deals well with small sample setting problems. This kind of problem is very common in hyperspectral image applications. In the proposed method, spatial features are extracted based on linear prediction error analysis and local binary patterns; spatial features and spectral features are then stacked into high dimensional vectors. The high dimensional vectors are fed into the RMG for classification. By randomly selecting a subset of features to create a graph, the proposed method can achieve excellent classification performance. The experiments on three real hyperspectral datasets have demonstrated that the proposed method exhibits better performance than several closely related methods.

Sparse Representation-Based Augmented Multinomial Logistic Extreme Learning Machine With Weighted Composite Features for Spectral–Spatial Classification of Hyperspectral Images

Although extreme learning machine (ELM) has been successfully applied to a number of pattern recognition problems, it fails to pro-vide sufficient good results in hyperspectral image (HSI) classification due to two main drawbacks. The first is due to the random weights and bias of ELM, which may lead to ill-posed problems. The second is the lack of spatial information for classification. To tackle these two problems, in this paper, we propose a new framework for ELM based spectral-spatial classification of HSI, where probabilistic modelling with sparse representation and weighted composite features (WCF) are employed respectively to derive the op-timized output weights and extract spatial features. First, the ELM is represented as a concave logarithmic likelihood function under statistical modelling using the maximum a posteriori (MAP). Second, the sparse representation is applied to the Laplacian prior to effi-ciently determine a logarithmic posterior with a unique maximum in order to solve the ill-posed problem of ELM. The variable splitting and the augmented Lagrangian are subsequently used to further reduce the computation complexity of the proposed algorithm and it has been proven a more efficient method for speed improvement. Third, the spatial information is extracted using the weighted compo-site features (WCFs) to construct the spectral-spatial classification framework. In addition, the lower bound of the proposed method is derived by a rigorous mathematical proof. Experimental results on two publicly available HSI data sets demonstrate that the proposed methodology outperforms ELM and a number of state-of-the-art approaches.

Spectral–Spatial Hyperspectral Image Classification via Non-local Means Filtering Feature Extraction

Hyperspectral image (HSI) classification has been a hot topic of research in recent years. The integration of spectral and spatial context is an effective method for HSI classification. This paper proposes a classification method of HSI based on non-local means (NLM) filtering. Firstly, the classification result of HSI is obtained by adopting the support vector machines. Then, the optimization probability image of spatial structure is obtained by using the spatial context information in the first principal component or the first three principal components of HSI to optimize the initial probability map through the NLM filtering. Finally, the final classification results are calculated based on the maximum probability. Experiment results on three real hyperspectral data demonstrate that the proposed NLM filtering based classification method can improve the classification accuracy significantly. Classification results show the effectiveness and superiority of the proposed methods when compared with other methods.

Recent Advances on Spectral–Spatial Hyperspectral Image Classification: An Overview and New Guidelines

Imaging spectroscopy, also known as hyperspectral imaging, has been transformed in the last four decades from being a sparse research tool into a commodity product available to a broad user community. Specially, in the last 10 years, a large number of new techniques able to take into account the special properties of hyperspectral data have been introduced for hyperspectral data processing, where hyperspectral image classification, as one of the most active topics, has drawn massive attentions. Spectral-spatial hyperspectral image classification can achieve better classification performance than its pixel-wise counterpart, since the former utilizes not only the information of spectral signature but also that from spatial domain. In this paper, we provide a comprehensive overview on the methods belonging to the category of spectral-spatial classification in a relatively unified context. First, we develop a concept of spatial dependency system that involves pixel dependency and label dependency, with two main factors: neighborhood covering and neighborhood importance. In terms of the way that the neighborhood information is used, the spatial dependency systems can be classified into fixed, adaptive, and global systems, which can accommodate various kinds of existing spectral-spatial methods. Based on such, the categorizations of single-dependency, bilayer-dependency, and multiple-dependency systems are further introduced. Second, we categorize the performings of existing spectral-spatial methods into four paradigms according to the different fusion stages wherein spatial information takes effect, i.e., preprocessing-based, integrated, postprocessing-based, and hybrid classifications. Then, typical methodologies are outlined. Finally, several representative spectral-spatial classification methods are applied on real-world hyperspectral data in our experiments.

Locality Adaptive Discriminant Analysis for Spectral–Spatial Classification of Hyperspectral Images

Linear discriminant analysis (LDA) is a popular technique for supervised dimensionality reduction, but with less concern about a local data structure. This makes LDA inapplicable to many real-world situations, such as hyperspectral image (HSI) classification. In this letter, we propose a novel dimensionality reduction algorithm, locality adaptive discriminant analysis (LADA) for HSI classification. The proposed algorithm aims to learn a representative subspace of data, and focuses on the data points with close relationship in spectral and spatial domains. An intuitive motivation is that data points of the same class have similar spectral feature and the data points among spatial neighborhood are usually associated with the same class. Compared with traditional LDA and its variants, LADA is able to adaptively exploit the local manifold structure of data. Experiments carried out on several real hyperspectral data sets demonstrate the effectiveness of the proposed method.