Hyperspectral Imagery Classification Research Articles

Convolutional transformer attention network with few-shot learning for grassland degradation monitoring using UAV hyperspectral imagery

ABSTRACT In recent years, the desertification of grasslands has increased due to various factors, including both global warming and human activities. It is an essential basis for grassland degradation monitoring to monitor the dynamic change of desert grassland vegetation communities and distribution statistics. Although unmanned aerial vehicle (UAV) remote sensing images have allowed us to achieve dynamic real-time grassland monitoring, the distribution of desert grassland ground objects can be random and narrow, thus increasing the difficulty of sample labelling of remote sensing imagery. Therefore, to reduce the number of samples required for the model, this research proposes a convolutional transformer attention network (CTAN) to identify desert grassland ground objects and validate it on a self-collected desert grassland dataset. The network utilizes the transformer model to enhance its learning of global pixels so that it suppresses the transmission of background pixels within the network. Furthermore, the edge convolution module is designed to strengthen the network’s learning for edge pixels, improving its identification effect. The results show that the network provides 97.22% of overall accuracy (OA), 94.35% of average accuracy (AA), and 0.9398 of Kappa for ground object recognition in desert grassland. The model improves OA by 2.36–9.85% points compared to methods in the same field and 0.8–6.35% points compared to methods in hyperspectral imagery classification. The experimental results show the superior performance of the CTAN model for recognizing desert grassland objects, which helps the management and restoration of desert grasslands.

Spectrally Segmented-Enhanced Neural Network for Precise Land Cover Object Classification in Hyperspectral Imagery

The paradigm shift brought by deep learning in land cover object classification in hyperspectral images (HSIs) is undeniable, particularly in addressing the intricate 3D cube structure inherent in HSI data. Leveraging convolutional neural networks (CNNs), despite their architectural constraints, offers a promising solution for precise spectral data classification. However, challenges persist in object classification in hyperspectral imagery or hyperspectral image classification, including the curse of dimensionality, data redundancy, overfitting, and computational costs. To tackle these hurdles, we introduce the spectrally segmented-enhanced neural network (SENN), a novel model integrating segmentation-based, multi-layer CNNs, SVM classification, and spectrally segmented dimensionality reduction. SENN adeptly integrates spectral–spatial data and is particularly crucial for agricultural land classification. By strategically fusing CNNs and support vector machines (SVMs), SENN enhances class differentiation while mitigating overfitting through dropout and early stopping techniques. Our contributions extend to effective dimensionality reduction, precise CNN-based classification, and enhanced performance via CNN-SVM fusion. SENN harnesses spectral information to surmount challenges in “hyperspectral image classification in hyperspectral imagery”, marking a significant advancement in accuracy and efficiency within this domain.

FPGA-based remote target classification in hyperspectral imaging using multi-graph neural network

Hyperspectral imagery (HSI) is widely used in remote sensing for target classification; however, its accurate classification remains challenging due to the scarcity of labeled data. Graph Neural Networks (GNNs) have emerged as a popular method for semi-supervised classification, attracting significant interest in the context of HSI analysis. Nevertheless, conventional GNN-based approaches often rely on a single graph filter to extract HSI characteristics, failing to fully exploit the potential benefits of different graph filters. Additionally, oversmoothing issues plague classical GNNs, further affecting classification performance. To address these drawbacks, we propose a novel approach called Spectral and Autoregressive Moving Average Graph Filter for the Multi-Graph Neural Network (SAM-GNN). This approach leverages two distinct graph filters: one specialized in extracting the spectral characteristics of nodes and the other effectively suppressing graph distortion. Through extensive evaluations, we compare the performance of SAM-GNN with other state-of-the-art methods, employing metrics such as overall accuracy (OA), individual class accuracy (IA), and Kappa coefficient (KC). The results shows that the SAM-GNN provides an improvement in KC, IA, and OA of 6.71%, 5.7%, and 3.93% for the Pavia University dataset and 4.67%, 3.67%, and 3.49% for the Cuprite dataset respectively. Furthermore, we implement SAM-GNN on the Virtex-7 field-programmable gate array (FPGA), demonstrating that the method achieves highly accurate target localization results, bringing us closer to real-world applications in HSI classification.

Research on near-ground forage hyperspectral imagery classification based on fusion preprocessing process

ABSTRACTAccurate identification and classification of forage grass are pivotal in optimizing forage resources and breeding superior forage varieties. Given the low accuracy in forage image identification and classification, and the loss of some features from preprocessing, we proposed an innovative approach that integrates preprocessing operations directly into the model instead of preceding feature analysis. We captured near-ground hyperspectral imagery of forage in the field and applied two deep learning models – Squeeze and Excitation ResNet (SEResNet) and Convolution Block Attention Module ResNet (CBAMResNet). These models not only harness the automatic learning capabilities of the ResNet deep network but also employ channel attention and a channel-plus-space dual attention mechanism to filter and label important features. This approach enhances data extraction and analysis, strengthen the correlation between the channel and space dimensions while eliminating redundancy and noise. We compared the performance of the proposed methods with the current popular methods by six evaluation parameters, including overall accuracy (OA), average accuracy (AA), Kappa coefficient, etc. Experiment results show the OA of SEResNet and CBAMResNet are 96.57% and 98.35% respectively. The experiments demonstrate the feasibility of incorporating preprocessing into the network and the effectiveness of the new idea for the classification research of forage.

Central Attention Network for Hyperspectral Imagery Classification.

In this article, the intrinsic properties of hyperspectral imagery (HSI) are analyzed, and two principles for spectral-spatial feature extraction of HSI are built, including the foundation of pixel-level HSI classification and the definition of spatial information. Based on the two principles, scaled dot-product central attention (SDPCA) tailored for HSI is designed to extract spectral-spatial information from a central pixel (i.e., a query pixel to be classified) and pixels that are similar to the central pixel on an HSI patch. Then, employed with the HSI-tailored SDPCA module, a central attention network (CAN) is proposed by combining HSI-tailored dense connections of the features of the hidden layers and the spectral information of the query pixel. MiniCAN as a simplified version of CAN is also investigated. Superior classification performance of CAN and miniCAN on three datasets of different scenarios demonstrates their effectiveness and benefits compared with state-of-the-art methods.

Hybrid Convolutional Network Combining Multiscale 3D Depthwise Separable Convolution and CBAM Residual Dilated Convolution for Hyperspectral Image Classification

In recent years, convolutional neural networks (CNNs) have been increasingly leveraged for the classification of hyperspectral imagery, displaying notable advancements. To address the issues of insufficient spectral and spatial information extraction and high computational complexity in hyperspectral image classification, we introduce the MDRDNet, an integrated neural network model. This novel architecture is comprised of two main components: a Multiscale 3D Depthwise Separable Convolutional Network and a CBAM-augmented Residual Dilated Convolutional Network. The first component employs depthwise separable convolutions in a 3D setting to efficiently capture spatial–spectral characteristics, thus substantially reducing the computational burden associated with 3D convolutions. Meanwhile, the second component enhances the network by integrating the Convolutional Block Attention Module (CBAM) with dilated convolutions via residual connections, effectively counteracting the issue of model degradation. We have empirically evaluated the MDRDNet’s performance by running comprehensive experiments on three publicly available datasets: Indian Pines, Pavia University, and Salinas. Our findings indicate that the overall accuracy of the MDRDNet on the three datasets reached 98.83%, 99.81%, and 99.99%, respectively, which is higher than the accuracy of existing models. Therefore, the MDRDNet proposed in this study can fully extract spatial–spectral joint information, providing a new idea for solving the problem of large model calculations in 3D convolutions.

Hyperspectral Image Classification via Spatial Shuffle-Based Convolutional Neural Network

The unique spatial–spectral integration characteristics of hyperspectral imagery (HSI) make it widely applicable in many fields. The spatial–spectral feature fusion-based HSI classification has always been a research hotspot. Typically, classification methods based on spatial–spectral features will select larger neighborhood windows to extract more spatial features for classification. However, this approach can also lead to the problem of non-independent training and testing sets to a certain extent. This paper proposes a spatial shuffle strategy that selects a smaller neighborhood window and randomly shuffles the pixels within the window. This strategy simulates the potential patterns of the pixel distribution in the real world as much as possible. Then, the samples of a three-dimensional HSI cube is transformed into two-dimensional images. Training with a simple CNN model that is not optimized for architecture can still achieve very high classification accuracy, indicating that the proposed method of this paper has considerable performance-improvement potential. The experimental results also indicate that the smaller neighborhood windows can achieve the same, or even better, classification performance compared to larger neighborhood windows.

Three-dimensional singular spectrum analysis for precise land cover classification from UAV-borne hyperspectral benchmark datasets

The precise classification of land covers with hyperspectral imagery (HSI) is a major research-focused topic in remote sensing, especially using unmanned aerial vehicle (UAV) systems as the abundant data sources have brought severe intra-class spectral variability and high spatial heterogeneity challenges, making precise classification difficult. To this end, a novel three-dimensional singular spectrum analysis (3DSSA) method is proposed for the 3D feature extraction of HSI. It aims to construct a low-rank trajectory tensor containing global and local features and extract both spectral discrimination features and spatial contextual features in conjunction with tensor singular value decomposition (t-SVD). To reduce the risk of tensor operations exceeding memory on large-scale HSI data, the extended regional clustering (RC) 3DSSA framework (RC-3DSSA) is proposed for precise HSI classification. RC-3DSSA uses RC processing to alleviate the scale diversity and further applies 3DSSA to tackle issues of intra-class spectral variability and spatial heterogeneity. In order to effectively evaluate the performance of RC-3DSSA, a new challenging classification dataset namely the Qingdao UAV-borne HSI (QUH) dataset was further built. It consists of three sub-datasets: QUH-Tangdaowan, QUH-Qingyun, and QUH-Pingan, which are freely available as benchmarks for precise land cover classification. The experimental results on QUH and two publicly available datasets show that the RC-3DSSA can accurately distinguish ground objects and reliably map their distribution when benchmarked with ten state-of-the-art methods. Specifically, the overall accuracies achieved are 86.62%, 87.51%, and 87.35% under 10% spatially disjoint training samples for the three UAV-borne HSI datasets, respectively, providing the best performance.

Unsupervised dimensionality reduction of medical hyperspectral imagery in tensor space

Background and objectivesCompared with traditional RGB images, medical hyperspectral imagery (HSI) has numerous continuous narrow spectral bands, which can provide rich information for cancer diagnosis. However, the abundant spectral bands also contain a large amount of redundancy information and increase computational complexity. Thus, dimensionality reduction (DR) is essential in HSI analysis. All vector-based DR methods ignore the cubic nature of HSI resulting from vectorization. To overcome the disadvantage of vector-based DR methods, tensor-based techniques have been developed by employing multi-linear algebra. MethodsTo fully exploit the structure features of medical HSI and enhance computational efficiency, a novel method called unsupervised dimensionality reduction via tensor-based low-rank collaborative graph embedding (TLCGE) is proposed. TLCGE introduces entropy rate superpixel (ERS) segmentation algorithm to generate superpixels. Then, a low-rank collaborative graph weight matrix is constructed on each superpixel, greatly improving the efficiency and robustness of the proposed method. After that, TLCGE reduces dimensions in tensor space to well preserve intrinsic structure of HSI. ResultsThe proposed TLCGE is tested on cholangiocarcinoma microscopic hyperspectral data sets. To further demonstrate the effectiveness of the proposed algorithm, other machine learning DR methods are used for comparison. Experimental results on cholangiocarcinoma microscopic hyperspectral data sets validate the effectiveness of the proposed TLCGE. ConclusionsThe proposed TLCGE is a tensor-based DR method, which can maintain the intrinsic 3-D data structure of medical HSI. By imposing the low-rank and sparse constraints on the objective function, the proposed TLCGE can fully explore the local and global structures within each superpixel. The computational efficiency of the proposed TLCGE is better than other tensor-based DR methods, which can be used as a preprocessing step in real medical HSI classification or segmentation.

Graph neural network-based remote target classification in hyperspectral imaging

ABSTRACT Hyperspectral Imagery (HSI) onboard target classification is a frequently used technique in remote sensing. Due to the small volume of labelled data, the classification of HSI has been a challenging subject in recent study. The graph neural network (GNN) has become a popular method for semi-supervised classification, and its use with hyperspectral images has gained a lot of interest. A graph filter or single graph neural network has been typically employed in the earlier GNN-based approaches, and it is used to obtain HSI characteristics. It has not fully utilized the benefits of different graph filters or graph neural networks. The issue of oversmoothing also persists with the classical GNNs. We propose a spectral filter and an autoregressive moving average filter for the multi-graph neural network; (SAF-GNN) to address such drawbacks as the one mentioned above. The first is excellent at extracting the nodes’ spectral characteristics, and the second is effective at suppressing graph distortion. With the help of performance measures like overall accuracy (OA), individual class accuracy (ICA), and Kappa coefficient (KC), the performance of the proposed method was compared to that of other state-of-the-art methods. The result shows that the SAF-GNN provides an improvement in OA, ICA and KC of 2.21%, 7.93% and 0.025 for the cuprite dataset and 4.84%, 11.78% and 0.025 for the Pavia University dataset, respectively. Comparison results show that the proposed method is suitable for remote target classification problem.

MSRA-G: Combination of multi-scale residual attention network and generative adversarial networks for hyperspectral image classification

Deep learning-based technology has been introduced to increase the classification accuracy of hyperspectral imagery (HSI). Nevertheless, it is still a challenging issue to derive a satisfying classification accuracy from limited training samples. A novel method (MSRA-G) that combines multi-scale residual attention (MSRA) with Generative Adversarial Networks (GANs) was proposed. In view of the low classification accuracy with limited training samples, the is first used to generate more separable synthetic samples. A network is then proposed to extract multi-scale context information for improving HSI classification. The proposed method constructs two multi-scale feature extraction modules to identify high-level spatial–spectral features based on the 3D–2D hybrid network. In addition, the residual connection mode and the attention mechanism are combined to establish the channel and spatial residual attention modules. Different weights are assigned to different features in the channel dimension and spatial dimension, and the features are selectively learned. Furthermore, to verify the performance of MSRA-G, experiments were carried out on three publicly available HSI datasets of Indian Pines, University of Pavia and Salinas Valley. The experimental results show that our proposed MSRA-G is superior to several popular classification models. It can still achieve satisfactory classification accuracies, even in the case of insufficient training samples.

Semi-supervised feature learning for disjoint hyperspectral imagery classification

With the introduction of spatial-spectral fusion and deep learning, the classification performance of hyperspectral imagery (HSI) has been promoted greatly. For some widely used datasets, the classification accuracy almost reaches 100%. However, for hyperspectral image classification, random sampling is still the most common strategy to collect the training and test samples. Because the training and test samples are randomly selected from the same image, so they have a high correlation and the classification results are overoptimistic. Besides, random sampling is not a good choice for practical applications because we cannot always collect training and test samples from the same region. Disjoint sampling selects training and testing samples from different local regions, which will provide a more objective performance evaluation for HSI classification models. In this paper, we first show the huge classification performance difference caused by different sampling strategies with a simple experiment, then we analyze the underlying reasons from the spectral information, spatial-spectral combination, sample overlapping and spatial distance, finally, a semi-supervised feature learning method is proposed for disjoint HSI classification, in which the spatial and spectral information are exploited effectively and reasonably. The experimental results based on three HSI datasets demonstrate the effectiveness of the proposed method.

Local–Global Active Learning Based on a Graph Convolutional Network for Semi-Supervised Classification of Hyperspectral Imagery

Deep learning is being increasingly employed for hyperspectral classification, although such use is often predicated on the availability of a sufficiently large set of labeled samples for training. To improve classification performance under a limited training-set size, a semi-supervised network with end-to-end local-global active learning based on graph convolutional networks is proposed. The proposed active learning extracts both global as well as local graph-based features to gauge the discriminative information in unlabeled samples, while semi-supervised classification expands the training set of a fully supervised classifier by attaching pseudo-labels to high-confidence unlabeled samples. Experimental results demonstrate that the proposed network outperforms not only other approaches to semi-supervised classification but also several existing fully supervised methods. The source code of this method can be found at https://github.com/XtaoS/semi-LG-AGCN.

Semi-Supervised Neural Architecture Search for Hyperspectral Imagery Classification Method With Dynamic Feature Clustering

Hyperspectral image(HSI) contains rich spatial and spectral information, which makes HSI classification task the research focus of HSI analysis within remote sensing community. Though deep learning based HSI classification methods obtain good performance in recent years, how to learn network structure better suitable for a given HSI instead of utilizing a manually designed one for HSI classification is still a challenging problem, especially providing only small amount of labeled samples. To address this problem, we propose the first semi-supervised HSI classification network constructed via the neural architecture search. Specifically, we propose a two-head semi-supervised HSI classification framework utilizing both labeled and unlabeled data, which consists of a shared feature extraction module, a classifier module for labeled samples together with a clustering module for unlabeled samples. To boost the performance of the constructed two-head network, we propose to utilize deep features instead of the original pixels for HSI clustering to generate pseudo labels for the unlabeled data. Within the conducted semi-supervised network, we specifically design a method to automatically search for the shared feature extraction module better suitable for the given HSI data, which leads to better HSI classification results. Experimental results on three HSI datasets demonstrate the effectiveness of the proposed method, providing only limited number of labeled training samples.

Analysis of patch and sample size effects for 2D-3D CNN models using multiplatform dataset: hyperspectral image classification of ROSIS and Jilin-1 GP01 imagery

Modern hyperspectral sensors provide a huge volume of data at spectral and spatial domains with high redundancy, which requires robust methods for analysis. In this study, 2D and 3D CNN models were applied to hyperspectral image datasets (ROSIS and Jilin-1 GP01) using varying patch and sample sizes to determine their combined impacts on the performance of deep learning models. Differences in classification performances in relation to particle and sample sizes were statistically analysed using McNemar?s test. According to the findings, raising the patch and sample size enhances the performance of the 2D/3D CNN model and produces more accurate results in the classification of hyperspectral imagery. To be more specific, the thematic maps produced with 400 and 600 samples resulted in a notable increase in overall accuracies (approximately 10%) compared to 50 and 100 samples per class. Statistical test results confirmed the importance of patch size selection for the case of limited training data and the maximum number of samples required for deep learning. It could be also mentioned that an appropriate patch and sample size should be selected considering the characteristics of the dataset and the type of deep learning algorithms.

A Comparison of Three Different Group Intelligence Algorithms for Hyperspectral Imagery Classification

The classification effect of hyperspectral remote sensing images is greatly affected by the problem of dimensionality. Feature extraction, as a common dimension reduction method, can make up for the deficiency of the classification of hyperspectral remote sensing images. However, different feature extraction methods and classification methods adapt to different conditions and lack comprehensive comparative analysis. Therefore, principal component analysis (PCA), linear discriminant analysis (LDA), and locality preserving projections (LPP) were selected to reduce the dimensionality of hyperspectral remote sensing images, and subsequently, support vector machine (SVM), random forest (RF), and the k-nearest neighbor (KNN) were used to classify the output images, respectively. In the experiment, two hyperspectral remote sensing data groups were used to evaluate the nine combination methods. The experimental results show that the classification effect of the combination method when applying principal component analysis and support vector machine is better than the other eight combination methods.

Hyperspectral imagery classification with minimum noise fraction, 2D spatial filtering and SVM

Hyperspectral image (HSI) classification is an important topic in remote sensing. In this paper, we propose a new method for HSI classification by using minimum noise fraction (MNF), spatial filtering (SF) and support vector machine (SVM). We use MNF to reduce the dimensionality of a hyperspectral data cube before performing classification. We apply 2D SF to the DR output band images and then use SVM to classify the pixels of the data cube. In this way, both spatial information and spectral information are taken into consideration in the classification. Experimental results show that our MNF+SF method is extremely competitive when compared to several existing classification methods.

Mapping Roofing with Asbestos-Containing Material by Using Remote Sensing Imagery and Machine Learning-Based Image Classification: A State-of-the-Art Review

Building roofing produced with asbestos-containing materials is a significant concern due to its detrimental health hazard implications. Efficiently locating asbestos roofing is essential to proactively mitigate and manage potential health risks from this legacy building material. Several studies utilised remote sensing imagery and machine learning-based image classification methods for mapping roofs with asbestos-containing materials. However, there has not yet been a critical review of classification methods conducted in order to provide coherent guidance on the use of different remote sensing images and classification processes. This paper critically reviews the latest works on mapping asbestos roofs to identify the challenges and discuss possible solutions for improving the mapping process. A peer review of studies addressing asbestos roof mapping published from 2012 to 2022 was conducted to synthesise and evaluate the input imagery types and classification methods. Then, the significant challenges in the mapping process were identified, and possible solutions were suggested to address the identified challenges. The results showed that hyperspectral imagery classification with traditional pixel-based classifiers caused large omission errors. Classifying very-high-resolution multispectral imagery by adopting object-based methods improved the accuracy results of ACM roof identification; however, non-optimal segmentation parameters, inadequate training data in supervised methods, and analyst subjectivity in rule-based classifications were reported as significant challenges. While only one study investigated convolutional neural networks for asbestos roof mapping, other applications of remote sensing demonstrated promising results using deep-learning-based models. This paper suggests further studies on utilising Mask R-CNN segmentation and 3D-CNN classification in the conventional approaches and developing end-to-end deep semantic classification models to map roofs with asbestos-containing materials.

ALPN: Active-Learning-Based Prototypical Network for Few-Shot Hyperspectral Imagery Classification

With the development of deep learning, the benchmark of hyperspectral imagery classification is constantly improving, but there are still significant challenges for hyperspectral imagery classification of few-shot scenes. This letter proposes an active-learning-based prototypical network (ALPN), which uses the prototypical network to extract representative features from a few samples. Moreover, it combines semisupervised clustering and active learning methods to select and request labels from valuable examples actively. In this way, the feature extraction ability of the network is gradually optimized. The experimental results validated that the classification accuracy and robustness of ALPN significant exceeded the comparison baselines. Furthermore, because it can be regarded as a sample selection method, ALPN can be easily combined with other models to obtain better classification results.

Hyperspectral Imagery Classification Based on Contrastive Learning

Supervised machine learning and deep learning methods perform well in hyperspectral image classification. However, hyperspectral images have few labeled samples, which make them difficult to be trained because supervised classification methods rely heavily on sample quantity and quality. Inspired by the idea of self-supervised learning, this article proposes a hyperspectral imagery classification algorithm based on contrast learning, which uses the information of abundant unlabeled samples to alleviate the problem of insufficient label information in hyperspectral data. The algorithm uses a two-stage training strategy. In the first stage, the model is pretrained in the way of self-supervised learning, using a large number of unlabeled samples combined with data enhancement to construct positive and negative sample pairs, and contrastive learning (CL) is carried out. The purpose is to enable the model to make judgments on positive and negative samples. In the second stage, based on the pretrained model, the features of the hyperspectral image are extracted for classification, and a small amount of labeled samples are used to fine-tune the features. Experiments show that the features extracted by self-supervised learning achieved improved results on downstream classification task.