Hyperspectral Image Classification Method Research Articles

Graph Dual Adversarial Network for Hyperspectral Image Classification

An end-to-end unsupervised domain adaptation method for hyperspectral image (HSI) classification based on graph dual adversarial network is proposed in this paper. First, in order to extract the domain invariant features of the source and target domains, the rich spectral information and spatial position of HSI are used to construct a spectral-spatial nearest neighbor graph, which is input into the graph convolutional network. Then, a prototype adversarial strategy is proposed, which uses the labeled data of source domain to reliably calculate the feature prototypes of different classes. Through the prototype adversarial strategy, the distances between different prototypes are appropriately extended, so that the clusters of different classes are far away from their respective decision boundaries, and the discriminability of features are also enhanced. The dual adversarial strategy is composed of the prototype adversarial strategy and domain adversarial strategy. It is worth noting that the dual adversarial strategy does not require feature extractor and discriminator to work in turn, which can be implemented through a gradient reversal layer. Finally, on the basis of adapting the overall features of both domains via the domain adversarial strategy, the source and target domain features are further adapted by minimizing the correlation alignment loss of each class of samples. Experimental results on two real HSI datasets of Botswana and Kennedy Space Center show the effectiveness of our proposed method.

Advances in Hyperspectral Image Classification Methods with Small Samples: A Review

Hyperspectral image (HSI) classification is one of the hotspots in remote sensing, and many methods have been continuously proposed in recent years. However, it is still challenging to achieve high accuracy classification in applications. One of the main reasons is the lack of labeled data. Due to the limitation of spatial resolution, manual labeling of HSI data is time-consuming and costly, so it is difficult to obtain a large amount of labeled data. In such a situation, many researchers turn their attention to the study of HSI classification with small samples. Focusing on this topic, this paper provides a systematic review of the research progress in recent years. Specifically, this paper contains three aspects. First, considering that the taxonomy used in previous review articles is not well-developed and confuses the reader, we propose a novel taxonomy based on the form of data utilization. This taxonomy provides a more accurate and comprehensive framework for categorizing the various approaches. Then, using the proposed taxonomy as a guideline, we analyze and summarize the existing methods, especially the latest research results (both deep and non-deep models) that were not included in the previous reviews, so that readers can understand the latest progress more clearly. Finally, we conduct several sets of experiments and present our opinions on current problems and future directions.

Hyperspectral remote sensing image classification based on residual generative Adversarial Neural Networks

Classification of hyperspectral images is an essential application of deep learning techniques. However, standard deep learning approaches require a large number of labelled samples for model training, and classification performance can be enhanced. In this study, we propose the use of residual generative adversarial networks for the classification of hyperspectral images based on deep learning Convolutional Neural Networks, a large number of labelled samples, and the high classification accuracy required for the classification task. The method is based on generative adversarial networks, with a 6-layer residual network containing up-sampling and convolutional layers replacing the inverse convolutional layer network structure of the generator to enhance the data generation capability, and an 18-layer residual convolutional network replacing the convolutional layer network structure of the discriminator to enhance the feature extraction capability. The hyperspectral image classification method with residual generation adversarial network enhances the information exchange between the shallow network and the deep network by adding a residual structure to the network, extracts the deep features of hyperspectral images and improves the accuracy of hyperspectral image classification. Extensive experiments on several benchmark hyperspectral datasets have shown that the method outperforms comparative methods by 0.7% to 22.3% on OA.

Learning from small data for hyperspectral image classification

With the development of deep learning technique, many hyperspectral image classification (HIC) methods achieve great progress based on the application of convolution neural network (CNN). However, most of them still face small data cases. In this paper, we study the challenges of small data in HIC and propose a stronger HIC small data learning framework (SDLF). More specifically, a feature enhancement module named high-low dimension hybrid representation (HLDR) is designed to improve the feature differentiation power. To generate higher quality classification labels for unlabeled data, a smooth label generation module (SLG) is introduced based on Spatio-Temporal constraint. Then, we introduce a dynamically training sample amplification (TSA) under a unified curriculum learning strategy to add generated reliable labels into training samples. Finally, integrating HLDR in feature representation, SLG in label generation, and TSA in training amplification, the proposed SDLF framework is constructed, which can ensure correctness of HIC network. Experimental results demonstrate that our SDLF can not only learn robust feature representation to enhance the differentiation power but also select reliably generated labels to enrich the training data, which is useful for learning from small data. Also, our SDLF method achieves a superior performance with small labeled training data on three typical datasets.

HyperSFormer: A Transformer-Based End-to-End Hyperspectral Image Classification Method for Crop Classification

Crop classification of large-scale agricultural land is crucial for crop monitoring and yield estimation. Hyperspectral image classification has proven to be an effective method for this task. Most current popular hyperspectral image classification methods are based on image classification, specifically on convolutional neural networks (CNNs) and recurrent neural networks (RNNs). In contrast, this paper focuses on methods based on semantic segmentation and proposes a new transformer-based approach called HyperSFormer for crop hyperspectral image classification. The key enhancement of the proposed method is the replacement of the encoder in SegFormer with an improved Swin Transformer while keeping the SegFormer decoder. The entire model adopts a simple and uniform transformer architecture. Additionally, the paper introduces the hyper patch embedding (HPE) module to extract spectral and local spatial information from the hyperspectral images, which enhances the effectiveness of the features used as input for the model. To ensure detailed model processing and achieve end-to-end hyperspectral image classification, the transpose padding upsample (TPU) module is proposed for the model’s output. In order to address the problem of insufficient and imbalanced samples in hyperspectral image classification, the paper designs an adaptive min log sampling (AMLS) strategy and a loss function that incorporates dice loss and focal loss to assist model training. Experimental results using three public hyperspectral image datasets demonstrate the strong performance of HyperSFormer, particularly in the presence of imbalanced sample data, complex negative samples, and mixed sample classes. HyperSFormer outperforms state-of-the-art methods, including fast patch-free global learning (FPGA), a spectral–spatial-dependent global learning framework (SSDGL), and SegFormer, by at least 2.7% in the mean intersection over union (mIoU). It also improves the overall accuracy and average accuracy values by at least 0.9% and 0.3%, respectively, and the kappa coefficient by at least 0.011. Furthermore, ablation experiments were conducted to determine the optimal hyperparameter and loss function settings for the proposed method, validating the rationality of these settings and the fusion loss function.

A Feature Embedding Network with Multiscale Attention for Hyperspectral Image Classification

In recent years, convolutional neural networks (CNNs) have been widely used in the field of hyperspectral image (HSI) classification and achieved good classification results due to their excellent spectral–spatial feature extraction ability. However, most methods use the deep semantic features at the end of the network for classification, ignoring the spatial details contained in the shallow features. To solve the above problems, this article proposes a hyperspectral image classification method based on a Feature Embedding Network with Multiscale Attention (MAFEN). Firstly, a Multiscale Attention Module (MAM) is designed, which is able to not only learn multiscale information about features at different depths, but also extract effective information from them. Secondly, the deep semantic features can be embedded into the low-level features through the top-down channel, so that the features at all levels have rich semantic information. Finally, an Adaptive Spatial Feature Fusion (ASFF) strategy is introduced to adaptively fuse features from different levels. The experimental results show that the classification accuracies of MAFEN on four HSI datasets are better than those of the compared methods.

Improving Feature Learning in Remote Sensing Images Using an Integrated Deep Multi-Scale 3D/2D Convolutional Network

Developing complex hyperspectral image (HSI) sensors that capture high-resolution spatial information and voluminous (hundreds) spectral bands of the earth’s surface has made HSI pixel-wise classification a reality. The 3D-CNN has become the preferred HSI pixel-wise classification approach because of its ability to extract discriminative spectral and spatial information while maintaining data integrity. However, HSI datasets are characterized by high nonlinearity, voluminous spectral features, and limited training sample data. Therefore, developing deep HSI classification methods that purely utilize 3D-CNNs in their network structure often results in computationally expensive models prone to overfitting when the model depth increases. In this regard, this paper proposes an integrated deep multi-scale 3D/2D convolutional network block (MiCB) for simultaneous low-level spectral and high-level spatial feature extraction, which can optimally train on limited sample data. The strength of the proposed MiCB model solely lies in the innovative arrangement of convolution layers, giving the network the ability (i) to simultaneously convolve the low-level spectral with high-level spatial features; (ii) to use multiscale kernels to extract abundant contextual information; (iii) to apply residual connections to solve the degradation problem when the model depth increases beyond the threshold; and (iv) to utilize depthwise separable convolutions in its network structure to address the computational cost of the proposed MiCB model. We evaluate the efficacy of our proposed MiCB model using three publicly accessible HSI benchmarking datasets: Salinas Scene (SA), Indian Pines (IP), and the University of Pavia (UP). When trained on small amounts of training sample data, MiCB is better at classifying than the state-of-the-art methods used for comparison. For instance, the MiCB achieves a high overall classification accuracy of 97.35%, 98.29%, and 99.20% when trained on 5% IP, 1% UP, and 1% SA data, respectively.

Hyperspectral Image Classification Based on Fusion of Convolutional Neural Network and Graph Network

Convolutional neural networks (CNNs) have attracted significant attention as a commonly used method for hyperspectral image (HSI) classification in recent years; however, CNNs can only be applied to Euclidean data and have difficulties in dealing with relationships due to their limitations of local feature extraction. Each pixel of a hyperspectral image contains a set of spectral bands that are correlated and interact with each other, and the methods used to process Euclidean data cannot effectively obtain these correlations. In contrast, the graph convolutional network (GCN) can be used in non-Euclidean data but usually leads to over-smoothing and ignores local detail features due to the need for superpixel segmentation processing to reduce computational effort. To overcome the above problems, we constructed a fusion network based on the GCN and CNN which contains two branches: a graph convolutional network based on superpixel segmentation and a convolutional network with an added attention mechanism. The graph convolutional branch can extract the structural features and capture the relationships between the nodes, and the convolutional branch can extract detailed features in the local fine region. Owing to the fact that the features extracted from the two branches are different, the classification performance can be improved by fusing the complementary features extracted from the two branches. To validate the proposed algorithm, experiments were conducted on three widely used datasets, namely Indian Pines, Pavia University, and Salinas. An overall accuracy of 98.78% was obtained in the Indian Pines dataset, and overall accuracies of 98.99% and 98.69% were obtained in the other two datasets. The results show that the proposed fusion network can obtain richer features and achieve a high classification accuracy.

Network Collaborative Pruning Method for Hyperspectral Image Classification Based on Evolutionary Multi-Task Optimization

Neural network models for hyperspectral images classification are complex and therefore difficult to deploy directly onto mobile platforms. Neural network model compression methods can effectively optimize the storage space and inference time of the model while maintaining the accuracy. Although automated pruning methods can avoid designing pruning rules, they face the problem of search efficiency when optimizing complex networks. In this paper, a network collaborative pruning method is proposed for hyperspectral image classification based on evolutionary multi-task optimization. The proposed method allows classification networks to perform the model pruning task on multiple hyperspectral images simultaneously. Knowledge (the important local sparse structure of the network) is automatically searched and updated by using knowledge transfer between different tasks. The self-adaptive knowledge transfer strategy based on historical information and dormancy mechanism is designed to avoid possible negative transfer and unnecessary consumption of computing resources. The pruned networks can achieve high classification accuracy on hyperspectral data with limited labeled samples. Experiments on multiple hyperspectral images show that the proposed method can effectively realize the compression of the network model and the classification of hyperspectral images.

MS3Net: Multiscale stratified-split symmetric network with quadra-view attention for hyperspectral image classification

Recently, hyperspectral image (HSI) classification has become a promising research direction in remote sensing image processing. Many HSI classification methods have been proposed based on convolutional neural networks (CNNs) and attention mechanisms (AMs). However, most current CNN-based methods only consider extracting features at a single scale in HSI, which may ignore the delicate features of some objects. Moreover, present AMs primarily focus on one feature dimension, such as spatial or channel attention, while disregarding dimension interaction. To conquer the above issues, a novel multiscale stratified-split symmetric network with quadra-view attention, namely MS3Net, is proposed for HSI classification. Generally, the proposed MS3Net has a dual-stream symmetric pipeline, which can better extract HSI's spectral signatures and spatial features. Specifically, the proposed MS3Net consists of three modules: a multiscale feature extraction module, a feature enhancement module, and a feature fusion module. Firstly, a stratified-split module is designed to extract multiscale spectral and spatial features. In addition, to reduce the complexity of the model, we designed pseudo-3-D spectral and spatial convolution to replace the traditional 3-D convolution operation. Secondly, a novel quadra-view attention module is proposed, guiding the model to focus on important features from multiple dimensions. Finally, the selective kernel feature fusion module is introduced, which can dynamically integrate spectral and spatial features. Experimental results on four benchmark HSI datasets with different scenes and resolutions confirm the visual and quantitative superiority of the proposed MS3Net over the state-of-the-art related methods in this research direction.

Hyperspectral Image Classification Based on Dense Pyramidal Convolution and Multi-Feature Fusion

In recent years, hyperspectral image classification techniques have attracted a lot of attention from many scholars because they can be used to model the development of different cities and provide a reference for urban planning and construction. However, due to the difficulty in obtaining hyperspectral images, only a limited number of pixels can be used as training samples. Therefore, how to adequately extract and utilize the spatial and spectral information of hyperspectral images with limited training samples has become a difficult problem. To address this issue, we propose a hyperspectral image classification method based on dense pyramidal convolution and multi-feature fusion (DPCMF). In this approach, two branches are designed to extract spatial and spectral features, respectively. In the spatial branch, dense pyramid convolutions and non-local blocks are used to extract multi-scale local and global spatial features in image samples, which are then fused to obtain spatial features. In the spectral branch, dense pyramidal convolution layers are used to extract spectral features in image samples. Finally, the spatial and spectral features are fused and fed into fully connected layers to obtain classification results. The experimental results show that the overall accuracy (OA) of the method proposed in this paper is 96.74%, 98.10%, 98.92% and 96.67% on the four hyperspectral datasets, respectively. Significant improvements are achieved compared to the five methods of SVM, SSRN, FDSSC, DBMA and DBDA for hyperspectral classification. Therefore, the proposed method can better extract and exploit the spatial and spectral information in image samples when the number of training samples is limited. Provide more realistic and intuitive terrain and environmental conditions for urban planning, design, construction and management.

Active learning with Bayesian CNN using the BALD method for Hyperspectral Image Classification

Deep learning DL techniques have recently been used to examine the classification of remote sensing data like hyperspectral images HSI. However, DL models are difficult to obtain since they rely largely on a large number of labeled training data. Therefore, a current challenge in the field of HSI classification is how to effectively incorporate DL models in constrained labeled data. The Bayesian Convolutional Neural Networks BCNN method is robust against overfitting on small datasets. One of the key methods for automating data selection is active learning AL, which has gained popularity in recent decades. By choosing the most informative samples, AL aims to reduce the costly data labeling procedure and build a robust training set that is resource-efficient. In this work, we aim to improve the performance of BCNN using AL method to build a competitive classifier considering the Bayesian Active Learning Disagreement BALD acquisition function (Dropout Bayesian Active Learning by Disagreement), which incorporates model uncertainty information. In a previous work, BCNN was built and applied on Pavia datasets giving 99.7% classification accuracy. For comparison traditional BCNN with BALD, The techniques were applied on the Indian Pines dataset. The average accuracy of the classification had increased from 90% to 98% using BALD method.

A new deep learning approach for hyperspectral image classification based on multifeature local kernel descriptors

During the last few years, many deep learning (DL) methods have been proposed for the classification of hyperspectral images (HSI), and the final results show that these models have great potential. However, most existing deep models only consider the original spectral HSI datacube, excluding other sources of information such as textural and geometrical features. In addition to their computational complexity, most DL approaches cannot effectively consider local nonlinear relationships between features. In order to address these issues, this paper develops a new DL approach called MLKD-CNN, which uses multifeature local kernel descriptors (MLKDs) and convolutional neural networks (CNN) to classify HSIs. This method produces local kernel descriptors (LKDs) of spectral, extended morphological profiles, and Gabor features in the feature generation step. A simple CNN is then used to classify MLKD generated from spatial and spectral features. The proposed MLKD considers local nonlinear relationships of both spectral and spectral features, making them capable of achieving high accuracy levels while reducing the computational burden of deep methods. The experiments on four benchmark HSIs show that the proposed MLKD-CNN outperforms some recent HSI classification methods in terms of accuracy and computation time.

Multi-scale spectral-spatial dual-transformer network for hyperspectral image classification

ABSTRACT Deep learning methods have shown great advantages in hyperspectral image (HSI) classification tasks. In particular, convolutional neural network (CNN)-based methods for HSI classification have made great progress. However, it is difficult for CNN to process long-range spatial and spectral information. Recently, a novel deep learning model called transformer has demonstrated its potential to replace CNN in various classification tasks with its amazing performance. In this letter, a multi-scale spectral-spatial dual-transformer network (MS3DT) is proposed to deeply consider the spectral-spatial features via transformer. Specifically, MS3DT consists of a feature pyramid network (FPN), a spectral transformer subnetwork (SPECT) and a spatial transformer subnetwork (SPAT). To utilize the complementary multi-scale characteristics, we introduce FPN to capture shallow-to-deep and spectral-spatial features. To improve the representational capacity in spatial and spectral domains, SPECT is exploited to extract long-range spectral correlation over local spectral features, and SPAT is designed to explore the contextual information for better refinement. Therefore, MS3DT is able to adaptively recalibrate the nonlinear interdependence of shallow-to-deep spectral-spatial features by merging the two subnetworks. Experiments on two widely used HSI datasets show that the MS3DT can outperform the state-of-the-art (SOTA) algorithm. The source codes will be available at https://github.com/RsAI-lab/MSSSDT.

A Hyperspectral Image Classification Method Based on the Nonlocal Attention Mechanism of a Multiscale Convolutional Neural Network.

Recently, convolution neural networks have been widely used in hyperspectral image classification and have achieved excellent performance. However, the fixed convolution kernel receptive field often leads to incomplete feature extraction, and the high redundancy of spectral information leads to difficulties in spectral feature extraction. To solve these problems, we propose a nonlocal attention mechanism of a 2D-3D hybrid CNN (2-3D-NL CNN), which includes an inception block and a nonlocal attention module. The inception block uses convolution kernels of different sizes to equip the network with multiscale receptive fields to extract the multiscale spatial features of ground objects. The nonlocal attention module enables the network to obtain a more comprehensive receptive field in the spatial and spectral dimensions while suppressing the information redundancy of the spectral dimension, making the extraction of spectral features easier. Experiments on two hyperspectral datasets, Pavia University and Salians, validate the effectiveness of the inception block and the nonlocal attention module. The results show that our model achieves an overall classification accuracy of 99.81% and 99.42% on the two datasets, respectively, which is higher than the accuracy of the existing model.

A Deep Sparse Representation with Random Dictionary for Hyperspectral Image Classification

Hyperspectral image (HSI) classification methods based on deep learning have demonstrated excellent performance, while these deep learning methods take a lot of time to train the parameters. In this paper, we propose a deep sparse representation (SR) network (DSRNet) without spending a lot of time training network parameters in the feature extraction stage. The contributions of this paper are three-fold. First, we introduce random dictionary into HSI classification, and solve sparse representation model under this dictionary. Second, we extend the shallow sparse representation model to the deep sparse representation model, where the SR model needs to be solved for each layer and used to extract the deep features of HSI. Finally, we investigate the classification performance of different classifiers on the deep features extracted by using DSRNet. Experimental results show that the proposed method can achieve better classification results compared with some closely related HSI classification methods and the other state-of-the-art deep learning methods.

Hyperspectral image classification method based on squeeze-and-excitation networks, depthwise separable convolution and multibranch feature fusion

Hyperspectral image classification method based on squeeze-and-excitation networks, depthwise separable convolution and multibranch feature fusion

Small Sample Hyperspectral Image Classification Based on the Random Patches Network and Recursive Filtering

In recent years, different deep learning frameworks were introduced for hyperspectral image (HSI) classification. However, the proposed network models have a higher model complexity, and do not provide high classification accuracy if few-shot learning is used. This paper presents an HSI classification method that combines random patches network (RPNet) and recursive filtering (RF) to obtain informative deep features. The proposed method first convolves image bands with random patches to extract multi-level deep RPNet features. Thereafter, the RPNet feature set is subjected to dimension reduction through principal component analysis (PCA), and the extracted components are filtered using the RF procedure. Finally, the HSI spectral features and the obtained RPNet-RF features are combined to classify the HSI using a support vector machine (SVM) classifier. In order to test the performance of the proposed RPNet-RF method, some experiments were performed on three widely known datasets using a few training samples for each class, and classification results were compared with those obtained by other advanced HSI classification methods adopted for small training samples. The comparison showed that the RPNet-RF classification is characterized by higher values of such evaluation metrics as overall accuracy and Kappa coefficient.

An efficient hyperspectral image classification method for limited training data

AbstractHyperspectral image classification has gained great progress in recent years based on deep learning model and massive training data. However, it is expensive and unpractical to label hyperspectral image data and implement model in constrained environment. To address this problem, this paper proposes an effective ghost module based spectral network for hyperspectral image classification. First, Ghost3D module is adopted to reduce the size of model parameter dramatically by redundant feature maps generation with linear transformation. Then Ghost2D module with channel‐wise attention is used to explore informative spectral feature representation. For large field covering, the non‐local operation is utilized to promote self‐attention. Compared with the state‐of‐the‐art hyperspectral image classification methods, the proposed approach achieves superior performance on three hyperspectral image data sets with fewer sample labelling and less resource consumption.

A Novel Hyperspectral Image Classification Method Using Class-Weighted Domain Adaptation Network

With the development of science and technology, hyperspectral image (HSI) classification has been studied in depth by researchers as one of the important means of human cognition in living environments and the exploration of surface information. Nevertheless, the shortage of labeled samples is a major difficulty in HSI classification. To address this issue, we propose a novel HSI classification method called class-weighted domain adaptation network (CWDAN). First, the convolutional domain adaption network (ConDAN) is designed to align the marginal distributions and second-order statistics, respectively, of both domains via multi-kernel maximum mean discrepancy (MK-MMD) and CORAL loss. Then, the class-weighted MMD (CWMMD) is defined to simultaneously consider the conditional distribution discrepancy and changes of class prior distributions, and the CWMMD-based domain adaptation term is incorporated into the classical broad learning system (BLS) to construct the weighted conditional broad network (WCBN). The WCBN is applied to reduce the conditional distribution discrepancy and class weight bias across domains, while performing breadth expansion on domain-invariant features to further enhance representation ability. In comparison with several existing mainstream methods, CWDAN has excellent classification performance on eight real HSI data pairs when only using labeled source domain samples.