Pyramidal Dilation Attention Convolutional Network With Active and Self-Paced Learning for Hyperspectral Image Classification

Hyperspectral image (HSI) classification is the subject of intense research in remote sensing. The tremendous success of deep learning in computer vision has recently sparked the interest in applying deep learning in hyperspectral image classification. However, most deep learning methods for hyperspectral image classification are based on convolutional neural networks (CNN). Those methods require heavy GPU memory resources and run time. Recently, another deep learning model, the transformer, has been applied for image recognition, and the study result demonstrates the great potential of the transformer network for computer vision tasks. In this paper, we propose a model for hyperspectral image classification based on the transformer, which is widely used in natural language processing. Besides, we believe we are the first to combine the metric learning and the transformer model in hyperspectral image classification. Moreover, to improve the model classification performance when the available training samples are limited, we use the 1-D convolution and Mish activation function. The experimental results on three widely used hyperspectral image data sets demonstrate the proposed model’s advantages in accuracy, GPU memory cost, and running time.

Deep neural network has been extensively applied to hyperspectral image (HSI) classification recently. However, its success is greatly attributed to numerous labeled samples, whose acquisition costs a large amount of time and money. In order to improve the classification performance while reducing the labeling cost, this article presents an active deep learning approach for HSI classification, which integrates both active learning and deep learning into a unified framework. First, we train a convolutional neural network (CNN) with a limited number of labeled pixels. Next, we actively select the most informative pixels from the candidate pool for labeling. Then, the CNN is fine-tuned with the new training set constructed by incorporating the newly labeled pixels. This step together with the previous step is iteratively conducted. Finally, Markov random field (MRF) is utilized to enforce class label smoothness to further boost the classification performance. Compared with the other state-of-the-art traditional and deep learning-based HSI classification methods, our proposed approach achieves better performance on three benchmark HSI data sets with significantly fewer labeled samples.

Deep learning has achieved great success in hyperspectral image (HSI) classification. However, its success relies on the availability of sufficient training samples. Unfortunately, the collection of training samples is expensive, time-consuming, and even impossible in some cases. Natural image datasets that are different from HSI, such as Image Net and mini-ImageNet, have abundant texture and structure information. Effective knowledge transfer between two heterogeneous datasets can significantly improve the accuracy of HSI classification. In this letter, heterogeneous few-shot learning (HFSL) for HSI classification is proposed with only a few labeled samples per class. First, few-shot learning is performed on the mini-ImageNet datasets to learn the transferable knowledge. Then, to make full use of the spatial and spectral information, a spectral&#x2013;spatial fusion network is devised. Spectral information is obtained by the residual network with pure 1-D operators. Spatial information is extracted by a convolution network with pure 2-D operators, and the weights of the spatial network are initialized by the weights of the model trained on the mini-ImageNet datasets. Finally, few-shot learning is fine-tuned on HSI to extract discriminative spectral&#x2013;spatial features and individual knowledge, which can improve the classification performance of the new classification task. Experiments conducted on two public HSI datasets demonstrate that the HFSL outperforms the existing few-shot learning methods and supervised learning methods for HSI classification with only a few labeled samples. Our source code is available at <uri>https://github.com/Li-ZK/HFSL</uri>.

Graph convolution network (GCN) has been extensively applied to the area of hyperspectral image (HSI) classification. However, the graph can not effectively describe the complex relationships between HSI pixels and the GCN still faces the challenge of insufficient labeled pixels. In order to alleviate the above two issues faced by the GCN in HSI classification, we propose a novel framework that integrates the active learning and the hypergraph neural network. First, we construct a hypergraph that can reveal the complex non-pairwise relationships embedded in the hyperspectral images. Next, we train a semi-supervised hypergraph neural network (GNN) with the fewer labeled training set. Then, exploiting the local structural properties of the hypergraph, the most useful HSI pixels are actively selected for labeling. Finally, we fine-tune the GNN with original training set along with the newly labeled pixels. And the last three steps are iteratively carried on. Compared with the other traditional and active learning approaches of HSI classification, the proposed active hypergraph neural network (ACGNN) can achieve better performance on the three HSI datasets.

Convolutional neural networks (CNNs) have been extensively studied for hyperspectral image classification (HSIC). However, CNNs are critically attributed to a large number of labeled training samples, which outlays high costs in terms of time and resources. Moreover, CNNs are trained on some samples and have been tested on the entire HSI. Perhaps, the entire HSI is taken into account at test time to appropriately generate the ground-truth maps. To obtain a higher accuracy while considering the limited availability of training samples and disjoint validation and test samples, this work proposes a fast and compact 3-D CNN-based active learning (AL) for HSIC that integrates both deep transfer learning and AL into a unified framework. In the proposed methodology, a 3-D CNN model is trained with very few training samples (i.e., 5%, only) and in the next phase, the most informative and heterogeneous samples are queried from the validation set (candidate set) based on the fuzziness, mutual information, and breaking ties of the trained model. The 3-D CNN model is later fine-tuned (rather than retraining from scratch) with the new training samples (i.e., 200 samples are selected in each iteration) to reduce the computational cost. The proposed method has been compared with the state-of-the-art traditional and deep models proposed for HSIC. Experimental results proved the superiority of our proposed method on several benchmark HSI datasets with significantly fewer labeled samples. MATLAB demo can be accessed on GitHub: github.com/mahmad00

With the rapid development of deep learning technology and improvement in computing capability, deep learning has been widely used in the field of hyperspectral image (HSI) classification. In general, deep learning models often contain many trainable parameters and require a massive number of labeled samples to achieve optimal performance. However, in regard to HSI classification, a large number of labeled samples is generally difficult to acquire due to the difficulty and time-consuming nature of manual labeling. Therefore, many research works focus on building a deep learning model for HSI classification with few labeled samples. In this article, we concentrate on this topic and provide a systematic review of the relevant literature. Specifically, the contributions of this paper are twofold. First, the research progress of related methods is categorized according to the learning paradigm, including transfer learning, active learning and few-shot learning. Second, a number of experiments with various state-of-the-art approaches has been carried out, and the results are summarized to reveal the potential research directions. More importantly, it is notable that although there is a vast gap between deep learning models (that usually need sufficient labeled samples) and the HSI scenario with few labeled samples, the issues of small-sample sets can be well characterized by fusion of deep learning methods and related techniques, such as transfer learning and a lightweight model. For reproducibility, the source codes of the methods assessed in the paper can be found at https://github.com/ShuGuoJ/HSI-Classification.git.

Hyperspectral image (HSI) classification is a challenging problem due to the high dimensional features, high intra-class variance, and limited prior information, and the classification is the basis for HSI applications. Active learning (AL) and semisupervised learning (SSL) are two promising approaches in the HSI classification. In AL, the traditional entropy query-by-bagging (EQB) algorithm only pays attention on uncertainty and ignore the diversity among the samples. Therefore, we propose averaged normalized entropy query-by-bagging (anEQB) algorithm. Meanwhile, the collaborative active learning and semisupervised learning framework (CASSL) may invoke many wrong pseudolabels and deteriorate the classification performance. To make up for the deficiency of CASSL, we complement different AL algorithms to constitute a multiple filtering mode semisupervised learning framework (MFMSLF). To further study, we introduce syncretic secondary filtering mode into multiple verification semisupervised framework and thus constitute a multiple secondary filtering mode semisupervised verification framework (MSFMSVF). We evaluate the performance of anEQB, MFMSLF, and MSFMSVF on different hyperspectral data sets and compare them with other state-of-the-art HSI classification methods. Numerical experimental results reveal the superior classification performance of anEQB, MFMSLF, and MSFMSVF, respectively. Experimental results also demonstrate that exploring the information and diversity of the samples from different criterion can improve the classification performance of the collaborative framework.

One of the challenges in hyperspectral image (HSI) classification is that there are limited labeled samples to train a classifier for very high-dimensional data. In practical applications, we often encounter an HSI domain (called target domain) with very few labeled data, while another HSI domain (called source domain) may have enough labeled data. Classes between the two domains may not be the same. This article attempts to use source class data to help classify the target classes, including the same and new unseen classes. To address this classification paradigm, a meta-learning paradigm for few-shot learning (FSL) is usually adopted. However, existing FSL methods do not account for domain shift between source and target domain. To solve the FSL problem under domain shift, a novel deep cross-domain few-shot learning (DCFSL) method is proposed. For the first time, DCFSL tackles FSL and domain adaptation issues in a unified framework. Specifically, a conditional adversarial domain adaptation strategy is utilized to overcome domain shift, which can achieve domain distribution alignment. In addition, FSL is executed in source and target classes at the same time, which can not only discover transferable knowledge in the source classes but also learn a discriminative embedding model to the target classes. Experiments conducted on four public HSI data sets demonstrate that DCFSL outperforms the existing FSL methods and deep learning methods for HSI classification. Our source code is available at <uri xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">https://github.com/Li-ZK/DCFSL-2021</uri> .

MugNet: Deep learning for hyperspectral image classification using limited samples

Active learning can effectively reduce labelling effort for remote sensing image classification. In this paper, we propose a new active learning method for hyperspectral image classification. We consider batch mode active learning and relatively large amount of data which can be a problem when using current state of the art algorithm based on kernel machines. The active learning procedure is based on the uncertainty sampling strategy and a deep neural network. Stacked autoencoders trained on redundant spatial and spectral features and a few labeled training samples are used to initialize a deep neural network. Uncertainty for a given sample is measured by the difference between the largest two class outputs of the neural network. The less difference there is, the more uncertainty the sample has. Batch of samples with most uncertainty will be selected after label query and added into the training set. Then the neural network is retrained. And such active batch selection will iterate until the budget (the upper limit of label queries) is reached. Experimental results on Pavia university dataset showed that our method outperforms the current support vector machines (SVMs) based multiclass/level uncertainty (MCLU) method both in classification accuracy and generalization capability.

Active learning can effectively reduce labelling effort for remote sensing image classification. In this paper, we propose a new active learning method for hyperspectral image classification. We consider batch mode active learning and relatively large amount of data which can be a problem when using current state of the art algorithm based on kernel machines. The active learning procedure is based on the uncertainty sampling strategy and a deep neural network. Stacked autoencoders trained on redundant spatial and spectral features and a few labeled training samples are used to initialize a deep neural network. Uncertainty for a given sample is measured by the difference between the largest two class outputs of the neural network. The less difference there is, the more uncertainty the sample has. Batch of samples with most uncertainty will be selected after label query and added into the training set. Then the neural network is retrained. And such active batch selection will iterate until the budget (the upper limit of label queries) is reached. Experimental results on Pavia university dataset showed that our method outperforms the current support vector machines (SVMs) based multiclass/level uncertainty (MCLU) method both in classification accuracy and generalization capability.

Deep learning (DL) has proven to be a promising technique for hyperspectral image (HSI) classification. However, due to complex network structure and massive parameters, it is challenging to achieve satisfying classification accuracy with only a small number of training samples. In this letter, we propose a novel HSI classification method by collaborating the 3-D separable ResNet (3-D-SRNet) with cross-sensor transfer learning. The 3-D-SRNet replaces 3-D convolutions with spatial and spectral separable 3-D convolutions, thus showing much less parameters than models that use standard 3-D convolutions. First, we pretrain a classification model with the proposed 3-D-SRNet on the source HSI data set with sufficient training samples compared with the target HSI data set. Then, the pretrained model is transferred to the target HSI data set for fine-tuning to finish the classification task. It is worth noting that the source data for pretraining can be captured by the different sensor with the target data. Compared with the conventional 3-D-ResNet, the proposed 3-D-SRNet has less parameters involving lower computation cost while achieving better classification performance. Experimental results on three benchmark data sets show that our method outperforms several state-of-the-art methods in HSI classification with small training samples.

Hyperspectral image (HSI) classification has recently been successfully explored by using deep learning (DL) methods. However, DL models rely heavily on a large number of labeled samples, which are laborious to obtain. Therefore, finding a way to efficiently embed DL models in limited labeled samples is a hot topic in the field of HSI classification. In this paper, an active learning-based siamese network (ALSN) is proposed to solve the limited labeled samples problem in HSI classification. First, we designed a dual learning-based siamese network (DLSN), which consists of a contrastive learning module and a classification module. Secondly, in view of the problem that active learning is difficult to effectively sample under the extremely limited labeling cost, we proposed an adversarial uncertainty-based active learning (AUAL) method to query valuable samples, and to promote DLSN to learn a more complete feature distribution by fine-tuning. Finally, an active learning architecture, based on inter-class uncertainty (ICUAL), is proposed to construct a lightweight sample pair training set, fully extracting the inter-class information of sample pairs and improving classification accuracy. Experiments on three generic HSI datasets strongly demonstrated the effectiveness of ALSN for HSI classification, with performance improvements over other related DL methods.

Classification of hyperspectral image (HSI) is an important research topic in the remote sensing community. Significant efforts (e.g., deep learning) have been concentrated on this task. However, it is still an open issue to classify the high-dimensional HSI with a limited number of training samples. In this paper, we propose a semi-supervised HSI classification method inspired by the generative adversarial networks (GANs). Unlike the supervised methods, the proposed HSI classification method is semi-supervised, which can make full use of the limited labeled samples as well as the sufficient unlabeled samples. Core ideas of the proposed method are twofold. First, the three-dimensional bilateral filter (3DBF) is adopted to extract the spectral-spatial features by naturally treating the HSI as a volumetric dataset. The spatial information is integrated into the extracted features by 3DBF, which is propitious to the subsequent classification step. Second, GANs are trained on the spectral-spatial features for semi-supervised learning. A GAN contains two neural networks (i.e., generator and discriminator) trained in opposition to one another. The semi-supervised learning is achieved by adding samples from the generator to the features and increasing the dimension of the classifier output. Experimental results obtained on three benchmark HSI datasets have confirmed the effectiveness of the proposed method , especially with a limited number of labeled samples.

Recently, due to the powerful capability at modeling the long-range relationships, Transformer-based methods have been widely explored in many research areas including hyperspectral image (HSI) classification. However, because of lots of trainable parameters and the lack of inductive bias, it is difficult to train a Transformer-based HSI classifier, especially when the number of training samples is limited. To address this issue, in this study, spectral-spatial masked Transformer (SS-MTr) is explored for HSI classification, which uses a two-stage training strategy. In the first stage, SS-MTr pre-trains a vanilla Transformer via reconstruction from masked HSI inputs, which embeds the local inductive bias into the Transformer. In the second stage, the well pre-trained Transformer is cooperated with a fully connected layer and then fine-tuned for the HSI classification. Furthermore, in order to incorporate discriminative feature learning into the SS-MTr, three SS-MTr-based methods, including contrastive SS-MTr (C-SS-MTr), supervised SS-MTr (S-SS-MTr), and supervised contrastive SS-MTr (SC-SS-MTr) are proposed by adding extra branches for specific tasks in parallel with the existing reconstruction task. Specifically, the proposed C-SS-MTr adds a contrastive loss which brings instance discriminability. Besides, the proposed S-SS-MTr builds an extra classification branch for embracing inter-class discriminability and intra-class similarity. Moreover, the proposed SC-SS-MTr combines C-SS-MTr and S-SS-MTr for better generalization. The proposed SS-MTr, C-SS-MTr, S-SS-MTr, and SC-SS-MTr are tested on three popular hyperspectral datasets (i.e., Indian Pines, Pavia University, and Houston). The obtained results reveal that the proposed models achieve competitive results compared with the state-of-the-art HSI classification methods. Code is available at https://github.com/mengduanjinghua/SS-MTr.