Limited Labeled Samples Research Articles

Spatial Peak-Aware Collaborative Representation for Hyperspectral Imagery Classification

In this letter, a novel spatial peak-aware collaborative representation (SPaCR) method is proposed for hyperspectral imagery (HSI) classification, which introduces spectral–spatial information among superpixel clusters into regularization terms to construct a new collaborative representation (CR)-based closed-form solution. The proposed method is composed of the following key steps. First, the raw HSI is clustered into many superpixels according to an oversegmentation strategy. Then, cluster pixels are determined based on spectral–spatial correlation between pixels within each superpixel. Next, spectral distance and spatial coherence of superpixel clusters corresponding to training samples and testing pixels are fused to define differences between pixels. Finally, the difference information between clusters as a spectral–spatial feature-induced regularization term is incorporated into the objective function. Experimental results on the Indian Pines and the University of Pavia HSIs indicated that the proposed SPaCR method, without any preprocessing and postprocessing, outperforms well-known and state-of-the-art classifiers on the limited labeled samples.

Deep Ensemble-Based Classifier for Transfer Learning in Rotating Machinery Fault Diagnosis

Nowadays, intelligent models can correctly detect faults by analysing signals from rotating machinery. However, most of the studies are run in controlled environments and the performance in industrial real-world environments is not yet fully validated. Hence, a suitable tool to implement fault diagnosers is transfer learning, this topic is under development and challenges persist. This paper proposes a framework for creating accurate 1D-CNN based fault classifiers that can be transferred between different rotating machines and working conditions. Multiple Bayesian processes select architecture parameters and hyperparameters, which minimize a loss function related to their transferability to other machines and to the same machine under different operating conditions (such as load and engine speed). The resulting model is fitted to heterogeneous fault diagnosis data resulting in a 1D-CNN ensemble that improves the performance of the unitary model. Subsequently, the transfer learning capability of the ensemble is analyzed on two source data sets using function and parameter based transfer. The results are compared with classical fault diagnosis classifiers. Finally, additional transfer operations on five target domain datasets validate our framework on limited labeled samples and allow interpretation of the ensemble results. The ultimate goal is to find an ensemble that can generalize fault diagnosis on rotating machinery for easy implementation and update in industrial settings.

Self-Supervised Divide-and-Conquer Generative Adversarial Network for Classification of Hyperspectral Images

Generative adversarial network (GAN) has been rapidly developed because of its powerful generating ability. However, imbalanced class distribution of hyperspectral images (HSIs) easily causes mode collapse in GAN. Moreover, limited training samples in HSIs restrict the generating ability of GAN. These issues may further deteriorate the classification performance of the discriminator. To conquer these issues, a novel self-supervised divide-and-conquer GAN (SDC-GAN) is proposed for HSI classification. In SDC-GAN, a pretext cluster task with an encoder-decoder architecture is designed by leveraging abundant unlabeled samples. By transferring the learned cluster representation from the cluster task, limited labeled samples are divided effectively in the downstream classification. According to the division of clustering, SDC-GAN constructs a generic and several specific branches for both the generator and discriminator. The generator generates all-class and specific-class samples by using the generic and specific branches separately and combines them adaptively. It can weaken the generation preference for the classes with large sample sizes and alleviate the mode collapse problem. Meanwhile, the classification ability of the discriminator is improved by integrating the judgment of specific branches into the generic branch. Experimental results show that SDC-GAN achieves competitive results for HSI classification compared with several state-of-the-art methods.

Hyperspectral Image Classification Based on Expansion Convolution Network

In recent years, convolutional neural networks (CNNs) have achieved excellent performance in hyperspectral image classification and have been widely used. However, the convolution kernel used in traditional CNN has the limitation of single scale which is not conducive to the improvement of hyperspectral classification performance. In addition, training a classification network of high-dimensional data based on limited labeled samples is still one of the challenges of hyperspectral image classification. To solve the above problems, a hyperspectral image classification method based on expansion convolution network (ECNet) is proposed. The expansion convolution injects holes into the standard convolution kernel to expand the receptive field (RF), so as to extract more context features. Because the shallow features of hyperspectral images contain more location and detail information, while the deep features contain stronger semantic information, in order to further enhance the correlation between deep and shallow information, inspired by ResNet, a similar feedback block (SFB) is introduced on the basis of ECNet, and the deep features and shallow features are fused through this feedback mechanism. Thus, an improved version of ECNet method is obtained, which is called FECNet. This study was tested on four commonly used hyperspectral data sets (i.e. Indian Pine (IP), Pavia University (UP), Kennedy Space Center (KSC), Salinas Valley (SV)) and on a higher resolution and complexly distributed land cover data set (University of Houston (HT)). The experimental results show that the proposed method has better classification performance than some state-of-the art methods, which shows that FECNet has a certain potential in hyperspectral image classification.

Generalized Zero-Shot Domain Adaptation for Unsupervised Cross-Domain PolSAR Image Classification

Cross-domain polarimetric synthetic aperture radar interpretation is urgently needed, due to the rapid data growth and label scarcity. However, the class distribution shift problems hinder the reuse of labeled samples among cross-domain images. Most of the existing domain adaptations can only handle the cross-domain case of same categories between source and target domains, while the categories of the target domain are usually more abundant than those of the source domain. To improve the usability of labeled samples among cross-domain images, an unsupervised generalized zero-shot domain adaptation (uGZSDA) based on scattering component semantics (SCSs) is proposed. By using SCSs and limited labeled samples (seen categories) in the source domain, more land cover types (seen and unseen categories) in the unlabeled target domain can be inferred. First, a stacked autoencoder (SAE) extracts source/target-domain features, and SCSs of typical land covers are constructed by cross-domain databases and statistical scattering components. Second, combining SAE features and source-domain samples, the most likely seen class samples in the target domain are selected by probability sorting, and the SAE is retrained by obtained selected seen samples. Third, the unseen class samples in the target domain are inferred by the retrained SAE, classification probability, and semantic similarity. Finally, the selected seen and inferred unseen class samples in the target domain are used to further retrain the SAE, and the target domain is classified by the retrained SAE and the classifier. The proposed uGZSDA is verified among 16 cross-domain PolSAR datasets. Using SCS and two to three types of seen samples from the source domain, the accuracies of seven types of land covers in the unlabeled target domain can reach 76–83.96%.

Consistency Regularization Auto-Encoder Network for Semi-Supervised Process Fault Diagnosis

Industrial processes are developing towards intelligence and complexity, which brings challenges to intelligent process monitoring. An effective fault diagnosis model plays a vital role in ensuring process safety. However, labeling process samples is time-consuming and costly, which make it hard to obtain enough labeled samples to train an effective diagnostic model. This motivates the development of semi-supervised learning which basic idea is to use unlabeled data to help limited labeled data for model training. In this paper, a consistency regularization auto-encoder (CRAE) framework based on encoder-decoder network is proposed to overcome the problem caused by limited labeled samples. The proposed CRAE captured temporal and spatial correlations from both labeled and unlabeled samples to realize fault diagnosis. Firstly, single process sample is processed as sample matrix using proposed data augmented strategy which can help the proposed method to extract more representative features. Secondly, local encoder and global encoder are proposed to extract local and global temporal and spatial features from sample matrices. Next, the local and global features are fused as the input of decoder network, which improve the ability of reconstruction. Finally, the consistency regularization method is introduced into the encoder-decoder framework to push the decision boundary to the low-density area, which make the distinction among different categories more obvious to help the model better achieve the classification task. Experiments on the Tennessee Eastman process show that the proposed method is effective for process fault diagnosis when labeled samples are limited compared to the other semi-supervised algorithms.

Domain-Adaptive Few-Shot Learning for Hyperspectral Image Classification

Recently, hyperspectral image (HSI) classification by deep learning is flourishing. However, only a few labeled samples are available in practice since it is time-and-labor-consuming to label pixels in HSI (called target domain). This paper proposes a domain-adaptive few-shot learning (DAFSL) method to tackle this problem. Specifically, some other HSIs (called source domain) with large labeled samples are fully used as complementary information and a generative architecture is employed to adapt embedded features in source domain to that of target domain. We first perform domain adaptation with unsupervised learning. In details, the embedded features are generated by the encoder of an autoencoder, where both source and target samples could be well recovered and the reconstruction loss is used to measure the gap between source domain and target domain. At the same time, the embedded features are put into a metric space for classification in source domain and the encoder parameter is fine-tuned together with the classifier in target domain with few labels, so that both general and discriminative features are well captured. The experiment results show that DAFSL outperforms the other mainstream methods with limited labeled samples.

Cross-Domain Contrastive Learning for Hyperspectral Image Classification

Despite the success of deep learning algorithms in hyperspectral image (HSI) classification, most deep learning models require a large amount of labeled data to optimize the numerous parameters. However, it is very expensive and time-consuming to collect a lot of labeled HSI samples. To cope with this problem, we propose a cross-domain contrastive learning (XDCL) framework to learn representations of HSIs in an unsupervised manner. We demonstrate that the features that are valuable for category identification are shared across the spectral and spatial domains, while the less useful contents tend to be independent. The XDCL extracts such domain-invariant information with a cross-domain discrimination task, i.e., predicting which two representations of different domains are matched. With this insight, our method learns semantically meaningful HSI representations. We develop a simple method to construct effective signals representing the two domains, respectively. Moreover, we randomly mask the signals to improve their semantic level and encourage the representations to dig out more useful abstract factors. In order to evaluate the representation quality, we use the learned representations to train a linear classifier on three hyperspectral datasets with limited labeled samples. Experimental results demonstrate that our method surpasses the state-of-the-art methods by a large margin.

A Unified Multiscale Learning Framework for Hyperspectral Image Classification

The highly correlated spectral features and the limited training samples pose challenges in hyperspectral image classification. In this article, to tackle the issues of end-to-end feature learning and transfer learning with limited labeled samples, we propose a unified multiscale learning (UML) framework, which is based on a fully convolutional network. A multiscale spatial-channel attention mechanism and a multiscale shuffle block are proposed in the UML framework to improve the problem of land-cover map distortion. The contextual information and the spectral feature are enhanced before the last classification layer based on three strategies in this work: 1) the channel shuffle operation, which was employed to learn the more effective spectral characteristics by disordering the channels of the feature map; 2) multiscale block, which considered the contextual information in multiple ranges; and 3) spatiospectral attention, which enhanced the expression of the important characteristic among all pixels. Three hyperspectral datasets, including two airborne hyperspectral images and one spaceborne hyperspectral image, were used to demonstrate the performance of the UML framework in both classification and transfer learning. The experimental results confirmed that the proposed method outperforms most of the state-of-the-art hyperspectral image classification methods. The source code is released at <uri xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">https://github.com/Hyper-NN/UML</uri> .

DS4L: Deep Semisupervised Shared Subspace Learning for Hyperspectral Image Classification

Hyperspectral image (HSI) classification is essential in remote sensing image analysis. The classification methods based on deep learning have attracted more and more attention. However, classification accuracy is seriously affected by the quantity of labeled data and redundant information. Therefore, a deep semisupervised shared subspace learning (DS <sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">4</sup> L) model is developed to overcome these problems in this article. DS <sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">4</sup> L is composed of two parts. First, the basic feature extraction (BFE) network is constructed to preliminary extract high-dimensional-space features of multiscale data and fusion them to one shared subspace. Then, a deep shared subspace learning (DSSL) network is proposed to obtain a deeper and more representative low-dimensional subspace. Moreover, to obtain a more representative subspace and alleviate dependence on labeled samples, the regular, irregular constraint, and cross-entropy (CE) loss are integrated into the model. The regular constraint is adopted to reconstruct the multiscale patches to ensure the quality of the subspace in an unsupervised manner. The irregular constraint can well embed labeled and unlabeled samples into the procedure of subspace learning (SL). Then, the CE loss is used to extract more discriminative subspace using the limited labeled samples. Finally, we perform experiments on three widely used HSI datasets. Compared with the basic SL model, the DS <sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">4</sup> L’s classification accuracy on the popular Salinas, Indian Pines, and PaviaU datasets are increased by 4.68%, 4.25%, and 1.57%, respectively.

Spectral–Spatial Hyperspectral Image Classification Based on Multiple Views and Multigraphs Fusion With Few Labeled Samples

The issue of limited labeled samples is still grave in hyperspectral image classification. Semisupervised learning (SSL) utilizing both labeled and unlabeled samples promotes a solution to this issue. However, it has been found that the performance of single SSL is frail when the labeled samples is limited. To tackle this problem, we propose 3D-Gabor and multiple graphs semisupervised framework (3DG-MGSF). The whole framework is tripartite, including multi-views generation and selection, multiple graphs-based label propagation, and double layers classification fusion process. Specifically, a number of 3D-Gabor filters with various directions and frequencies are employed to generate multiple views. Afterwards, a double multi-views selection procedure is applied to ensure the sufficiency and diversity of multiple views. Subsequently, it is time for the multiple graphs-based label propagation to put on its pump. Moreover, spatial and spectral classifications are combined based on the weighted re-fusion algorithm to obtain the final classification. Experimental results illustrate numerically and visually the significantly superior performance of proposed algorithm compared with four state-of-the-art algorithms with few labeled samples.

Active Deep Feature Extraction for Hyperspectral Image Classification Based on Adversarial Learning

The issues of spectral redundancy and limited training samples hinder the widespread application and development of hyperspectral images. In this letter, a novel active deep feature extraction scheme is proposed by incorporating both representative and informative measurement. Firstly, an adversarial autoencoder is modified to suit the classification task with deep feature extraction. Dictionary learning and a multi-variance and distributional distance (MVDD) measure are then introduced to choose the most valuable candidate training samples, where we use the limited labeled samples to obtain a high classification accuracy. Comparative experiments with the proposed querying strategy were carried out with two hyperspectral datasets. The experimental results obtained with the two datasets demonstrate that the proposed scheme is superior to the others. With this method, the unstable increase in accuracy is eliminated by incorporating both informative and representative measurement.

Self-supervised signal representation learning for machinery fault diagnosis under limited annotation data

Recently, convolutional neural networks (CNNs) have achieved remarkable success in machinery fault diagnosis. However, these methods usually require mass of manually labeled data, which is expensive and impractical. To this end, this paper explores the application of self-supervised learning (SS-Learning) paradigm in the field of machinery fault diagnosis, and proposes a new fault diagnosis framework based on self-supervised representation learning. This method can directly learn representative features that can be used for signal classification from unlabeled signals. In addition, it enables the network to have a deeper semantic understanding of vibration signals. In this way, the proposed method can significantly improve the performance of the diagnostic model in the case of limited labeled data. Furthermore, this paper deeply analyzes the mechanism behind the SS-Learning algorithm and the reasons for its excellent performance. The proposed SS-Learning algorithm is verified on three real fault diagnosis datasets high-speed train (HST) wheelset bearing dataset, CWRU dataset and motor bearing dataset). When there are only 50 labeled samples, the proposed SS-Learning algorithm achieves an accuracy of 85% on the motor dataset, which is 17.86% higher than the ordinary CNN. It is proven that the proposed method can provide a powerful supervision signal for feature learning of unlabeled samples and obtain quite competitive fault diagnosis performance with limited labeled samples.

Co-optimization Learning Network for MRI Segmentation of Ischemic Penumbra Tissues.

Convolutional neural networks (CNNs) have brought hope for the medical image auxiliary diagnosis. However, the shortfall of labeled medical image data is the bottleneck that limits the performance improvement of supervised CNN methods. In addition, annotating a large number of labeled medical image data is often expensive and time-consuming. In this study, we propose a co-optimization learning network (COL-Net) for Magnetic Resonance Imaging (MRI) segmentation of ischemic penumbra tissues. COL-Net base on the limited labeled samples and consists of an unsupervised reconstruction network (R), a supervised segmentation network (S), and a transfer block (T). The reconstruction network extracts the robust features from reconstructing pseudo unlabeled samples, which is the auxiliary branch of the segmentation network. The segmentation network is used to segment the target lesions under the limited labeled samples and the auxiliary of the reconstruction network. The transfer block is used to co-optimization the feature maps between the bottlenecks of the reconstruction network and segmentation network. We propose a mix loss function to optimize COL-Net. COL-Net is verified on the public ischemic penumbra segmentation challenge (SPES) with two dozen labeled samples. Results demonstrate that COL-Net has high predictive accuracy and generalization with the Dice coefficient of 0.79. The extended experiment also shows COL-Net outperforms most supervised segmentation methods. COL-Net is a meaningful attempt to alleviate the limited labeled sample problem in medical image segmentation.

Classification of histopathological whole slide images based on multiple weighted semi-supervised domain adaptation

Deep learning has become more important in histopathological images classification for computer-aided cancer diagnosis. However, accurate histopathological image classification based on deep network relies on lots of labeled images, while the expert annotation of whole slide images (WSIs) is time-consuming and laborious. Therefore, how to obtain good classification results with limited labeled samples is still a major challenging task. To overcome the above difficulty, a deep transferred semi-supervised domain adaptation model (HisNet-SSDA) is proposed for classification of histopathological WSIs. Semi-supervised domain adaptation transfers knowledge from a label-rich source domain to a partially labeled target domain. First, a transferred pre-trained network HisNet is designed for high-level feature extraction of the randomly sampled patches from the source and target domains. Then the features of the two domains are aligned through semi-supervised domain adaptation utilizing a multiple weighted loss functions criterion which contains a novel manifold regularization term. The predicted probabilities of sampled patches are aggregated for the image-level classification. Classification results evaluated on two colon cancer datasets demonstrate the remarkable performance of the proposed method (accuracy: 94.32%±0.49%, sensitivity: 94.59%±0.46%, specificity: 94.06%±0.27% and accuracy: 91.92%±0.32%, sensitivity: 92.01%±0.47%, specificity: 91.83%±0.23%), which indicate that the proposed method can be an effective tool for WSIs classification in clinical practice.

Multiscale spectral‐spatial cross‐extraction network for hyperspectral image classification

Convolutional neural networks (CNN) are becoming increasingly popular in modern remote sensing image classification tasks and have exhibited excellent results. For the existing CNN-based hyperspectral image (HSI) classification methods, most of which extract spatial or spectral features separately by convolution. But nearly all of these methods ignore the fact that the weighted summation of convolution may lead to appear new features in another dimension. To address this issue, a novel multiscale spectral-spatial cross-extraction network (MSSCEN) is proposed for HSI classification. Specifically, the proposed MSSCEN introduces spectral-spatial features cross extraction module (SSCEM), which fed extracted features from previous layer into spatial and spectral extraction branches separately again, so that the changes that occurred in the other domain after each convolution can be fully utilized. In addition, a new independent data augmentation module based on U-Net is designed to mitigate the problem of limited labelled samples. The paper conducts experiments on three classic hyperspectral datasets and the results demonstrate that the proposed method achieves the best classification accuracy than other state-of-the-art methods.

Attention-Guided Siamese Fusion Network for Change Detection of Remote Sensing Images

Change detection for remote sensing images is an indispensable procedure for many remote sensing applications, such as geological disaster assessment, environmental monitoring, and urban development monitoring. Through this technique, the difference in certain areas after some emergencies can be determined to estimate their influence. Additionally, by analyzing the sequential difference maps, the change tendency can be found to help to predict future changes, such as urban development and environmental pollution. The complex variety of changes and interferential changes caused by imaging processing, such as season, weather and sensors, are critical factors that affect the effectiveness of change detection methods. Recently, there have been many research achievements surrounding this topic, but a perfect solution to all the problems in change detection has not yet been achieved. In this paper, we mainly focus on reducing the influence of imaging processing through the deep neural network technique with limited labeled samples. The attention-guided Siamese fusion network is constructed based on one basic Siamese network for change detection. In contrast to common processing, besides high-level feature fusion, feature fusion is operated during the whole feature extraction process by using an attention information fusion module. This module can not only realize the information fusion of two feature extraction network branches, but also guide the feature learning network to focus on feature channels with high importance. Finally, extensive experiments were performed on three public datasets, which could verify the significance of information fusion and the guidance of the attention mechanism during feature learning in comparison with related methods.

Deep Spectral Spatial Inverted Residual Network for Hyperspectral Image Classification

Convolutional neural networks (CNNs) have been widely used in hyperspectral image classification in recent years. The training of CNNs relies on a large amount of labeled sample data. However, the number of labeled samples of hyperspectral data is relatively small. Moreover, for hyperspectral images, fully extracting spectral and spatial feature information is the key to achieve high classification performance. To solve the above issues, a deep spectral spatial inverted residuals network (DSSIRNet) is proposed. In this network, a data block random erasing strategy is introduced to alleviate the problem of limited labeled samples by data augmentation of small spatial blocks. In addition, a deep inverted residuals (DIR) module for spectral spatial feature extraction is proposed, which locks the effective features of each layer while avoiding network degradation. Furthermore, a global 3D attention module is proposed, which can realize the fine extraction of spectral and spatial global context information under the condition of the same number of input and output feature maps. Experiments are carried out on four commonly used hyperspectral datasets. A large number of experimental results show that compared with some state-of-the-art classification methods, the proposed method can provide higher classification accuracy for hyperspectral images.

Reverification procedure of pseudolabels based on superpixels segmentation for hyperspectral image classification

The issue of limited labeled samples is still grave in hyperspectral image (HSI) classification. Collaborative learning promotes a solution to this issue by combining active learning (AL) and semisupervised learning. However, it has been found that the performance of wrong pseudolabels added into the iteration may seriously deteriorate classification performance. To tackle this problem, we propose a reverification of pseudolabels algorithm based on superpixels segmentation, which is tripartite, including two AL selection strategies, the first-verification procedure based on three classifiers, and reverification procedure improving the correctness of pseudolabeled samples based on superpixels segmentation. Specifically, two AL strategies mainly generate training samples sets for two check classifiers, and three classifiers are main forces to implement the first-verification for unlabeled samples with predictive labels. Subsequently, a reverification procedure based on local similarity in superpixels is applied to reverify these unlabeled samples with predictive labels passing the first-verification procedure. The proposed algorithm is tested on three widely used HSI datasets and compared with three state-of-the-art collaborative learning algorithms with onefold verification. Experimental results illustrate numerically and visually the significantly superior performance of our proposed algorithm considering the spatial information to reverify the correctness of pseudolabels to unlabeled samples.

Defect classification on limited labeled samples with multiscale feature fusion and semi-supervised learning

Defect classification on limited labeled samples with multiscale feature fusion and semi-supervised learning