Semi-supervised Classification Research Articles

Positive Unlabeled Learning Selected Not At Random (PULSNAR): class proportion estimation without the selected completely at random assumption

Positive and unlabeled (PU) learning is a type of semi-supervised binary classification where the machine learning algorithm differentiates between a set of positive instances (labeled) and a set of both positive and negative instances (unlabeled). PU learning has broad applications in settings where confirmed negatives are unavailable or difficult to obtain, and there is value in discovering positives among the unlabeled (e.g., viable drugs among untested compounds). Most PU learning algorithms make the selected completely at random (SCAR) assumption, namely that positives are selected independently of their features. However, in many real-world applications, such as healthcare, positives are not SCAR (e.g., severe cases are more likely to be diagnosed), leading to a poor estimate of the proportion, α, of positives among unlabeled examples and poor model calibration, resulting in an uncertain decision threshold for selecting positives. PU learning algorithms vary; some estimate only the proportion, α, of positives in the unlabeled set, while others calculate the probability that each specific unlabeled instance is positive, and some can do both. We propose two PU learning algorithms to estimate α, calculate calibrated probabilities for PU instances, and improve classification metrics: i) PULSCAR (positive unlabeled learning selected completely at random), and ii) PULSNAR (positive unlabeled learning selected not at random). PULSNAR employs a divide-and-conquer approach to cluster SNAR positives into subtypes and estimates α for each subtype by applying PULSCAR to positives from each cluster and all unlabeled. In our experiments, PULSNAR outperformed state-of-the-art approaches on both synthetic and real-world benchmark datasets.

Semi-supervised spectral classification of DESI white dwarfs by dimensionality reduction

Abstract As a new generation of large-sky spectroscopic surveys comes online, the enormous data volume poses unprecedented challenges in classifying spectra. Modern unsupervised techniques have the power to group spectra based on their dominant features, circumventing the complete reliance on training data suffered by supervised methods. We outline the use of dimensionality reduction to generate a 2D map of the structure of an intermediate-resolution spectroscopic dataset. This technique efficiently separates white dwarfs of different spectral classes in the Dark Energy Spectroscopic Instrument’s Early Data Release (DESI EDR), identifying spectral features that had been missed even by visual classification. By focusing the method on particular spectral regions, we identify white dwarfs with helium features at 90percnt recall, and cataclysmic variables at 100percnt recall, illustrating rapid selection of low-contamination samples from spectroscopic surveys. We also demonstrate the use of dimensionality reduction in a supervised manner, outlining a procedure to classify any white dwarf spectrum in comparison with those in the DESI EDR. With upcoming surveys promising tens of millions of spectra, our work highlights the potential for semi-supervised techniques as an efficient means of classification and dataset visualisation.

Unsupervised PolSAR Image Classification Based on Superpixel Pseudo-Labels and a Similarity-Matching Network

Supervised polarimetric synthetic aperture radar (PolSAR) image classification demands a large amount of precisely labeled data. However, such data are difficult to obtain. Therefore, many unsupervised methods have been proposed for unsupervised PolSAR image classification. The classification maps of unsupervised methods contain many high-confidence samples. These samples, which are often ignored, can be used as supervisory information to improve classification performance on PolSAR images. This study proposes a new unsupervised PolSAR image classification framework. The framework combines high-confidence superpixel pseudo-labeled samples and semi-supervised classification methods. The experiments indicated that this framework could achieve higher-level effectiveness in unsupervised PolSAR image classification. First, superpixel segmentation was performed on PolSAR images, and the geometric centers of the superpixels were generated. Second, the classification maps of rotation-domain deep mutual information (RDDMI), an unsupervised PolSAR image classification method, were used as the pseudo-labels of the central points of the superpixels. Finally, the unlabeled samples and the high-confidence pseudo-labeled samples were used to train an excellent semi-supervised method, similarity matching (SimMatch). Experiments on three real PolSAR datasets illustrated that, compared with the excellent RDDMI, the accuracy of the proposed method was increased by 1.70%, 0.99%, and 0.8%. The proposed framework provides significant performance improvements and is an efficient method for improving unsupervised PolSAR image classification.

SOAMC: A Semi-Supervised Open-Set Recognition Algorithm for Automatic Modulation Classification

Traditional automatic modulation classification methods operate under the closed-set assumption, which proves to be impractical in real-world scenarios due to the diverse nature of wireless technologies and the dynamic characteristics of wireless propagation environments. Open-set environments introduce substantial technical challenges, particularly in terms of detection effectiveness and computational complexity. To address the limitations of modulation classification and recognition in open-set scenarios, this paper proposes a semi-supervised open-set recognition approach, termed SOAMC (Semi-Supervised Open-Set Automatic Modulation Classification). The primary objective of SOAMC is to accurately classify unknown modulation types, even when only a limited subset of samples is manually labeled. The proposed method consists of three key stages: (1) A signal recognition pre-training model is constructed using data augmentation and adaptive techniques to enhance robustness. (2) Feature extraction and embedding are performed via a specialized extraction network. (3) Label propagation is executed using a graph convolutional neural network (GCN) to efficiently annotate the unlabeled signal samples. Experimental results demonstrate that SOAMC significantly improves classification accuracy, particularly in challenging scenarios with limited amounts of labeled data and high signal similarity. These findings are critical for the practical identification of complex and diverse modulation signals in real-world wireless communication systems.

Ensemble-Enhanced Semi-Supervised Learning with Optimized Graph Construction for High-Dimensional Data.

Graph-based methods have demonstrated exceptional performance in semi-supervised classification. However, existing graph-based methods typically construct either a predefined graph in the original space or an adaptive graph within the output space, which often limits their ability to fully utilize prior information and capture the optimal intrinsic data distribution, particularly in high-dimensional data with abundant redundant and noisy features. This paper introduces a novel approach: Semi-Supervised Classification with Optimized Graph Construction (SSC-OGC). SSC-OGC leverages both predefined and adaptive graphs to explore intrinsic data distribution and effectively employ prior information. Additionally, a graph constraint regularization term (GCR) and a collaborative constraint regularization term (CCR) are incorporated to further enhance the quality of the adaptive graph structure and the learned subspace, respectively. To eliminate the negative effect of constructing a predefined graph in the original data space, we further propose a Hybrid Subspace Ensemble-enhanced framework based on the proposed Optimized Graph Construction method (HSE-OGC). Specifically, we construct multiple hybrid subspaces, which consist of meticulously chosen features from the original data to achieve high-quality and diverse space representations. Then, HSE-OGC constructs multiple predefined graphs within hybrid subspaces and trains multiple SSC-OGC classifiers to complement each other, significantly improving the overall performance. Experimental results conducted on various high-dimensional datasets demonstrate that HSE-OGC exhibits outstanding performance.

Consistency Self-Training Semi-Supervised Method for Road Extraction from Remote Sensing Images

Road extraction techniques based on remote sensing image have significantly advanced. Currently, fully supervised road segmentation neural networks based on remote sensing images require a significant number of densely labeled road samples, limiting their applicability in large-scale scenarios. Consequently, semi-supervised methods that utilize fewer labeled data have gained increasing attention. However, the imbalance between a small quantity of labeled data and a large volume of unlabeled data leads to local detail errors and overall cognitive mistakes in semi-supervised road extraction. To address this challenge, this paper proposes a novel consistency self-training semi-supervised method (CSSnet), which effectively learns from a limited number of labeled data samples and a large amount of unlabeled data. This method integrates self-training semi-supervised segmentation with semi-supervised classification. The semi-supervised segmentation component relies on an enhanced generative adversarial network for semantic segmentation, which significantly reduces local detail errors. The semi-supervised classification component relies on an upgraded mean-teacher network to handle overall cognitive errors. Our method exhibits excellent performance with a modest amount of labeled data. This study was validated on three separate road datasets comprising high-resolution remote sensing satellite images and UAV photographs. Experimental findings showed that our method consistently outperformed state-of-the-art semi-supervised methods and several classic fully supervised methods.

Tri-training algorithm based nuclear power systems semi-supervised fault diagnosis under multiple restricted data conditions

An improved tri-training semi-supervised classification method was proposed as a solution to the restricted data conditions of supervised classification models relying on labeled data, semi-supervised models not considering imbalanced proportion of faults data and the presence of faults data with similar feature or varying severity in existing data-driven nuclear power system fault diagnosis studies. Based on the structure of three identical sub-classifiers in the original tri-training algorithm, data update paths have been added, the updated data selection has been strengthened, and the initial information and permissions of sub-models have been differentiated. The novel method only requires the dataset to meet the Smoothness Assumption, and through multiple strict data update paths, it improves the utilization of unlabeled training data while reducing Pseudo-labeling process errors. The difference in permissions and initial information between sub-classifiers is utilized to improve the training rigor for fault types with smaller sample sizes, in order to alleviate the problem of imbalanced data. Based on a marine nuclear power system secondary circuit model and a widely recognized nuclear power plant dataset, multiple nuclear power system common restricted fault diagnosis datasets faced in ships and power plants were obtained to verify the novel method's advantages and generalization. The conclusion is that under the same conditions, compared with the original tri-training method and other semi-supervised learning methods, the Accuracy, AUC, Precision Rate and Recall Rate of the novel method have improved by about 21 %, 10 %, 20 %, and 21 %, respectively, and can reduce about 25 % of misjudgments between similar feature faults.

Semi-Supervised Modulation Classification via an Ensemble SigMatch Method

Semi-Supervised Modulation Classification via an Ensemble SigMatch Method

GSSCL: A framework for Graph Self-Supervised Curriculum Learning based on clustering label smoothing

Graph self-supervised learning is an effective technique for learning common knowledge from unlabeled graph data through pretext tasks. To capture the interrelationships between nodes and their essential roles globally, existing methods use clustering labels as self-supervised signals. However, in some cases, these methods may introduce noise, leading to over-fitting of the model and a reduction in performance. To address these issues, a novel framework for Graph Self-Supervised Curriculum Learning based on clustering label smoothing called GSSCL has been proposed. GSSCL clusters knowledge in an easy-to-difficult manner, reducing the heavy dependence on the reliability of clustering and improving the generalizability of the model. Moreover, the Silhouette Coefficient is employed to evaluate the clustering confident scores for all nodes. Some nodes are selected based on high confident scores to perform self-supervised learning. To account for the possibility of complex heterophilous information in graphs (e.g., noisy links), clustering pseudo-label smoothing is performed on K-nearest neighbor graphs built upon the similarities between node features instead of the original graph structures. The obtained multi-scale knowledge is then applied to curriculum learning. Finally, comprehensive experiments conducted across diverse public graph benchmarks demonstrate the superior performance of the proposed framework. It exhibits comparable results to state-of-the-art methods across semi-supervised node classification and clustering tasks.

Indoor occupancy monitoring using environmental feature fusion and semi-supervised machine learning models

Smart buildings optimize energy consumption and occupant comfort through Heating, Ventilation and Air Conditioning, and lighting management. Nevertheless, large venues require data fusion techniques to improve analysis and forecasting. This study aims to evaluate the effectiveness of using different feature fusion techniques, environmental sensors, and semi-supervised learning to estimate indoor occupancy in a 230 m2 office. Using five Internet of Things devices measuring air temperature, relative humidity, and barometric pressure, data was collected for 99 days with 6800 entries (on average) and only 14% labeled. Eight feature selection methods were evaluated along with three supervised and two semi-supervised classification methods. Results indicate that the Chi-squared-based approach for feature fusion outperformed others. Similarly, the semi-supervised Self-Training model achieved better performance than the supervised methods. This research shows that combining semi-supervised learning and data fusion allows for estimating the occupancy level in large indoor spaces with high accuracy and low labeling costs. Highlights This study pioneers in exploring semi-supervised learning and distinct feature fusion methods for estimating indoor occupancy levels in a 230 m 2 open office using only Internet of Things (IoT) environmental sensors (air temperature, relative humidity, and barometric pressure). A comprehensive comparison of statistical methods, feature selection, and dimensionality reduction techniques are conducted to determine their ability to generate robust feature fusion sets. The feature fusion selected through the Chi-squared test stood out with a high accuracy F1-score (average of 0.95) and an average accuracy of 0.99. The Self-Training model reached the best performance from semi-supervised learning, with an average F1-Score of 0.90 and an average accuracy of 0.97, based on a dataset with a large proportion of unlabelled data (16,847 entries) and only 9367 labels. For supervised learning, Random Forest achieved a high accuracy (average of 0.98) and F1-score (average of 0.93) across various feature sets.

ST-LP: self-training and label propagation for semi-supervised classification

ST-LP: self-training and label propagation for semi-supervised classification

Semi-Supervised Subcategory Centroid Alignment-Based Scene Classification for High-Resolution Remote Sensing Images

It is usually hard to obtain adequate annotated data for delivering satisfactory scene classification results. Semi-supervised scene classification approaches can transfer the knowledge learned from previously annotated data to remote sensing images with scarce samples for satisfactory classification results. However, due to the differences between sensors, environments, seasons, and geographical locations, cross-domain remote sensing images exhibit feature distribution deviations. Therefore, semi-supervised scene classification methods may not achieve satisfactory classification accuracy. To address this problem, a novel semi-supervised subcategory centroid alignment (SSCA)-based scene classification approach is proposed. The SSCA framework is made up of two components, namely the rotation-robust convolutional feature extractor (RCFE) and the neighbor-based subcategory centroid alignment (NSCA). The RCFE aims to suppress the impact of rotation changes on remote sensing image representation, while the NSCA aims to decrease the impact of intra-category variety across domains on cross-domain scene classification. The SSCA algorithm and several competitive approaches are validated on two datasets to demonstrate its effectiveness. The results prove that the proposed SSCA approach performs better than most competitive approaches by no less than 2% overall accuracy.

GMNI: Achieve good data augmentation in unsupervised graph contrastive learning

Graph contrastive learning (GCL) shows excellent potential in unsupervised graph representation learning. Data augmentation (DA), responsible for generating diverse views, plays a vital role in GCL, and its optimal choice heavily depends on the downstream task. However, it is impossible to measure task-relevant information under an unsupervised setting. Therefore, many GCL methods risk insufficient information by failing to preserve essential information necessary for the downstream task or risk encoding redundant information. In this paper, we propose a novel method called Minimal Noteworthy Information for unsupervised Graph contrastive learning (GMNI), featuring automated DA. It achieves good DA by balancing missing and excessive information, approximating the optimal views in contrastive learning. We employ an adversarial training strategy to generate views that share minimal noteworthy information (MNI), reducing nuisance information by minimization optimization and ensuring sufficient information by emphasizing noteworthy information. Besides, we introduce randomness based on MNI to augmentation, thereby enhancing view diversity and stabilizing the model against perturbations. Extensive experiments on unsupervised and semi-supervised learning over 14 datasets demonstrate the superiority of GMNI over GCL methods with automated and manual DA. GMNI achieves up to a 1.64% improvement over the state-of-the-art in unsupervised node classification, up to a 1.97% improvement in unsupervised graph classification, and up to a 3.57% improvement in semi-supervised graph classification.

Semi-Supervised Interior Decoration Style Classification with Contrastive Mutual Learning

Precisely identifying interior decoration styles holds substantial significance in directing interior decoration practices. Nevertheless, constructing accurate models for the automatic classification of interior decoration styles remains challenging due to the scarcity of expert annotations. To address this problem, we propose a novel pseudo-label-guided contrastive mutual learning framework (PCML) for semi-supervised interior decoration style classification by harnessing large amounts of unlabeled data. Specifically, PCML introduces two distinct subnetworks and selectively utilizes the diversified pseudo-labels generated by each for mutual supervision, thereby mitigating the issue of confirmation bias. For labeled images, the inconsistent pseudo-labels generated by the two subnetworks are employed to identify images that are prone to misclassification. We then devise an inconsistency-aware relearning (ICR) regularization model to perform a review training process. For unlabeled images, we introduce a class-aware contrastive learning (CCL) regularization to learn their discriminative feature representations using the corresponding pseudo-labels. Since the use of distinct subnetworks reduces the risk of both models producing identical erroneous pseudo-labels, CCL can reduce the possibility of noise data sampling to enhance the effectiveness of contrastive learning. The performance of PCML is evaluated on five interior decoration style image datasets. For the average AUC, accuracy, sensitivity, specificity, precision, and F1 scores, PCML obtains improvements of 1.67%, 1.72%, 3.65%, 1.0%, 4.61%, and 4.66% in comparison with the state-of-the-art method, demonstrating the superiority of our method.

Bidirectional consistency with temporal-aware for semi-supervised time series classification

Semi-supervised learning (SSL) has achieved significant success due to its capacity to alleviate annotation dependencies. Most existing SSL methods utilize pseudo-labeling to propagate useful supervised information for training unlabeled data. However, these methods ignore learning temporal representations, making it challenging to obtain a well-separable feature space for modeling explicit class boundaries. In this work, we propose a semi-supervised Time Series classification framework via Bidirectional Consistency with Temporal-aware (TS-BCT), which regularizes the feature space distribution by learning temporal representations through pseudo-label-guided contrastive learning. Specifically, TS-BCT utilizes time-specific augmentation to transform the entire raw time series into two distinct views, avoiding sampling bias. The pseudo-labels for each view, generated through confidence estimation in the feature space, are then employed to propagate class-related information into unlabeled samples. Subsequently, we introduce a temporal-aware contrastive learning module that learns discriminative temporal-invariant representations. Finally, we design a bidirectional consistency strategy by incorporating pseudo-labels from two distinct views into temporal-aware contrastive learning to construct a class-related contrastive pattern. This strategy enables the model to learn well-separated feature spaces, making class boundaries more discriminative. Extensive experimental results on real-world datasets demonstrate the effectiveness of TS-BCT compared to baselines.

Toward Robust Graph Semi-Supervised Learning Against Extreme Data Scarcity.

The success of graph neural networks (GNNs) in graph-based web mining highly relies on abundant human-annotated data, which is laborious to obtain in practice. When only a few labeled nodes are available, how to improve their robustness is key to achieving replicable and sustainable graph semi-supervised learning. Though self-training is powerful for semi-supervised learning, its application on graph-structured data may fail because 1) larger receptive fields are not leveraged to capture long-range node interactions, which exacerbates the difficulty of propagating feature-label patterns from labeled nodes to unlabeled nodes and 2) limited labeled data makes it challenging to learn well-separated decision boundaries for different node classes without explicitly capturing the underlying semantic structure. To address the challenges of capturing informative structural and semantic knowledge, we propose a new graph data augmentation framework, augmented graph self-training (AGST), which is built with two new (i.e., structural and semantic) augmentation modules on top of a decoupled GST backbone. In this work, we investigate whether this novel framework can learn a robust graph predictive model under the low-data context. We conduct comprehensive evaluations on semi-supervised node classification under different scenarios of limited labeled-node data. The experimental results demonstrate the unique contributions of the novel data augmentation framework for node classification with few labeled data.

SNCA: Semi-Supervised Node Classification for Evolving Large Attributed Graphs

SNCA: Semi-Supervised Node Classification for Evolving Large Attributed Graphs

GMMDA: Gaussian mixture modeling of graph in latent space for graph data augmentation

Graph data augmentation (GDA), which manipulates graph structure and/or attributes, has been demonstrated as an effective method for improving the generalization of graph neural networks on semi-supervised node classification. As a data augmentation technique, label preservation is critical, that is, node labels should not change after data manipulation. However, most existing methods overlook the label preservation requirements. Determining the label-preserving nature of a GDA method is highly challenging, owing to the non-Euclidean nature of the graph structure. In this study, for the first time, we formulate a label-preserving problem (LPP) in the context of GDA. The LPP is formulated as an optimization problem in which, given a fixed augmentation budget, the objective is to find an augmented graph with minimal difference in data distribution compared to the original graph. To solve the LPP problem, we propose GMMDA, a generative data augmentation (DA) method based on Gaussian mixture modeling (GMM) of a graph in a latent space. We designed a novel learning objective that jointly learns a low-dimensional graph representation and estimates the GMM. The learning is followed by sampling from the GMM, and the samples are converted back to the graph as additional nodes. To uphold label preservation, we designed a minimum description length (MDL)-based method to select a set of samples that produces the minimum shift in the data distribution captured by the GMM. Through experiments, we demonstrate that GMMDA can improve the performance of graph convolutional network on Cora, Citeseer and Pubmed by as much as 7.75%\\documentclass[12pt]{minimal} \\usepackage{amsmath} \\usepackage{wasysym} \\usepackage{amsfonts} \\usepackage{amssymb} \\usepackage{amsbsy} \\usepackage{mathrsfs} \\usepackage{upgreek} \\setlength{\\oddsidemargin}{-69pt} \\begin{document}$$7.75\\%$$\\end{document}, 8.75%\\documentclass[12pt]{minimal} \\usepackage{amsmath} \\usepackage{wasysym} \\usepackage{amsfonts} \\usepackage{amssymb} \\usepackage{amsbsy} \\usepackage{mathrsfs} \\usepackage{upgreek} \\setlength{\\oddsidemargin}{-69pt} \\begin{document}$$8.75\\%$$\\end{document} and 5.87%\\documentclass[12pt]{minimal} \\usepackage{amsmath} \\usepackage{wasysym} \\usepackage{amsfonts} \\usepackage{amssymb} \\usepackage{amsbsy} \\usepackage{mathrsfs} \\usepackage{upgreek} \\setlength{\\oddsidemargin}{-69pt} \\begin{document}$$5.87\\%$$\\end{document}, respectively, significantly outperforming the state-of-the-art methods.

Semi-supervised lung adenocarcinoma histopathology image classification based on multi-teacher knowledge distillation.

In this study, we propose a semi-supervised learning scheme using a patch-based deep learning framework to tackle the challenge of high-precision classification of seven lung tumor growth patterns, despite having a small amount of labeled data in whole slide images (WSIs). This scheme aims to enhance generalization ability with limited data and reduce dependence on large amounts of labeled data. It effectively addresses the common challenge of high demand for labeled data in medical image analysis.

Approach. To address these challenges, the study employs a semi-supervised learning approach enhanced by a dynamic confidence threshold mechanism. This mechanism adjusts based on the quantity and quality of pseudo labels generated. This dynamic thresholding mechanism helps avoid the imbalance of pseudo-label categories and the low number of pseudo-labels that may result from a higher fixed threshold. Furthermore, the research introduces a multi-teacher knowledge distillation technique. This technique adaptively weights predictions from multiple teacher models to transfer reliable knowledge and safeguard student models from low-quality teacher predictions.

Main results. The framework underwent rigorous training and evaluation using a dataset of 150 WSIs, each representing one of the seven growth patterns. The experimental results demonstrate that the framework is highly accurate in classifying lung tumor growth patterns in histopathology images. Notably, the performance of the framework is comparable to that of fully supervised models and human pathologists. In addition, the framework's evaluation metrics on a publicly available dataset are higher than those of previous studies, indicating good generalizability.

Significance. This research demonstrates that a semi-supervised learning approach can achieve results comparable to fully supervised models and expert pathologists, thus opening new possibilities for efficient and cost-effective medical images analysis. The implementation of dynamic confidence thresholding and multi-teacher knowledge distillation techniques represents a significant advancement in applying deep learning to complex medical image analysis tasks. This advancement could lead to faster and more accurate diagnoses, ultimately improving patient outcomes and fostering the overall progress of healthcare technology.&#xD.

Artificial neural networks with better analysis reliability in data mining

If there are relatively few cases, semi-supervised learning approaches make advantage of a large amount of unlabeled data to assist develop a better classifier. To expand the labeled training set and update the classifier, a fundamental method is to select and label the unlabeled instances for which the current classifier has higher classification confidence. This approach is primarily used in two distinct semi-supervised learning paradigms: co-training and self-training. However, compared to self-labeled examples that would be tagged by a classifier, the real labeled instances will be more trustworthy. Incorrect label assignment to unlabeled occurrences might potentially compromise the classifier's accuracy in classification. This research presents a novel instance selection method based on actual labeled data. This will take into account the classifier's current performance on unlabeled data in addition to its performance on actual labeled data alone. This uses the accuracy changes in the newly trained classifier over the original labeled data as a criterion in each iteration to determine whether or not the selected most confident unlabeled examples would be accepted by a subsequent iteration. Naïve Bayes (NB) will be used as the basic classifier in the co-training and self-training studies. The findings indicate that the accuracy and categorization of self-training and co-training will be greatly enhanced by SIS. As compared to semi-supervised classification methods, it will enhance accuracy, precision, recall, and F1 score, according to the findings.