Semi-supervised Learning Research Articles

Semi-Supervised Object Detection for Remote Sensing Images Using Consistent Dense Pseudo-Labels

Semi-supervised learning aims to improve the generalization performance of a model by exploiting the large quantity of unlabeled data together with limited labeled data during training. When applied to object detection in remote sensing images, semi-supervised learning can not only effectively alleviate the time-consuming and costly labeling of bounding boxes but also improve the performance and generalization of corresponding object detection methods. However, most current semi-supervised learning-based object detection methods (especially combined with pseudo-labels) for remote sensing images ignore a key issue, that is, the consistency of pseudo-labels. In this paper, a novel semi-supervised learning-based method for object detection in remote sensing images called CDPL is proposed, which includes an adaptive mechanism that directly incorporates the potential object information into the dense pseudo-label selection process and carefully selects the appropriate dense pseudo-labels in the scenes where objects are densely distributed. CDPL consists of two main components: feature-aligned dense pseudo-label selection and sparse pseudo-label-based regression object alignment. The experimental results for typical remote sensing datasets show that the proposed method results in a satisfactory performance improvement.

Decoding cancer prognosis with deep learning: the ASD-cancer framework for tumor microenvironment analysis.

Deep learning is revolutionizing biomedical research by facilitating the integration of multi-omics data sets while bridging classical bioinformatics with existing knowledge. Building on this powerful potential, Zhang et al. proposed a semi-supervised learning framework called Autoencoder-Based Subtypes Detector for Cancer (ASD-cancer) to improve the multi-omics data analysis (H. Zhang, X. Xiong, M. Cheng, et al., 2024, mSystems 9:e01395-24, https://doi.org/10.1128/msystems.01395-24). By utilizing autoencoders pre-trained on The Cancer Genome Atlas data, the ASD-cancer framework outperforms the baseline model. This approach also makes the framework scalable, enabling it to process new data sets through transfer learning without retraining. This commentary explores the methodological innovations and scalability of ASD-cancer while suggesting future directions, such as the incorporation of additional data layers and the development of adaptive AI models through continuous learning. Notably, integrating large language models into ASD-cancer could enhance its interpretability, providing more profound insights into oncological research and increasing its influence in cancer subtyping and further analysis.

Glomerular basement membrane thickness estimation and stratification via active semi-supervised learning model.

The measure of Glomerular Basement Membrane (GBM) thickness is used as diagnostic criteria for kidney glomerular diseases. The GBM thickness measurement, a time-consuming task, is performed by expert pathologists on transmission electron microscopy (TEM) images, therefore, it is affected by subjectivity and reproducibility issues. Here we introduce a fully automated pipeline for the GBM segmentation and successive thickness estimation, starting from TEM images. This method is based on an active semi-supervised learning training procedure of a convolutional neural network model. Starting from the areas automatically identified by the model, we provide a robust measurement of membrane thickness using pixels distance matrix and computer vision techniques. Using these values, we trained a machine learning model to automatically determine the GBM thickness. To verify the accuracy of the method, we compared the predicted results with the full iconographic materials and diagnostic record reports from 42 renal biopsies having normal-thick (n. 21), thin- (n. 10), thick-GBM (n. 11). The obtained segmentations were used for the automated estimation of GBM thickness via computer vision algorithms and compared with manual measurements, obtaining a correlation of Pearson's R2 of 0.85. The GBM thickness was stratified into 3 classes, namely normal, thin, thick with a 0.63 Matthews correlation coefficient and a 0.76 accuracy. The proposed pipeline obtained state-of-the-art performance in GBM segmentation, proving its robustness under image variations, such as magnification, contrast, and complex geometrical shapes. Automated measures could assist clinicians in standard clinical practice speeding up routine procedures with high diagnostic accuracy.

Enhancing ECA (Event- Condition-Action) Rules: Fine-Tuning BERT for Security and Privacy Violation Detection

In the world of Internet of Things, Event Condition Action rules are the secret sauce for smart device interaction. An Event triggers the rule. If a specific Condition is met, then an Action happens automatically. This article addresses trigger–action platforms, which empower users to define custom behaviors for IoT devices and web services through conditional rules. While these platforms enhance user creativity in automation, they also pose significant risks, such as unintentional disclosure of private information or exposure to cyber threats. The proposed solution leverages Natural Language Processing techniques to identify automation rules within these platforms that may compromise user security or privacy. Natural Language Processing based models are applied to analyze the semantic and contextual information of trigger–action rules, utilizing classification techniques on various rule features. Evaluation on the If-This-Then-That platform, using a dataset of 76,741 rules labeled through an ensemble of three semi-supervised learning techniques, demonstrates that the results from the Bidirectional Encoder Representations from Transformers based model training demonstrate promising outcomes, with an average validation accuracy of 89% over 2 epochs. The Test Accuracy of around 90.65% is achieved. Predicted outputs showcase the model's ability to categorize applets into different risk classes, including instances of cyber security threats and physical harm.

Bi-Level Optimization for Semi-Supervised Learning with Pseudo-Labeling

Semi-supervised learning (SSL) is a fundamental task in machine learning, empowering models to extract valuable insights from datasets with limited labeled samples and a large amount of unlabeled data. Although pseudo-labeling is a widely used approach for SSL that generates pseudo-labels for unlabeled data and leverages them as ground truth labels for training, traditional pseudo-labeling techniques often face challenges that significantly decrease the quality of pseudo-labels and hence the overall model performance. In this paper, we propose a novel Bi-level Optimization method for Pseudo-label Learning (BOPL) to boost semi-supervised training. It treats pseudo-labels as latent variables, and optimizes the model parameters and pseudo-labels jointly within a bi-level optimization framework. By enabling direct optimization over the pseudo-labels towards maximizing the prediction model performance, the method is expected to produce high-quality pseudo-labels. To evaluate the effectiveness of the proposed approach, we conduct extensive experiments on multiple SSL benchmarks. The experimental results show the proposed BOPL outperforms the state-of-the-art SSL techniques.

Towards Realistic Semi-supervised Medical Image Classification

Existing semi-supervised learning (SSL) approaches follow the idealized closed-world assumption, neglecting the challenges present in realistic medical scenarios, such as open-set distribution and imbalanced class distribution. Although some methods in natural domains attempt to address the open-set problem, they are insufficient for medical domains, where intertwined challenges like class imbalance and small inter-class lesion discrepancies persist. Thus, this paper presents a novel self-recalibrated semantic training framework, which is tailored for SSL in medical imaging by ingeniously harvesting realistic unlabeled samples. Inspired by the observation that certain open-set samples share some similar disease-related representations with in-distribution samples, we first propose an informative sample selection strategy that identifies high-value samples to serve as augmentations, thereby effectively enriching the semantics of known categories. Furthermore, we adopt a compact semantic clustering strategy to address the semantic confusion raised by the above newly introduced open-set semantics. Moreover, to mitigate the interference of class imbalance in open-set SSL, we introduce a less biased dual-balanced classifier with similarity pseudo-label regularization and category-customized regularization. Extensive experiments on a variety of medical image datasets demonstrate the superior performance of our proposed method over state-of-the-art Closed-set and Open-set SSL methods.

Semi-Supervised Multi-View Multi-Label Learning with View-Specific Transformer and Enhanced Pseudo-Label

Multi-view multi-label learning has become a research focus for describing objects with rich expressions and annotations. However, real-world data often contains numerous unlabeled instances, due to the high cost and technical limitations of manual labeling. This crucial problem involves three main challenges: i) How to extract advanced semantics from available views? ii) How to build a refined classification framework with limited labeled space? iii) How to provide more high-quality supervisory information? To address these problems, we propose a Semi-Supervised Multi-View Multi-Label Learning Method with View-Specific Transformer and Enhanced Pseudo-Label named SMVTEP. Specifically, Generative Adversarial Networks are employed to extract informative shared and specific representations and their consistency and distinctiveness are ensured through the adversarial mechanism and information theory based contrastive learning. Then we build specific classifiers for each extracted feature and apply instance-level manifold constraints to reduce bias across classifiers. Moreover, we design a transformer-style fusion approach that simultaneously captures the imbalance of expressive power among views, mapping effects on specific labels, and label dependencies by incorporating confidence scores and category semantics into the self-attention mechanism. Furthermore, after using Mixup for data augmentation, category-enhanced pseudo-labels are leveraged to improve the reliability of additional annotations by aligning the label distribution of unlabeled samples with the true distribution. Finally, extensive experimental results validate the effectiveness of SMVTEP against state-of-the-art methods.

RegMixMatch: Optimizing Mixup Utilization in Semi-Supervised Learning

Consistency regularization and pseudo-labeling have significantly advanced semi-supervised learning (SSL). Prior works have effectively employed Mixup for consistency regularization in SSL. However, our findings indicate that applying Mixup for consistency regularization may degrade SSL performance by compromising the purity of artificial labels. Moreover, most pseudo-labeling based methods utilize thresholding strategy to exclude low-confidence data, aiming to mitigate confirmation bias; however, this approach limits the utility of unlabeled samples. To address these challenges, we propose RegMixMatch, a novel framework that optimizes the use of Mixup with both high- and low-confidence samples in SSL. First, we introduce semi-supervised RegMixup, which effectively addresses reduced artificial labels purity by using both mixed samples and clean samples for training. Second, we develop a class-aware Mixup technique that integrates information from the top-2 predicted classes into low-confidence samples and their artificial labels, reducing the confirmation bias associated with these samples and enhancing their effective utilization. Experimental results demonstrate that RegMixMatch achieves state-of-the-art performance across various SSL benchmarks.

Semi-supervised medical image segmentation based on dual swap data mixing and cross EMA strategies.

Semi-supervised medical image segmentation methods based on mean teacher (MT) framework provide a promising means for addressing the dense prediction problems with limited annotated images and numerous unlabeled images. However, the confirmation bias caused by the distribution difference between labeled and unlabeled data and the parameters-coupling problem of MT prevent the model from further improving the segmentationperformance. To reduce confirmation bias and alleviate the parameter coupling problem in MT framework, a novel data augmentation strategy and a cross exponential moving averaging (crossEMA) architecture are proposed in thiswork. Specifically, a dual swap mixing data augmentation method was first proposed, which exchanges the patches between labeled and unlabeled images twice to decrease the confirmation bias caused by distribution divergency. Subsequently, a novel architecture for both student and teacher networks was designed with structurally identical dual decoders, one of which adopted a dropout operation. Labeled, unlabeled, and mixed images are fed into this MT architecture. For unlabeled data, the pseudo-labels generated by the dual decoders of the teacher network were used to supervise the predictions of the corresponding decoders of the student network. For mixed data, the real labels of the labeled data are mixed with the pseudo-labels of the unlabeled data predicted by the teacher network to form the supervisory information, which is used to constrain the prediction consistency for mixed data between student and teacher networks. To overcome the parameter coupling problem between the student and teacher networks, the encoder parameters of the teacher network were updated using an exponential moving average (EMA) strategy, while its dual decoder parameters were updated using a cross EMA strategy, which means the perturbed decoder parameters of the student network were updated with the non-perturbed decoder parameters of the student network and viceversa. By comparing with several state-of-the-art (SOTA) semi-supervised segmentation methods on four publicly available datasets, we validated that the proposed method outperforms existing models. The Dice similarity coefficient (DSC) and volume similarity (VS) were improved by at least 2.33% and 1.86%, respectively, compared to the corresponding sub-optimal methods. Through multiple ablation experiments, we verified that the proposed dual swap strategy can reduce the distributional differences between unlabeled data and labeled+mixed data. In addition, the cross EMA strategy can avoid early convergence of the student and teachernetworks. The proposed strategies can alleviate the confirmation bias caused by the distribution discrepancy between labeled and unlabeled data in semi-supervised learning, as well as the issue of parameter coupling between the student and teacher networks in the MT architecture, providing therefore a promising approach to semi-supervised medical image segmentation.

Beyond Virtual Points: Depth-Enhanced LiDAR-only 3D Object Detection with Semi-Supervised Learning (Student Abstract)

The task of 3D object detection is crucial for various applications that rely on identifying objects in three-dimensional space using inputs like LiDAR point clouds and images. However, LiDAR-based detection faces challenges due to the sparsity of point clouds, especially at greater distances. To address this, depth completion models have been used to generate virtual points from RGB images, but they struggle with real-time applications due to high computational costs. Our work eliminates the depth completion process, significantly improving processing speed while minimizing performance degradation. Consequently, our method has achieved an optimal balance between speed and accuracy on the KITTI leaderboard.

SGTC: Semantic-Guided Triplet Co-training for Sparsely Annotated Semi-Supervised Medical Image Segmentation

Although semi-supervised learning has made significant advances in the field of medical image segmentation, fully annotating a volumetric sample slice by slice remains a costly and time-consuming task. Even worse, most of the existing approaches pay much attention to image-level information and ignore semantic features, resulting in the inability to perceive weak boundaries. To address these issues, we propose a novel Semantic-Guided Triplet Co-training (SGTC) framework, which achieves high-end medical image segmentation by only annotating three orthogonal slices of a few volumetric samples, significantly alleviating the burden of radiologists. Our method consist of two main components. Specifically, to enable semantic-aware, fine-granular segmentation and enhance the quality of pseudo-labels, a novel semantic-guided auxiliary learning mechanism is proposed based on the pretrained CLIP. In addition, focusing on a more challenging but clinically realistic scenario, a new triple-view disparity training strategy is proposed, which uses sparse annotations (i.e., only three labeled slices of a few volumes) to perform co-training between three sub-networks, significantly improving the robustness. Extensive experiments on three public medical datasets demonstrate that our method outperforms most state-of-the-art semi-supervised counterparts under sparse annotation settings.

Int*-Match: Balancing Intra-Class Compactness and Inter-Class Discrepancy for Semi-Supervised Speaker Recognition

Open-set speaker recognition is to identify whether the voices are from the same speaker. One challenge of speaker recognition is collecting large amounts of high-quality data. Based on the promising results of image classification, one intuitively feasible solution is semi-supervised learning (SSL) which uses confidence thresholds to assign pseudo labels for unlabeled data. However, we empirically demonstrated that applying SSL methods to speaker recognition is non-trivial. These methods focus solely on inter-class discrepancy as thresholds to select pseudo labels, overlooking intra-class compactness, which is particularly important for open-set speaker recognition tasks. Motivated by this, we propose Int*-Match, a semi-supervised speaker recognition method selecting reliable pseudo labels with intra-class compactness and inter-class discrepancy for speaker recognition. In particular, we use the inter-class discrepancy of labeled data as the threshold for pseudo-label selection and adjust the threshold based on the intra-class compactness of the pseudo labels dynamically and adaptively. Our systematic experiments demonstrate the superiority of Int*-Match, presenting an outstanding Equal Error Rate (EER) of 1.00% on the VoxCeleb1 original test set, which is merely 0.06% below the performance achieved by fully supervised learning.

Dual-calibrated Co-training Framework for Personalized Federated Semi-Supervised Medical Image Segmentation

Federated Semi-Supervised Learning (FSSL) has emerged as a crucial topic in medical image analysis, allowing multiple medical institutions to collaboratively train a global model using limited labeled data. However, existing FSSL methods focus solely on an effective combination of federated learning and semi-supervised learning, ignoring the heterogeneity of client data and the inadaptability of semi-supervised methods in diverse environments, which leads to knowledge bias in local models and impedes stable convergence. To this end, we explore the application of personalization in FSSL and propose a novel dual-calibrated co-training framework. To adapt to the unique feature distribution of client data, we consider collaborative relationships among clients to aggregate a personalized model for each client. We further build a dual-student architecture with the personalized model and private local model on the client side, which encourages model disagreement for co-training while enhancing participant privacy. Most importantly, we design dual calibration strategies that adaptively optimize the model: Local calibration improves the boundary discrimination of the local model by dynamically replacing pseudo-label boundary patches; Global calibration corrects model direction based on the real-time perception of the biases between local dual-student models. Experimental results show the effectiveness of our method on a private medical dataset and two public medical datasets.

Multi-clue Consistency Learning to Bridge Gaps Between General and Oriented Object in Semi-supervised Detection

While existing semi-supervised object detection (SSOD) methods perform well in general scenes, they encounter challenges in handling oriented objects in aerial images. We experimentally find three gaps between general and oriented object detection in semi-supervised learning: 1) Sampling inconsistency: the common center sampling is not suitable for oriented objects with larger aspect ratios when selecting positive labels from labeled data. 2) Assignment inconsistency: balancing the precision and localization quality of oriented pseudo-boxes poses greater challenges which introduces more noise when selecting positive labels from unlabeled data. 3) Confidence inconsistency: there exists more mismatch between the predicted classification and localization qualities when considering oriented objects, affecting the selection of pseudo-labels. Therefore, we propose a Multi-clue Consistency Learning (MCL) framework to bridge gaps between general and oriented objects in semi-supervised detection. Specifically, considering various shapes of rotated objects, the Gaussian Center Assignment is specially designed to select the pixel-level positive labels from labeled data. We then introduce the Scale-aware Label Assignment to select pixel-level pseudo-labels instead of unreliable pseudo-boxes, which is a divide-and-rule strategy suited for objects with various scales. The Consistent Confidence Soft Label is adopted to further boost the detector by maintaining the alignment of the predicted results. Comprehensive experiments on DOTA-v1.5 and DOTA-v1.0 benchmarks demonstrate that our proposed MCL can achieve state-of-the-art performance in the semi-supervised oriented object detection task.

SeFAR: Semi-supervised Fine-grained Action Recognition with Temporal Perturbation and Learning Stabilization

Human action understanding is crucial for the advancement of multimodal systems. While recent developments, driven by powerful large language models (LLMs), aim to be general enough to cover a wide range of categories, they often overlook the need for more specific capabilities. In this work, we address the more challenging task of Fine-grained Action Recognition (FAR), which focuses on detailed semantic labels within shorter temporal duration (e.g., ``salto backward tucked with 1 turn"). Given the high costs of annotating fine-grained labels and the substantial data needed for fine-tuning LLMs, we propose to adopt semi-supervised learning (SSL). Our framework, SeFAR, incorporates several innovative designs to tackle these challenges. Specifically, to capture sufficient visual details, we construct Dual-level temporal elements as more effective representations, based on which we design a new strong augmentation strategy for the Teacher-Student learning paradigm through involving moderate temporal perturbation. Furthermore, to handle the high uncertainty within the teacher model's predictions for FAR, we propose the Adaptive Regulation to stabilize the learning process. Experiments show that SeFAR achieves state-of-the-art performance on two FAR datasets, FineGym and FineDiving, across various data scopes, as well as two classical coarse-grained datasets, UCF101 and HMDB51. Further analysis and ablation studies validate the effectiveness of our designs. Additionally, we show that the features extracted by SeFAR could largely promote the ability of multimodal models to understand fine-grained and domain-specific semantics.

Advancing cardiac MRI multi-structure segmentation: A semi-supervised multidimensional consistency constraint learning network.

Deep convolutional neural networks (DCNNs) have been proposed for medical Magnetic Resonance Imaging (MRI) segmentation, but their effectiveness is often limited by challenges in semantic discrimination, boundary delineation, and spatial context modeling. To address these challenges, we present the Multidimensional Consistency Constraint Learning Network (MDCC-Net) for multi-structure segmentation of cardiac MRI using a semi-supervised approach. MDCC-Net incorporates a shared encoder, multiple differentiated decoders, and leverages pyramid boundary consistency features and spatial consistency constraints. The model employs mutual consistency constraints and pseudo-labels to enhance segmentation performance. Additionally, MDCC-Net uses a combination of Dice loss and mean squared error loss to facilitate convergence and improve accuracy. Experiments on the ACDC cardiac MRI dataset demonstrate that MDCC-Net achieves state-of-the-art performance in multi-structure segmentation of the left ventricle (LV), myocardium (MYO), and right ventricle (RV). Specifically, MDCC-Net attained a Dice coefficient (Dice) of 0.8763 and a Jaccard index of 0.7906 on average. The right ventricle's Average Surface Distance (ASD) reached a best performance of 0.5391, and the left ventricle's Dice attained an optimal value of 0.8965. These results highlight the model's superior ability to utilize semi-supervised data through consistency and entropy minimization constraints. In addition, the generalization of MDCC-Net is verified on the M&Ms dataset. MDCC-Net significantly enhances the multi-structure segmentation of cardiac MRI under multidimensional consistency constraints. This approach provides a foundational study for integrating multifeature fusion in clinical automated and semiautomated multi-organ and multi-tissue segmentation, thus potentially improving diagnostic and treatment planning processes in clinical settings.

Progressive Distribution Matching for Federated Semi-Supervised Learning

Federated Learning (FL) enables collaborative learning from distributed data while preserving the privacy of participating clients. While supervised federated learning with labeled data has made notable strides and achieved success, federated semi-supervised learning (FSSL) lags in its progress. Existing works for FSSL heavily rely on fully-labeled clients, while ignoring the distribution of pseudo-labels generated from skewed unlabeled data. In this work, we offer empirical and theoretical insights into the challenges encountered when applying conventional semi-supervised algorithms in the federated regime. Specifically, we highlight how the inherent data heterogeneity in FSSL can exacerbate issues within the pseudo-labeling process. Motivated by these observations, we propose federated learning with progressive distribution matching (FedPDM) to regularize the distribution of pseudo-labels, aiming to progressively reshape it to align with the ground-truth distribution. The matching problem could be formulated as an optimal transport (OT) problem and efficiently solved by Sinkhorn-Knopp iteration. Through extensive experiments, we demonstrate the superiority of FedPDM on a variety of models and datasets compared with prior arts for FSSL.

Geodesic Flow Kernels for Semi-Supervised Learning on Mixed-Variable Tabular Dataset

Tabular data poses unique challenges due to its heterogeneous nature, combining both continuous and categorical variables. Existing approaches often struggle to effectively capture the underlying structure and relationships within such data. We propose GFTab (Geodesic Flow Kernels for Semi-Supervised Learning on Mixed-Variable Tabular Dataset), a semi-supervised framework specifically designed for tabular datasets. GFTab incorporates three key innovations: 1) Variable-specific corruption methods tailored to the distinct properties of continuous and categorical variables, 2) A Geodesic flow kernel based similarity measure to capture geometric changes between corrupted inputs, and 3) Tree-based embedding to leverage hierarchical relationships from available labeled data. To rigorously evaluate GFTab, we curate a comprehensive set of 21 tabular datasets spanning various domains, sizes, and variable compositions. Our experimental results show that GFTab outperforms existing ML/DL models across many of these datasets, particularly in settings with limited labeled data.

Combating Semantic Contamination in Learning with Label Noise

Noisy labels can negatively impact the performance of deep neural networks. One common solution is label refurbishment, which involves reconstructing noisy labels through predictions and distributions. However, these methods may introduce problematic semantic associations, a phenomenon that we identify as Semantic Contamination. Through an analysis of Robust LR, a representative label refurbishment method, we found that utilizing the logits of views for refurbishment does not adequately balance the semantic information of individual classes. Conversely, using the logits of models fails to maintain consistent semantic relationships across models, which explains why label refurbishment methods frequently encounter issues related to Semantic Contamination. To address this issue, we propose a novel method called Collaborative Cross Learning, which utilizes semi-supervised learning on refurbished labels to extract appropriate semantic associations from embeddings across views and models. Experimental results show that our method outperforms existing approaches on both synthetic and real-world noisy datasets, effectively mitigating the impact of label noise and Semantic Contamination.

Enhancing Generalizability via Utilization of Unlabeled Data for Occupancy Perception

3D occupancy perception accurately estimates the volumetric status and semantic labels of a scene, attracting significant attention in the field of autonomous driving. However, enhancing the model's ability to generalize across different driving scenarios or sensing systems, often requires redesigning the model or extra-expensive annotations. To this end, following a comprehensive analysis of the occupancy model architecture, we proposed the UGOCC method that utilizes domain adaptation to efficiently harness unlabeled autonomous driving data, thereby enhancing the model's generalizability. Specifically, we design the depth fusion module by employing self-supervised depth estimation, and propose a strategy based on semantic attention and domain adversarial learning to improve the generalizability of the learnable fusion module. Additionally, we propose an OCC-specific pseudo-label selection tailored for semi-supervised learning, which optimizes the overall network's generalizability. Our experiment results on two challenging datasets nuScenes and Waymo, demonstrate that our method not only achieves state-of-the-art generalizability but also enhances the model's perceptual capabilities within the source domain by utilizing unlabeled data.