Robust Semi-supervised Learning Research Articles

Robust Structure-Aware Graph-based Semi-Supervised Learning: Batch and Recursive Processing

Graph-based semi-supervised learning plays an important role in large scale image classification tasks. However, the problem becomes very challenging in the presence of noisy labels and outliers. Moreover, traditional robust semi-supervised learning solutions suffers from prohibitive computational burdens thus cannot be computed for streaming data. Motivated by that, we present a novel unified framework robust structure-aware semi-supervised learning called Unified RSSL (URSSL) for batch processing and recursive processing robust to both outliers and noisy labels. Particularly, URSSL applies joint semi-supervised dimensionality reduction with robust estimators and network sparse regularization simultaneously on the graph Laplacian matrix iteratively to preserve the intrinsic graph structure and ensure robustness to the compound noise. First, in order to relieve the influence from outliers, a novel semi-supervised robust dimensionality reduction is applied relying on robust estimators to suppress outliers. Meanwhile, to tackle noisy labels, the denoised graph similarity information is encoded into the network regularization. Moreover, by identifying strong relevance of dimensionality reduction and network regularization in the context of robust semi-supervised learning (RSSL), a two-step alternative optimization is derived to compute optimal solutions with guaranteed convergence. We further derive our framework to adapt to large scale semi-supervised learning particularly suitable for large scale image classification and demonstrate the model robustness under different adversarial attacks. For recursive processing, we rely on reparameterization to transform the formulation to unlock the challenging problem of robust streaming-based semi-supervised learning. Last but not least, we extend our solution into distributed solutions to resolve the challenging issue of distributed robust semi-supervised learning when images are captured by multiple cameras at different locations. Extensive experimental results demonstrate the promising performance of this framework when applied to multiple benchmark datasets with respect to state-of-the-art approaches for important applications in the areas of image classification and spam data analysis.

Triplet Adaptation Framework for Robust Semi-Supervised Learning.

Semi-supervised learning (SSL) suffers from severe performance degradation when labeled and unlabeled data come from inconsistent and imbalanced distribution. Nonetheless, there is a lack of theoretical guidance regarding a remedy for this issue. To bridge the gap between theoretical insights and practical solutions, we embark to an analysis of generalization bound of classic SSL algorithms. This analysis reveals that distribution inconsistency between unlabeled and labeled data can cause a significant generalization error bound. Motivated by this theoretical insight, we present a Triplet Adaptation Framework (TAF) to reduce the distribution divergence and improve the generalization of SSL models. TAF comprises three adapters: Balanced Residual Adapter, aiming to map the class distribution of labeled and unlabeled data to a uniform distribution for reducing class distribution divergence; Representation Adapter, aiming to map the representation distribution of unlabeled data to labeled one for reducing representation distribution divergence; and Pseudo-Label Adapter, aiming to align the predicted pseudo-labels with the class distribution of unlabeled data, thereby preventing erroneous pseudo-labels from exacerbating representation divergence. These three adapters collaborate synergistically to reduce the generalization bound, ultimately achieving a more robust and generalizable SSL model. Extensive experiments across various robust SSL scenarios validate the efficacy of our method.

Phenome-wide identification of therapeutic genetic targets, leveraging knowledge graphs, graph neural networks, and UK Biobank data.

The ongoing expansion of human genomic datasets propels therapeutic target identification; however, extracting gene-disease associations from gene annotations remains challenging. Here, we introduce Mantis-ML 2.0, a framework integrating AstraZeneca's Biological Insights Knowledge Graph and numerous tabular datasets, to assess gene-disease probabilities throughout the phenome. We use graph neural networks, capturing the graph's holistic structure, and train them on hundreds of balanced datasets via a robust semi-supervised learning framework to provide gene-disease probabilities across the human exome. Mantis-ML 2.0 incorporates natural language processing to automate disease-relevant feature selection for thousands of diseases. The enhanced models demonstrate a 6.9% average classification power boost, achieving a median receiver operating characteristic (ROC) area under curve (AUC) score of 0.90 across 5220 diseases from Human Phenotype Ontology, OpenTargets, and Genomics England. Notably, Mantis-ML 2.0 prioritizes associations from an independent UK Biobank phenome-wide association study (PheWAS), providing a stronger form of triaging and mitigating against underpowered PheWAS associations. Results are exposed through an interactive web resource.

Unknown-Aware Graph Regularization for Robust Semi-supervised Learning from Uncurated Data

Recent advances in semi-supervised learning (SSL) have relied on the optimistic assumption that labeled and unlabeled data share the same class distribution. However, this assumption is often violated in real-world scenarios, where unlabeled data may contain out-of-class samples. SSL with such uncurated unlabeled data leads training models to be corrupted. In this paper, we propose a robust SSL method for learning from uncurated real-world data within the context of open-set semi-supervised learning (OSSL). Unlike previous works that rely on feature similarity distance, our method exploits uncertainty in logits. By leveraging task-dependent predictions of logits, our method is capable of robust learning even in the presence of highly correlated outliers. Our key contribution is to present an unknown-aware graph regularization (UAG), a novel technique that enhances the performance of uncertainty-based OSSL frameworks. The technique addresses not only the conflict between training objectives for inliers and outliers but also the limitation of applying the same training rule for all outlier classes, which are existed on previous uncertainty-based approaches. Extensive experiments demonstrate that UAG surpasses state-of-the-art OSSL methods by a large margin across various protocols. Codes are available at https://github.com/heejokong/UAGreg.

Robust Semi-Supervised Learning With Multi-Consistency and Data Augmentation

Robust Semi-Supervised Learning With Multi-Consistency and Data Augmentation

Robust Semi-Supervised Learning by Wisely Leveraging Open-Set Data.

Open-set Semi-supervised Learning (OSSL) holds a realistic setting that unlabeled data may come from classes unseen in the labeled set, i.e., out-of-distribution (OOD) data, which could cause performance degradation in conventional SSL models. To handle this issue, except for the traditional in-distribution (ID) classifier, some existing OSSL approaches employ an extra OOD detection module to avoid the potential negative impact of the OOD data. Nevertheless, these approaches typically employ the entire set of open-set data during their training process, which may contain data unfriendly to the OSSL task that can negatively influence the model performance. This inspires us to develop a robust open-set data selection strategy for OSSL. Through a theoretical understanding from the perspective of learning theory, we propose Wise Open-set Semi-supervised Learning (WiseOpen), a generic OSSL framework that selectively leverages the open-set data for training the model. By applying a gradient-variance-based selection mechanism, WiseOpen exploits a friendly subset instead of the whole open-set dataset to enhance the model's capability of ID classification. Moreover, to reduce the computational expense, we also propose two practical variants of WiseOpen by adopting low-frequency update and loss-based selection respectively. Extensive experiments demonstrate the effectiveness of WiseOpen in comparison with the state-of-the-art.

A Novel Robust Metric Distance Optimization-Driven Manifold Learning Framework for Semi-Supervised Pattern Classification

In this work, we address the problem of improving the classification performance of machine learning models, especially in the presence of noisy and outlier data. To this end, we first innovatively design a generalized adaptive robust loss function called Vθ(x). Intuitively, Vθ(x) can improve the robustness of the model by selecting different robust loss functions for different learning tasks during the learning process via the adaptive parameter θ. Compared with other robust loss functions, Vθ(x) has some desirable salient properties, such as symmetry, boundedness, robustness, nonconvexity, and adaptivity, making it suitable for a wide range of machine learning applications. Secondly, a new robust semi-supervised learning framework for pattern classification is proposed. In this learning framework, the proposed robust loss function Vθ(x) and capped L2,p-norm robust distance metric are introduced to improve the robustness and generalization performance of the model, especially when the outliers are far from the normal data distributions. Simultaneously, based on this learning framework, the Welsch manifold robust twin bounded support vector machine (WMRTBSVM) and its least-squares version are developed. Finally, two effective iterative optimization algorithms are designed, their convergence is proved, and their complexity is calculated. Experimental results on several datasets with different noise settings and different evaluation criteria show that our methods have better classification performance and robustness. With the Cancer dataset, when there is no noise, the classification accuracy of our proposed methods is 94.17% and 95.62%, respectively. When the Gaussian noise is 50%, the classification accuracy of our proposed methods is 91.76% and 90.59%, respectively, demonstrating that our method has satisfactory classification performance and robustness.

Robust Teacher: Self-correcting pseudo-label-guided semi-supervised learning for object detection

Semi-supervised object detection methods have become increasingly popular in computer vision owing to their success in reducing data labeling costs. However, low-quality pseudo-labels exhibit difficulties in correction and severely limit the performance of existing state-of-the-art models. To address these problems, based on the contemporary teacher–student dual framework, we develop a novel self-correcting pseudo-label module to generate more reliable predictions for unlabeled data. Simultaneously, to mitigate the inherent class bias in pseudo-labels, we integrate multi-label classification results for measuring the re-balanced focal loss to enable class-agnostic, robust semi-supervised learning. Furthermore, pseudo-label-guided copy-paste is proposed as a simple yet efficient data augmentation technique to enhance instance representation learning within diverse complex scenes. The above key designs constitute our proposed approach, which we call Robust Teacher. Extensive experiments on PASCAL VOC and MS COCO demonstrate that Robust Teacher achieves competitive results and outperforms state-of-the-art models by a large margin. Our comprehensive ablation studies further verify the effectiveness of the key components. The code will be publicly available at https://github.com/Complicateddd/RobustT.

Adversarial smoothing tri-regression for robust semi-supervised industrial soft sensor

In industrial processes, soft sensor techniques are used to predict the hard-to-measure quality variables under the classic supervised learning paradigm. However, the data challenges, i.e., the widespread noises and inadequate labeled samples, usually make the data-driven sensors weak for application. In this paper, a simple but effective regularization method termed as adversarial smoothing regularization (ASR) is proposed, which measures the local smoothness of prediction around each input sample. By minimizing the divergence between the predictions for noisy and clean inputs, the proposed ASR regulates the learning model to be robust against local perturbation in a semi-supervised manner. Theoretical analysis explains the smoothing capability of ASR, and shows that it is helpful for the improvement of generalization performance. We also design a tri-regression framework to further use the information of unlabeled samples with pseudo labels and present the adversarial smoothing tri-regression (ASTR) model for soft sensor. Based on two industrial processes, comprehensive soft sensor experiments and noise tests are performed to show the robust semi-supervised learning capability of the proposed ASR and ASTR.

Anomaly and Novelty detection for robust semi-supervised learning

Three important issues are often encountered in Supervised and Semi-Supervised Classification: class-memberships are unreliable for some training units (label noise), a proportion of observations might depart from the main structure of the data (outliers) and new groups in the test set may have not been encountered earlier in the learning phase (unobserved classes). The present work introduces a robust and adaptive Discriminant Analysis rule, capable of handling situations in which one or more of the afore-mentioned problems occur. Two EM-based classifiers are proposed: the first one that jointly exploits the training and test sets (transductive approach), and the second one that expands the parameter estimate using the test set, to complete the group structure learned from the training set (inductive approach). Experiments on synthetic and real data, artificially adulterated, are provided to underline the benefits of the proposed method.

Distributionally Robust Semi-Supervised Learning for People-Centric Sensing

Semi-supervised learning is crucial for alleviating labelling burdens in people-centric sensing. However, humangenerated data inherently suffer from distribution shift in semi-supervised learning due to the diverse biological conditions and behavior patterns of humans. To address this problem, we propose a generic distributionally robust model for semi-supervised learning on distributionally shifted data. Considering both the discrepancy and the consistency between the labeled data and the unlabeled data, we learn the latent features that reduce person-specific discrepancy and preserve task-specific consistency. We evaluate our model in a variety of people-centric recognition tasks on real-world datasets, including intention recognition, activity recognition, muscular movement recognition and gesture recognition. The experiment results demonstrate that the proposed model outperforms the state-of-the-art methods.

Robust semi-supervised extreme learning machine

The existence of outliers among labeled data is a major challenge for semi-supervised learning. An effective method to handle this problem is to employ the non-convex loss functions, which give constant penalties to outliers to avoid their negative influences. Along this line, in this paper, by adopting the non-convex squared loss function, we propose a novel robust semi-supervised learning algorithm to overcome the limitation of the classical semi-supervised extreme learning machine (SS-ELM) that it is sensitivity to outliers, termed as robust SS-ELM, or RSS-ELM for short. After expressing the non-convex squared loss function by a difference of two convex ones, RSS-ELM is effectively solved with the help of the concave-convex procedure (CCCP) approach. For the specific implementation, RSS-ELM iteratively builds the output function by solving a sequence of linear systems at each iteration. Moreover, we analyze the computational complexity of RSS-ELM, and prove its convergence and robustness from a theoretical point of view. The proposed RSS-ELM includes the conventional ELM and SS-ELM as its special cases. Extensive experiments conducted across multiple image datasets and benchmark datasets validate that RSS-ELM not only inherits the advantages of semi-supervised learning, but also enjoys the merit of robustness.

Large-Scale Robust Semisupervised Classification.

Semisupervised learning aims to leverage both labeled and unlabeled data to improve performance, where most of them are graph-based methods. However, the graph-based semisupervised methods are not capable for large-scale data since the computational consumption on the construction of graph Laplacian matrix is huge. On the other hand, the substantial unlabeled data in training stage of semisupervised learning could cause large uncertainties and potential threats. Therefore, it is crucial to enhance the robustness of semisupervised classification. In this paper, a novel large-scale robust semisupervised learning method is proposed in the framework of capped l2,p -norm. This strategy is superior not only in computational cost because it makes the graph Laplacian matrix unnecessary, but also in robustness to outliers since the capped l2,p -norm used for loss measurement. An efficient optimization algorithm is exploited to solve the nonconvex and nonsmooth challenging problem. The complexity of the proposed algorithm is analyzed and discussed in theory detailedly. Finally, extensive experiments are conducted over six benchmark data sets to demonstrate the effectiveness and superiority of the proposed method.

Robust Semi-Supervised Learning through Label Aggregation

Semi-supervised learning is proposed to exploit both labeled and unlabeled data. However, as the scale of data in real world applications increases significantly, conventional semi-supervised algorithms usually lead to massive computational cost and cannot be applied to large scale datasets. In addition, label noise is usually present in the practical applications due to human annotation, which very likely results in remarkable degeneration of performance in semi-supervised methods. To address these two challenges, in this paper, we propose an efficient RObust Semi-Supervised Ensemble Learning (ROSSEL) method, which generates pseudo-labels for unlabeled data using a set of weak annotators, and combines them to approximate the ground-truth labels to assist semi-supervised learning. We formulate the weighted combination process as a multiple label kernel learning (MLKL) problem which can be solved efficiently. Compared with other semi-supervised learning algorithms, the proposed method has linear time complexity. Extensive experiments on five benchmark datasets demonstrate the superior effectiveness, efficiency and robustness of the proposed algorithm.

A robust semi-supervised learning approach via mixture of label information

Due to the fact that limited amounts of labeled data are normally available in real-world, semi-supervised learning has become a popular option, where we expect to use unlabeled data information to improve the learning performance. However, how to use such unlabeled information to make the predicted labels more reliable remains to be a key for any successful learning. In this paper, we propose a semi-supervised learning framework via combination of semi-supervised clustering and semi-supervised classification. In our approach, the predicted labels are selected by both the constrained k-means and safe semi-supervised SVM (S4VMs) to improve the reliability of the predicted labels. Extensive evaluations on collection of benchmarks and real-world action recognition datasets show that the proposed technique outperforms the others.

Robust Semi-Supervised Learning Algorithm Based on Maximum Correntropy Criterion

PDF HTML阅读 XML下载 导出引用 引用提醒 基于最大相关熵准则的鲁棒半监督学习算法 DOI: 10.3724/SP.J.1001.2012.03977 作者: 作者单位: 作者简介: 通讯作者: 中图分类号: 基金项目: 国家自然科学基金(60873054, 50909012); 国家教育部高等学校博士学科点专项科研基金(20100041120009) Robust Semi-Supervised Learning Algorithm Based on Maximum Correntropy Criterion Author: Affiliation: Fund Project: 摘要 | 图/表 | 访问统计 | 参考文献 | 相似文献 | 引证文献 | 资源附件 | 文章评论 摘要:分析了噪声对半监督学习Gaussian-Laplacian 正则化(Gaussian-Laplacian regularized,简称GLR)框架的影响,针对最小二乘准则对噪声敏感的特点,结合信息论的最大相关熵准则(maximum correntropy criterion,简称MCC), 提出了一种基于最大相关熵准则的鲁棒半监督学习算法(简称GLR-MCC),并证明了算法的收敛性.半二次优化技术被用来求解相关熵目标函数.在每次迭代中,复杂的信息论优化问题被简化为标准的半监督学习问题.典型机器学习数据集上的仿真实验结果表明,在标签噪声和遮挡噪声的情况下,该算法能够有效地提高半监督学习算法性能. Abstract:This paper analyzes the problem of sensitivity to noise in the mean square criterion of Gaussian- Laplacian regularized (GLR) algorithm. A robust semi-supervised learning algorithm based on maximum correntropy criterion (MCC), called GLR-MCC, is proposed to improve the robustness of GLR along with its convergence analysis. The half quadratic optimization technique is used to simplify the correntropy optimization problem to a standard semi-supervised problem in each iteration. Experimental results on typical machine learning data sets show that the proposed GLR-MCC can effectively improve the robustness of mislabeling noise and occlusion as compared with related semi-supervised learning algorithms. 参考文献 相似文献 引证文献