Positive Unlabeled Learning Research Articles

Wind Turbine Blade Early Fault Detection With Faulty Label Unknown and Labeling Bias

In practical industrial applications, the need for a large number of accurately labeled training samples is a significant challenge for fault detection tasks. However, labeling all training samples is expensive and prone to labeling errors, especially for early fault detection of wind turbine blades. This paper proposes a labeling bias (LB) hypothesis. Assuming the labeler only needs to label parts of normal samples that are easy to judge, we design a probability ratio least-squares importance fitting (PRL-SIF) method based on variable homogeneity. Unlike other state-of-the-art positive unlabeled (PU) learning methods, PRL-SIF does not require knowledge of the class priors to achieve training. Furthermore, to better handle the multi-dimensional time-series data of wind turbines, we provide a data preprocessing method based on functional analysis to achieve time series feature extraction and dimensionality reduction. The effectiveness and robustness of the proposed method are verified on 23 real-world wind turbine datasets. Experimental results show that the proposed method can achieve nearly 90% accuracy while only labeling 20% of normal samples.

GradPU: Positive-Unlabeled Learning via Gradient Penalty and Positive Upweighting

Positive-unlabeled learning is an essential problem in many real-world applications with only labeled positive and unlabeled data, especially when the negative samples are difficult to identify. Most existing positive-unlabeled learning methods will inevitably overfit the positive class to some extent due to the existence of unidentified positive samples. This paper first analyzes the overfitting problem and proposes to bound the generalization errors via Wasserstein distances. Based on that, we develop a simple yet effective positive-unlabeled learning method, GradPU, which consists of two key ingredients: A gradient-based regularizer that penalizes the gradient norms in the interpolated data region, which improves the generalization of positive class; An unnormalized upweighting mechanism that assigns larger weights to those positive samples that are hard, not-well-fitted and less frequently labeled. It enforces the training error of each positive sample to be small and increases the robustness to the labeling bias. We evaluate our proposed GradPU on three datasets: MNIST, FashionMNIST, and CIFAR10. The results demonstrate that GradPU achieves state-of-the-art performance on both unbiased and biased positive labeling scenarios.

Positive Distribution Pollution: Rethinking Positive Unlabeled Learning from a Unified Perspective

Positive Unlabeled (PU) learning, which has a wide range of applications, is becoming increasingly prevalent. However, it suffers from problems such as data imbalance, selection bias, and prior agnostic in real scenarios. Existing studies focus on addressing part of these problems, which fail to provide a unified perspective to understand these problems. In this paper, we first rethink these problems by analyzing a typical PU scenario and come up with an insightful point of view that all these problems are inherently connected to one problem, i.e., positive distribution pollution, which refers to the inaccuracy in estimating positive data distribution under very little labeled data. Then, inspired by this insight, we devise a variational model named CoVPU, which addresses all three problems in a unified perspective by targeting the positive distribution pollution problem. CoVPU not only accurately separates the positive data from the unlabeled data based on discrete normalizing flows, but also effectively approximates the positive distribution based on our derived unbiased rebalanced risk estimator and supervises the approximation based on a novel prior-free variational loss. Rigorous theoretical analysis proves the convergence of CoVPU to an optimal Bayesian classifier. Extensive experiments demonstrate the superiority of CoVPU over the state-of-the-art PU learning methods under these problems.

Bayesian analysis for imbalanced positive-unlabelled diagnosis codes in electronic health records

With the increasing availability of electronic health records (EHR), significant progress has been made on developing predictive inference and algorithms by health data analysts and researchers. However, the EHR data are notoriously noisy due to missing and inaccurate inputs despite the information is abundant. One serious problem is that only a small portion of patients in the database has confirmatory diagnoses while many other patients remain undiagnosed because they did not comply with the recommended examinations. The phenomenon leads to a so-called positive-unlabelled situation and the labels are extremely imbalanced. In this paper, we propose a model-based approach to classify the unlabelled patients by using a Bayesian finite mixture model. We also discuss the label switching issue for the imbalanced data and propose a consensus Monte Carlo approach to address the imbalance issue and improve computational efficiency simultaneously. Simulation studies show that our proposed model-based approach outperforms existing positive-unlabelled learning algorithms. The proposed method is applied on the Cerner EHR for detecting diabetic retinopathy (DR) patients using laboratory measurements. With only 3% confirmatory diagnoses in the EHR database, we estimate the actual DR prevalence to be 25% which coincides with reported findings in the medical literature.

Neural Clustering and Ranking Approach for Gas-Theft Suspect Detection

Some boiler room users steal natural gas by refitting equipment without permission in winter, resulting in gas safety hazards and social problems. Instead of random manual on-site inspection, it is crucial to discover gas-theft suspects timely and automatically by analyzing the gas consumption data. Unfortunately, gas-theft behaviors are complex and various, while the caught gas thefts by gas companies are limited. In this paper, we propose a neural clustering and ranking approach to detect gas theft suspects under the positive-unlabeled learning framework. Our approach contains two modules: joint clustering for normal user identification and triplet ranking for suspicious user detection. The former module considers the regular behaviors to distinguish between normal and unstable users by integrating representation learning and clustering. Then, considering the identified normal samples and the labeled gas thefts, the later module excavates the behavior correlations to discover suspects among unstable users through triplet relation ranking. Thus, normal user identification and suspicious user detection are seamlessly connected to overcome the label scarcity problem. We conduct extensive experiments on three real-world datasets, and the results demonstrate the advantages of our approach over various baselines.

Anomaly Detection in Automatic Meter Intelligence System Using Positive Unlabeled Learning and Multiple Symbolic Aggregate Approximation.

With the development of automatic electrical devices in smart grids, the data generated by time and transmitted are vast and thus impossible to control consumption by humans. The problem of abnormal detection in power consumption is crucial in monitoring and controlling smart grids. This article proposes the detection of electrical meter anomalies by detecting abnormal patterns and learning unlabeled data. Furthermore, a framework for big data and machine learning-based anomaly detection framework are introduced. The experimental results show that the time series anomaly detection for electric meters has better results in accuracy and time than the expert alternatives.

Positive-unlabeled learning for binary and multi-class cell detection in histopathology images with incomplete annotations

Cell detection in histopathology images is of great interest to clinical practice and research, and convolutional neural networks (CNNs) have achieved remarkable cell detection results. Typically, to train CNN-based cell detection models, every positive instance in the training images needs to be annotated, and instances that are not labeled as positive are considered negative samples. However, manual cell annotation is complicated due to the large number and diversity of cells, and it can be difficult to ensure the annotation of every positive instance. In many cases, only incomplete annotations are available, where some of the positive instances are annotated and the others are not, and the classification loss term for negative samples in typical network training becomes incorrect. In this work, to address this problem of incomplete annotations, we propose to reformulate the training of the detection network as a positive-unlabeled learning problem. Since the instances in unannotated regions can be either positive or negative, they have unknown labels. Using the samples with unknown labels and the positively labeled samples, we first derive an approximation of the classification loss term corresponding to negative samples for binary cell detection, and based on this approximation we further extend the proposed framework to multi-class cell detection. For evaluation, experiments were performed on four publicly available datasets. The experimental results show that our method improves the performance of cell detection in histopathology images given incomplete annotations for network training.

A hybrid positive unlabeled learning framework for uncovering scaffolds across human proteome by measuring the propensity to drive phase separation.

Scaffold proteins drive liquid-liquid phase separation (LLPS) to form biomolecular condensates and organize various biochemical reactions in cells. Dysregulation of scaffolds can lead to aberrant condensate assembly and various complex diseases. However, bioinformatics predictors dedicated to scaffolds are still lacking and their development suffers from an extreme imbalance between limited experimentally identified scaffolds and unlabeled candidates. Here, using the joint distribution of hybrid multimodal features, we implemented a positive unlabeled (PU) learning-based framework named PULPS that combined ProbTagging and penalty logistic regression (PLR) to profile the propensity of scaffolds. PULPS achieved the best AUC of 0.8353 and showed an area under the lift curve (AUL) of 0.8339 as an estimation of true performance. Upon reviewing recent experimentally verified scaffolds, we performed a partial recovery with 2.85% increase in AUL from 0.8339 to 0.8577. In comparison, PULPS showed a 45.7% improvement in AUL compared with PLR, whereas 8.2% superiority over other existing tools. Our study first proved that PU learning is more suitable for scaffold prediction and demonstrated the widespread existence of phase separation states. This profile also uncovered potential scaffolds that co-drive LLPS in the human proteome and generated candidates for further experiments. PULPS is free for academic research at http://pulps.zbiolab.cn.

NIAPU: network-informed adaptive positive-unlabeled learning for disease gene identification.

Gene-disease associations are fundamental for understanding disease etiology and developing effective interventions and treatments. Identifying genes not yet associated with a disease due to a lack of studies is a challenging task in which prioritization based on prior knowledge is an important element. The computational search for new candidate disease genes may be eased by positive-unlabeled learning, the machine learning (ML) setting in which only a subset of instances are labeled as positive while the rest of the dataset is unlabeled. In this work, we propose a set of effective network-based features to be used in a novel Markov diffusion-based multi-class labeling strategy for putative disease gene discovery. The performances of the new labeling algorithm and the effectiveness of the proposed features have been tested on 10 different disease datasets using three ML algorithms. The new features have been compared against classical topological and functional/ontological features and a set of network- and biological-derived features already used in gene discovery tasks. The predictive power of the integrated methodology in searching for new disease genes has been found to be competitive against state-of-the-art algorithms. The source code of NIAPU can be accessed at https://github.com/AndMastro/NIAPU. The source data used in this study are available online on the respective websites. Supplementary data are available at Bioinformatics online.

A novel observation points‐based positive‐unlabeled learning algorithm

AbstractIn this study, an observation points‐based positive‐unlabeled learning algorithm (hence called OP‐PUL) is proposed to deal with positive‐unlabeled learning (PUL) tasks by judiciously assigning highly credible labels to unlabeled samples. The proposed OP‐PUL algorithm has three components. First, an observation point classifier ensemble (OPCE) algorithm is constructed to divide unlabeled samples into two categories, which are temporary positive and permanent negative samples. Second, a temporary OPC (TOPC) is trained based on the combination of original positive samples and permanent negative samples and then the permanent positive samples that are correctly classified with TOPC are retained from the temporary positive samples. Third, a permanent OPC (POPC) is finally trained based on the combination of original positive samples, permanent positive samples and permanent negative samples. An exhaustive experimental evaluation is conducted to validate the feasibility, rationality and effectiveness of the OP‐PUL algorithm, using 30 benchmark PU data sets. Results show that (1) the OP‐PUL algorithm is stable and robust as unlabeled samples and positive samples are increased in unlabeled data sets and (2) the permanent positive samples have a consistent probability distribution with the original positive samples. Moreover, a statistical analysis reveals that POPC in the OP‐PUL algorithm can yield better PUL performances on the 30 data sets in comparison with four well‐known PUL algorithms. This demonstrates that OP‐PUL is a viable algorithm to deal with PUL tasks.

Mapping rapeseed in China during 2017-2021 using Sentinel data: an automated approach integrating rule-based sample generation and a one-class classifier (RSG-OC)

ABSTRACT Rapeseed mapping is important for national food security and government regulation of land use. Various methods, including empirical index-based and machine learning-based methods, have been developed to identify rapeseed using remote sensing. Empirical index-based methods commonly employed empirically designed indices to enhance rapeseed’s bright yellow spectral feature during the flowering period, which is straightforward to implement. Unfortunately, the heavy cloud cover in the flowering period of China would lead to serious omission errors; and the required flowering period varies spatially and yearly, which often cannot be acquired accurately. Machine learning-based methods mitigate the reliance on clear observations during the flowering period by inputting all-season imagery to adaptively learn features. However, it is difficult to collect sufficient samples across all of China considering the large intraclass variation in both land cover types of rapeseed and non-rapeseed. This study proposed an automated rapeseed mapping approach integrating rule-based sample generation and a one-class classifier (RSG-OC) to overcome the shortcomings of the two types of methods. First, a set of sample selection rules based on empirical indices of rapeseed were developed to automatically generate samples in cloud-free pixels during the predicted flowering period throughout China. Second, all available features composited based on the rapeseed phenological calendar were used for classification to eliminate the phenology differences in different regions. Third, a specific sample augmentation that removes the observation in the flowering period was employed to improve the generalization to the pixels without cloud-free observation in the flowering period. Finally, to avoid the need for diverse samples of nonrapeseed classes, a typical one-class classifier, positive unlabeled learning implemented by random forest (PUL-RF) trained by the generated samples, was applied to map rapeseed. With the proposed method, China rapeseed was mapped at 20 m resolution during 2017–2021 based on the Google Earth Engine (GEE). Validation on six typical rapeseed planting areas demonstrates that RSG-OC achieves an average accuracy of 94.90%. In comparison, the average accuracy of the other methods ranged from 83.33% to 88.25%, all of which were poorer than the proposed method. Additional experiments show that the performance of RSG-OC was not sensitive to cloud contamination, inaccurate predicted flowering time and the threshold of sample selection rule. These results indicate that the rapeseed maps produced in China are overall reliable and that the proposed method is an effective and robust method for annual rapeseed mapping across China.

Foundations for improved vaccine correlate of risk analysis using positive-unlabeled learning

ABSTRACT Insights into mechanisms of protection afforded by vaccine efficacy field trials can be complicated by both low rates of exposure and protection. However, these barriers do not preclude the discovery of correlates of reduced risk (CoR) of infection, which are a critical first step in defining correlates of protection (CoP). Given the significant investment in large-scale human vaccine efficacy trials and immunogenicity data collected to support CoR discovery, novel approaches for analyzing efficacy trials to optimally support discovery of CoP are critically needed. By simulating immunological data and evaluating several machine learning approaches, this study lays the groundwork for deploying Positive/Unlabeled (P/U) learning methods, which are designed to differentiate between two groups in cases where only one group has a definitive label and the other remains ambiguous. This description applies to case–control analysis designs for field trials of vaccine efficacy: infected subjects, or cases, are by definition unprotected, whereas uninfected subjects, or controls, may have been either protected or unprotected but simply never exposed. Here, we investigate the value of applying P/U learning to classify study subjects using model immunogenicity data based on predicted protection status in order to support new insights into mechanisms of vaccine-mediated protection from infection. We demonstrate that P/U learning methods can reliably infer protection status, supporting the discovery of simulated CoP that are not observed in conventional comparisons of infection status cases and controls, and we propose next steps necessary for the practical deployment of this novel approach to correlate discovery.

Prediction of drug side effects with transductive matrix co-completion.

Side effects of drugs could cause severe health problems and the failure of drug development. Drug-target interactions are the basis for side effect production and are important for side effect prediction. However, the information on the known targets of drugs is incomplete. Furthermore, there could be also some missing data in the existing side effect profile of drugs. As a result, new methods are needed to deal with the missing features and missing labels in the problem of side effect prediction. We propose a novel computational method based on transductive matrix co-completion and leverage the low-rank structure in the side effects and drug-target data. Positive-unlabelled learning is incorporated into the model to handle the impact of unobserved data. We also introduce graph regularization to integrate the drug chemical information for side effect prediction. We collect the data on side effects, drug targets, drug-associated proteins and drug chemical structures to train our model and test its performance for side effect prediction. The experiment results show that our method outperforms several other state-of-the-art methods under different scenarios. The case study and additional analysis illustrate that the proposed method could not only predict the side effects of drugs but also could infer the missing targets of drugs. The data and the code for the proposed method are available at https://github.com/LiangXujun/GTMCC. Supplementary data are available at Bioinformatics online.

Improving Non-Negative Positive-Unlabeled Learning for News Headline Classification

With the development of Internet technology, network platforms have gradually become a tool for people to obtain hot news. How to filter out the current hot news from a large number of news collections and push them to users has important application value. In supervised learning scenarios, each piece of news needs to be labeled manually, which takes a lot of time and effort. From the perspective of semi-supervised learning, on the basis of the non-negative Positive-Unlabeled (nnPU) learning, this paper proposes a novel algorithm, called ’Enhanced nnPU with Focal Loss’ (FLPU), for news headline classification, which uses the Focal Loss to replace the way the classical nnPU calculates the empirical risk of positive and negative samples. Then, by introducing the Virtual Adversarial Training (VAT) of the Adversarial training for large neural LangUage Models (ALUM) into FLPU, another (and better) algorithm, called ’FLPU+ALUM’, is proposed for the same purpose, aiming to label only a small number of positive samples. The superiority of both algorithms to state-of-the-art PU algorithms considered is demonstrated by means of experiments, conducted on two datasets for performance comparison. Moreover, through another set of experiments, it is shown that, if enriched by the proposed algorithms, the RoBERTa-wwm-ext model can achieve better classification performance than the state-of-the-art binary classification models included in the comparison. In addition, a ’Ratio Batch’ method is elaborated and proposed as more stable for use in scenarios involving only a small number of labeled positive samples, which is also experimentally demonstrated.

Molecular screening for solid–solid phase transitions by machine learning

The solid–solid phase transition in molecular solids can be predicted by positive-unlabeled learning using molecular descriptors.

Pi‐Score: An Estimation Strategy of the Class Prior in Positive‐Unlabeled Learning for Electrical Insulator Defect Detection with Incomplete Annotations

Insulators in high‐voltage power systems serve as brackets for overhead lines and prevent these wires from becoming grounded. Due to long‐term exposure to a harsh environment, it is indispensable to apply periodic inspection for defective insulators, facilitating the timely overhaul of insulators. In the field of object detection, convolutional neural networks (CNNs) have been introduced and have achieved good performance. Therefore, various CNN‐based detectors are applied to the insulator detection task. Because collecting ideal annotations is time‐consuming and labor‐intensive, research with imperfect annotations has drawn more attention. However, fewer insulator defect detection approaches account for this imperfect annotation problem. This paper focuses on a novel and challenging insulator defect detection scenario: a part of the insulators in datasets is unannotated and viewed as the background. We introduce positive‐unlabeled (PU) learning to solve the problem of incomplete annotation for insulator defect detection. To further improve PU learning, a Pi‐Score algorithm is proposed to estimate class prior, a crucial parameter in PU loss. Our designed framework is built on Faster R‐CNN and incorporates the improved PU learning in the region proposal network. Experimental results on the Insulator Defect Image Dataset (IDID) demonstrate that the proposed framework achieved an average precision (AP) metric that is approximately 1%–2% higher than the positive–negative mainstream detectors with varying degrees of missing annotation. Meanwhile, the proposed framework obtained 0.33 and 0.47 higher AP metrics than the mainstream PU detectors with complete and half of IDID’s annotations.

Improving Drug Combination Repositioning using Positive Unlabelled Learning and Ensemble Learning

Drug repositioning is a cost-effective and time-effective concept that enables the use of existing drugs/drug combinations for therapeutic effects. The number of drug combinations used for therapeutic effects is smaller than all possible drug combinations in the present drug databases. These databases consist of a smaller set of labelled positives and a majority of unlabelled drug combinations. Therefore, there is a need for determining both reliable positive and reliable negative samples to develop binary classification models. Since, we only have labelled positives, the unlabelled data has to be separated into positives and negatives by a reliable technique. This study proposes and demonstrates the significance of using Positive Unlabelled Learning, for determining reliable positive and negative drug combinations for drug repositioning. In the proposed approach, the dataset with known positives and unlabelled samples was clustered by a Deep Learning based Self Organizing Map. Then, an ensemble learning methodology was followed by employing three classification models. The proposed PUL model was compared with the frequently used approach that randomly selects negative drug pairs from unlabelled samples. A significant improvement of 19.15%, 20.56% and 20.23% in the Precision, Recall and F-measure, respectively, was observed for the proposed PUL-based ensemble learning approach. Moreover, 128 drug repositioning candidates were predicted by the proposed methodology. Further, we found literature-based evidence to support five drug combinations that may be able to be repositioned. These discoveries show our proposed PUL approach as a promising strategy that is applicable in drug combination prediction for repositioning.

Small molecule drug and biotech drug interaction prediction based on multi-modal representation learning

BackgroundDrug–drug interactions (DDIs) occur when two or more drugs are taken simultaneously or successively. Early detection of adverse drug interactions can be essential in preventing medical errors and reducing healthcare costs. Many computational methods already predict interactions between small molecule drugs (SMDs). As the number of biotechnology drugs (BioDs) increases, so makes the threat of interactions between SMDs and BioDs. However, few computational methods are available to predict their interactions.ResultsConsidering the structural specificity and relational complexity of SMDs and BioDs, a novel multi-modal representation learning method called Multi-SBI is proposed to predict their interactions. First, multi-modal features are used to adequately represent the heterogeneous structure and complex relationships of SMDs and BioDs. Second, an undersampling method based on Positive-unlabeled learning (PU-sampling) is introduced to obtain negative samples with high confidence from the unlabeled data set. Finally, both learned representations of SMD and BioD are fed into DNN classifiers to predict their interaction events. In addition, we also conduct a retrospective analysis.ConclusionsOur proposed multi-modal representation learning method can extract drug features more comprehensively in heterogeneous drugs. In addition, PU-sampling can effectively reduce the noise in the sampling procedure. Our proposed method significantly outperforms other state-of-the-art drug interaction prediction methods. In a retrospective analysis of DrugBank 5.1.0, 14 out of the 20 predictions with the highest confidence were validated in the latest version of DrugBank 5.1.8, demonstrating that Multi-SBI is a valuable tool for predicting new drug interactions through effectively extracting and learning heterogeneous drug features.

Learning from undercoded clinical records for automated International Classification of Diseases (ICD) coding.

To develop an unbiased objective for learning automatic coding algorithms from clinical records annotated with only partial relevant International Classification of Diseases codes, as annotation noise in undercoded clinical records used as training data can mislead the learning process of deep neural networks. We use Medical Information Mart for Intensive Care III as our dataset. We employ positive-unlabeled learning to achieve unbiased loss estimation, which is free of misleading training signal. We then utilize reweighting mechanism to compensate for the imbalance between positive and negative samples. To further close the performance gap caused by poor quality annotation, we integrate the supervision provided by the automatic annotation tool Medical Concept Annotation Toolkit which can ease the heavy burden of manual validation. Our benchmarking results show that positive-unlabeled learning with reweighting outperforms competitive baseline methods over a range of missing label ratios. Integrating supervision provided by annotation tool further boosted the performance. Considering the annotation noise and severe imbalance, unbiased loss estimation and reweighting mechanism are both important for learning from undercoded clinical records. Unbiased loss requires the estimation of false negative ratios and estimation through trained models is practical and competitive. The combination of positive-unlabeled learning with reweighting and supervision provided by the annotation tool is a promising solution to learn from undercoded clinical records.

Detecting Arbitrage on Ethereum Through Feature Fusion and Positive-Unlabeled Learning

Due to the lack of supervision in the decentralized exchanges (DEXs), arbitrageurs can utilize information and take advantage of price gap to make profits over such platforms such as Ethereum blockchain. DEX arbitrage poses possibilities and opportunities for defrauding and can seriously impair the operation of the Ethereum ecosystem. It motivates this work to explore and characterize the unique features of arbitrage which differ from other frauds such as money laundering and Ponzi games for better detection. This work makes the first attempt for detecting arbitrage on Ethereum through feature fusion and positive-unlabeled learning (PU learning). We first conduct an in-depth analysis and exploit two-fold arbitrage features by fusion including: 1) statistical features that explicitly represent the node activity levels according to expert knowledge; and 2) structural features that implicitly encode the transactions information by graph machine learning. We then apply PU learning to generate negative instances for compensating the imbalanced arbitrage datasets. We evaluate our proposed method through extensive experiments over a real-world dataset and demonstrate that it can achieve 90% accuracy in detecting arbitrage activities on Ethereum.