Positive And Unlabeled Learning Research Articles

Probabilistic Reachability Prediction of Unbounded Petri Nets: A Machine Learning Method

Unbounded Petri nets (UPNs) can describe and analyze discrete event systems with infinite states (DESIS). Due to the infinite state space and the combination explosion problem, the reachability analysis of UPNs is an NP-Hard problem. The existing reachability analysis methods cannot achieve an accurate result at reasonable costs (computational time and space) due to the finite reachability tree with <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$\omega$</tex-math> </inline-formula> -numbers. Based on the idea of approximating infinite space with finite states, given some limited reachable markings of a UPN, we propose a method that can quantitatively solve the UPN’s reachability problem with machine learning. Firstly, we define the probabilistic reachability of markings and transform the UPN’s reachability problem into the prediction problem of markings. The proposed method based on positive and unlabeled learning (PUL) and bagging trains a classifier to predict the probabilistic reachability of unknown markings. Finally, to predict the markings outside the positive sample set and unlabeled sample set, an iterative strategy is designed to update the classifier. Based on seven general UPNs, the results of the experiments show that the proposed method has a good performance in the accuracy and time consumption for the UPN’s reachability problem. <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">Note to Practitioners</i> —In discrete event systems, the reachability problem mainly studies reachable states of the system and the relationship between states, which is the basis of the system’s states, behaviors, attributes and performance analysis. For discrete event systems with infinite states, it is hard to analyze the reachable relationship between states within a finite time due to the infinite state space and the combination explosion problem. The main motivation of the paper is to propose a method that can predict the reachable relationship between the states with a probability value within a finite time. By machine learning algorithms, the method learns the feature information of the known reachable states. The reachability of unknown states in the infinite state space can be predicted approximately. The proposed approximation method can be applied to analyze the reachability properties of general discrete event systems with infinite states, such as checking whether a fault occurs in operating systems, whether a message is delivered in communication and so on.

A Quantum-Inspired Direct Learning Strategy for Positive and Unlabeled Data

Learning from only positive and unlabeled (PU) data has broad applications in fields such as web data mining, product recommendations and medical diagnosis, which aims to train a binary classifier in the absence of negative labeled data. However, due to the lack of negative label information, prevailing PU learning methods usually rely on prior knowledge of unknown class distributions heavily. In fact, without additional constraints imposed by the prior knowledge, a direct learning strategy to coordinate the underlying clustering information in unlabeled data with the label information from positive training data is often considered challenging. To tackle this challenge, we propose a direct PU learning strategy using quantum formalization. By employing neural networks as backends, the samples are mapped into two-qubit composite systems, which should be understood here as mathematical entities encapsulating various classical distributions of two classical bits. Subsequently, the two qubits within the systems are trained to be as independent as possible from each other, capturing patterns of different classes. At the same time, their measurement results serving as the model outputs are encouraged to be maximally dissimilar. These characteristics enable effective training of classifiers on PU data. After formulating an appropriate discriminant rule, we introduce a quantum-inspired PU method named qPU using the direct learning strategy. This method not only has the potential to alleviate parameter sensitivity issues caused by prior estimation in other methods but is also straightforward to implement. Finally, experiments conducted on 13 classical datasets validate the effectiveness of qPU.

Spatiotemporal Pattern of Invasive Pedicularis in the Bayinbuluke Land, China, during 2019–2021: An Analysis Based on PlanetScope and Sentinel-2 Data

The accurate identification and monitoring of invasive plants are of great significance to sustainable ecological development. The invasive Pedicularis poses a severe threat to native biodiversity, ecological security, socioeconomic development, and human health in the Bayinbuluke Grassland, China. It is imperative and useful to obtain a precise distribution map of Pedicularis for controlling its spread. This study used the positive and unlabeled learning (PUL) method to extract Pedicularis from the Bayinbuluke Grassland based on multi-period Sentinel-2 and PlanetScope remote sensing images. A change rate model for a single land cover type and a dynamic transfer matrix were constructed under GIS to reflect the spatiotemporal distribution of Pedicularis. The results reveal that (1) the PUL method accurately identifies Pedicularis in satellite images, achieving F1-scores above 0.70 and up to 0.94 across all three datasets: PlanetScope data (seven features), Sentinel-2 data (seven features), and Sentinel-2 data (thirteen features). (2) When comparing the three datasets, the number of features is more important than the spatial resolution in terms of use in the PUL method of Pedicularis extraction. Nevertheless, when compared with PlanetScope data, Sentinel-2 data demonstrated a higher level of accuracy in predicting the distribution of Pedicularis. (3) During the 2019–2021 growing season, the distribution area of Pedicularis decreased, and the distribution was mainly concentrated in the northeast and southeast of Bayinbuluke Swan Lake. The acquired spatiotemporal pattern of invasive Pedicularis could potentially be used to aid in controlling Pedicularis spread or elimination, and the methods proposed in this study could be adopted by the government as a low-cost strategy to identify priority areas in which to concentrate efforts to control and continue monitoring Pedicularis invasion.

Automated Detection Method to Extract Pedicularis Based on UAV Images

Pedicularis has adverse effects on vegetation growth and ecological functions, causing serious harm to animal husbandry. In this paper, an automated detection method is proposed to extract Pedicularis and reveal the spatial distribution. Based on unmanned aerial vehicle (UAV) images, this paper adopts logistic regression, support vector machine (SVM), and random forest classifiers for multi-class classification. One-class SVM (OCSVM), isolation forest, and positive and unlabeled learning (PUL) algorithms are used for one-class classification. The results are as follows: (1) The accuracy of multi-class classifiers is better than that of one-class classifiers, but it requires all classes that occur in the image to be exhaustively assigned labels. Among the one-class classifiers that only need to label positive or positive and labeled data, the PUL has the highest F score of 0.9878. (2) PUL performs the most robustly to change features in one-class classifiers. All one-class classifiers prove that the green band is essential for extracting Pedicularis. (3) The parameters of the PUL are easy to tune, and the training time is easy to control. Therefore, PUL is a promising one-class classification method for Pedicularis extraction, which can accurately identify the distribution range of Pedicularis to promote grassland administration.

A loss matrix-based alternating optimization method for sparse PU learning

Positive and unlabeled (PU) learning is an important research topic in machine learning area, whose aim is to learn a good classifier from PU data. Due to its wide applications, a variety of PU algorithms with promising performance have been proposed. However, few of them consider PU learning from high-dimensional PU data. To fill the gap, in this work, we focus on designing the sparse PU classifier. However, it is difficult to achieve it, since the labels of U samples are uncertain. To this end, a loss matrix-based alternating optimization method, named LMAO-PU is proposed, where a two-stage alternating optimization idea is suggested to solve the difficulty in sparse PU learning. Firstly, a loss matrix is designed to measure the classification performance of the PU classifier on U samples. Then, a two-stage alternating optimization under the guide of loss matrix is developed. Specifically, the first stage is to optimize the PU model, aiming to address the challenge of labels uncertainty of U samples, where two objectives, TPR (True Positive Rate) and a suggested LRU (Loss Rate on Unlabeled), are optimized simultaneously. The second stage is to perform the sparse model optimization, where Sparsity and ErrorRate are adopted as the objectives to obtain the sparse model. The two-stage optimization procedure mentioned above is carried alternately and the sparse PU classifier with high quality is finally achieved. Experiments on 10 high-dimensional datasets demonstrate the superiority of proposed method over six state-of-the-art baselines in terms of sparsity, accuracy and area under ROC curve.

A graph-based approach for positive and unlabeled learning

Positive and Unlabeled Learning (PUL) uses unlabeled documents and a few positive documents for retrieving a set of “interest” documents from a text collection. Usually, PUL approaches are based on the vector space model. However, when dealing with semi-supervised learning for text classification or information retrieval, graph-based approaches have been proved to outperform vector space model-based approaches. So, in this article, a graph-based approach for PUL is proposed: Label Propagation for Positive and Unlabeled Learning (LP-PUL). The proposed framework consists of three steps: (i) building a similarity graph, (ii) identifying reliable negative documents, and (iii) performing label propagation to classify the remaining unlabeled documents as positive or negative. We carried out experiments to measure the impact of the different choices in each step of the proposed framework. We also demonstrated that the proposal surpasses the classification performance of other PUL (RC-SVM, PU-LP, and PE-PUC) or one-class learning (k-NN-based, k-Means-based, and Dense Autoencoder) algorithms in terms of F1. Considering the best results of any algorithm used in the experimental evaluation, PU-PUL can improve the classification performance from 2%, when using only 1 labeled document, to 28%, when 30 labeled documents are employed.

An evolutionary multi-objective approach to learn from positive and unlabeled data

Positive and unlabeled (PU) learning has attracted increasing interests in recent years. Despite that a number of PU learning algorithms have been proposed, most of them are subject to some assumptions about unlabeled sample distribution and objective functions, which makes them difficult to be adopted for real applications. To this end, in this paper, an evolutionary multi-objective approach, namely MOEA-PUL, is suggested for PU learning, whose aim is to build a PU classifier without any prior assumption for data distribution and objective functions. To be specific, the PU learning is formulated as a bi-objective optimization problem, where the true positive rate (TPR) and a new metric, termed unlabeled accuracy rate (UAR) are used as the two objectives. A multi-objective evolutionary algorithm is proposed to solve this bi-objective optimization problem, under the framework of NSGA-II, where a PU similarity based initialization strategy and an elite label based learning strategy are developed. Empirical studies on 12 datasets demonstrate the superiority of MOEA-PUL over the existing PU learning algorithms.

A positive and unlabeled learning algorithm for mineral prospectivity mapping

Application of supervised machine learning algorithms for mineral prospectivity mapping (MPM) requires positive and negative training samples. Typically, known mineral deposits are considered as positive training samples. However, the selection of negative training samples in the process of MPM is challenging. The one-class classification methods require positive and unlabeled samples or only positive samples; while without requiring negative training samples. In this study, the positive and unlabeled learning (PUL) algorithm was employed to produce a potential map for Fe polymetallic mineralization in southwestern Fujian province, China. This study first examined the sensitivity of the PUL algorithm to different training sets of labeled and unlabeled locations. The predictive results on 10 random unlabeled datasets confirm that PUL modeling with different training sets is reproducible and stable. In addition, the trained model provides a strong spatial correlation between the predictive variables and the locations of known mineral deposits. Finally, the performance of PUL1algorithm is compared to one-class support vector machine (OCSVM), artificial neural networks (ANN), and logistic regression (LR). Comparative results indicate that the PUL model can achieve a better performance in terms of both fitting-rate, prediction-rate, and AUC value compared with OCSVM, ANN and LR. The labelling efforts can be significantly reduced because the PUL algorithm requires only a small number of positive samples and utilizes unlabeled data in training.

Fake Review Detection Based on PU Learning and Behavior Density

Today, app stores offer ranking lists to help users to find quality apps that meet their needs. In order to prevent people from spreading fake reviews which can be used to defame certain apps or manipulate the ranking lists of the app store, we propose a method based on Positive and Unlabeled (PU) learning and behavior density to detect fake reviews. To identify the trusted negative samples, the classifier is trained by the Biased-SVM algorithm. Then, the preliminary screening results of the classifier are combined with user behavior density to identify fake reviews. The traditional fully supervised detection method relies on manually labeled data, the quality of which directly affects the trained classifier. Our proposed method can overcome such a deficiency, and achieve effective learning when there are only a small number of positive samples and a large number of unlabeled samples. Through experiments and case analysis, we demonstrate that our method has high detection accuracy.

PUMAD: PU Metric learning for anomaly detection

Anomaly detection task, which identifies abnormal patterns in data, has been widely applied to various domains. Most recent work on anomaly detection have focused on an accurate modeling of the normal data based on unsupervised methods. To get a satisfactory anomaly detection accuracy, they need pure normal data without abnormal data. This scenario requires many labels to get pure normal data. In many real-world scenarios, there exist abundant unlabeled data and a limited number of partially labeled anomalies. This paper proposes a novel anomaly detection method, PUMAD, which uses a Positive and Unlabeled (PU) learning approach to learn from abundant unlabeled data and a small number of partially labeled anomalies (i.e., positives). PUMAD successfully works on the anomaly detection scenario by exploiting deep metric learning with a hashing-based filtering method. Extensive experimental results on real-world benchmark datasets demonstrate that our approach based on PU learning is effective to detect anomalies. PUMAD achieves a much higher accuracy of up to 24% than state-of-the-art competitors.

Construct a biased SVM classifier based on Chebyshev distance for PU learning

In some real applications, only limited labeled positive examples and many unlabeled examples are available, but there are no negative examples. Such learning is termed as positive and unlabeled (PU) learning. PU learning algorithm has been studied extensively in recent years. However, the classical ones based on the Support Vector Machines (SVMs) are assumed that labeled positive data is independent and identically distributed (i.i.d) and the sample size is large enough. It leads to two obvious shortcomings. On the one hand, the performance is not satisfactory, especially when the number of the labeled positive examples is small. On the other hand, classification results are not optimistic when datasets are Non-i.i.d. For this reason, this paper proposes a novel SVM classifier using Chebyshev distance to measure the empirical risk and designs an efficient iterative algorithm, named L∞ - BSVM in short. L∞ - BSVM includes the following merits: (1) it allows all sample points to participate in learning to prompt classification performance, especially in the case where the size of labeled data is small; (2) it minimizes the distance of the sample points that are (outliers in Non-i.i.d) farthest from the hyper-plane, where outliers are sufficiently taken into consideration (3) our iterative algorithm can solve large scale optimization problem with low time complexity and ensure the convergence of the optimum solution. Finally, extensive experiments on three types of datasets: artificial Non-i.i.d datasets, fault diagnosis of railway turnout with few labeled data (abnormal turnout) and six benchmark real-world datasets verify above opinions again and demonstrate that our classifier is much better than state-of-the-art competitors, such as B-SVM, LUHC, Pulce, B-LSSVM, NB and so on.

IBridge: Inferring bridge links that diffuse information across communities

While the accuracy of link prediction has been improved continuously, the utility of the inferred new links is rarely concerned especially when it comes to information diffusion. This paper defines the utility of links based on average shortest distance and more importantly defines a special type of links named bridge links based on community structure (overlapping or not) of the network. In sociology, bridge links are usually regarded as weak ties and play a more crucial role in information diffusion. Considering that the accuracy of previous link prediction methods is high in predicting strong ties but not much high in predicting weak ties, we propose a new link prediction method named iBridge, which aims to infer new bridge links using biased structural metrics in a PU (positive and unlabeled) learning framework. The experimental results in 3 real online social networks show that iBridge outperforms several comparative link prediction methods (based on supervised learning or PU learning) in inferring the bridge links and meantime, the overall performance of inferring bridge links and non-bridge links is not compromised, thus verifying its robustness in inferring all new links.

Study on the potential of whitening transformation in improving single crop mapping accuracy

In some practical classification applications, our focus may be just a specific crop (e.g., rice, corn, or soybean). Therefore, how to distinguish the crop of interest from other classes accurately and quickly is an issue worth studying. We propose an approach that combines whitening transformation (WT)/interest-class-based whitening transformation (ICWT) and one-class classification (OCC) methods to identify the crop of interest. The image created by applying WT/ICWT to the original image is referred to as the totally/partially whitened image, respectively. Three typical OCC classifiers—one-class support vector machine (OCSVM), positive and unlabeled learning (PUL), and maximum entropy (MaxEnt)—are trained to identify the crop of interest, using a totally/partially whitened image. The research results demonstrate that (1) both ICWT and WT enabled OCSVM to improve the producer’s accuracy of the crop of interest by more than 20%, and the performance of the proposed approach reached or surpassed that of conventional supervised methods; (2) PUL and MaxEnt performed better than OCSVM, especially in identifying local major crops using different phased data; and (3) the proposed approach is efficient in single crop identification with remote sensing imagery and is worth recommending for practical applications.

Boosting Positive and Unlabeled Learning for Anomaly Detection With Multi-Features

One of the key challenges of machine learning-based anomaly detection relies on the difficulty of obtaining anomaly data for training, which is usually rare, diversely distributed, and difficult to collect. To address this challenge, we formulate anomaly detection as a Positive and Unlabeled (PU) learning problem where only labeled positive (normal) data and unlabeled (normal and anomaly) data are required for learning an anomaly detector. As a semi-supervised learning method, it does not require providing labeled anomaly data for the training, thus it is easily deployed to various applications. As the unlabeled data can be extremely unbalanced, we introduce a novel PU learning method, which can tackle the situation where an unlabeled data set is mostly composed of positive instances. We start by using a linear model to extract the most reliable negative instances followed by a self-learning process to add reliable negative and positive instances with different speeds based on the estimated positive class prior. Furthermore, when feedback is available, we adopt boosting in the self-learning process to advantageously exploit the instability characteristic of PU learning. The classifiers in the self-learning process are weighted combined based on the estimated error rate to build the final classifier. Extensive experiments on six real datasets and one synthetic dataset show that our methods have better results under different conditions compared to existing methods.

Multi-Manifold Positive and Unlabeled Learning for Visual Analysis

Positive and Unlabeled (PU) learning has attracted intensive research interests in recent years, which is capable of training a binary classifier solely based on positive and unlabeled examples when the negative data are absent or too are diverse. However, the existing PU learning methods largely overlook the relationship between the examples when handling the unlabeled data, leading to insufficient exploitation of data structure which actually contains useful distribution information. Therefore, by following the multi-manifold assumption which is observed in many real-world vision problems, this paper proposes a novel algorithm termed “Multi-Manifold PU learning” (MMPU), which assumes that the data belonging to different classes lie on different underlying manifolds. As such, the structural information revealed by the dataset is deployed, which is helpful in deciding the labels of unlabeled examples. Our MMPU contains two main steps, namely, multi-manifold exploration and positive confidence training, where the former is accomplished by computing the local similarity, structural similarity, and semantic similarity of pairwise data, and the latter establishes a binary classifier in reproducing kernel Hilbert space based on the real-valued confidence level of each example to be positive. Experimentally, we not only test the proposed MMPU on five highly nonlinear synthetic datasets but also apply MMPU to various typical computer vision tasks, including handwritten digit recognition, violent behavior detection, and hyperspectral image classification. The results demonstrate that MMPU can obtain a superior performance compared to the state-of-the-art PU learning methodologies.

Prediction of Synthesis of 2D Metal Carbides and Nitrides (MXenes) and Their Precursors with Positive and Unlabeled Machine Learning.

Growing interest in the potential applications of two-dimensional (2D) materials has fueled advancement in the identification of 2D systems with exotic properties. Increasingly, the bottleneck in this field is the synthesis of these materials. Although theoretical calculations have predicted a myriad of promising 2D materials, only a few dozen have been experimentally realized since the initial discovery of graphene. Here, we adapt the state-of-the-art positive and unlabeled (PU) machine learning framework to predict which theoretically proposed 2D materials have the highest likelihood of being successfully synthesized. Using elemental information and data from high-throughput density functional theory calculations, we apply the PU learning method to the MXene family of 2D transition metal carbides, carbonitrides, and nitrides, and their layered precursor MAX phases, and identify 18 MXene compounds that are highly promising candidates for synthesis. By considering both the MXenes and their precursors, we further propose 20 synthesizable MAX phases that can be chemically exfoliated to produce MXenes.

A biased least squares support vector machine based on Mahalanobis distance for PU learning

In many domains, the presence of both positive and negative examples is not satisfied and only one class of examples is available. This special case of binary classification is called as PU (positive and unlabeled) learning in short. At present, many classification algorithms have been introduced for PU learning, such as BLSSVM, BSVM and so on. However, all of these classical approaches were measured by Euclidean distance, which did not take into account the correlative information of each class and the fluctuation of various attributions. In order to reflect this information, we propose a new Mahalanobis distance-based least squares support vector machines (MD-BLSSVM) classifier, in which two Mahalanobis distances are constructed according to the covariance matrices of two class data for optimizing the hyper-planes. Actually, MD-BLSSVM has a special case of BLSSVMs when the covariance matrices are degenerated to the identity matrix. The merits of MD-BLSSVM are (1) Mahalanobis distance of two classes can measure more suitable distance with certain weights on attributions; (2) Excellent kernel technique can be introduced in a reproducing kernel Hilbert space after making certain linear transformation ingeniously; (3) A solution is obtained simply by solving the system of linear equations. In all, MD-BLSSVM is appropriate to many real problems, especially for the case that the distribution and correlation of two classes of data are obviously different. The experimental results on several artificial and benchmark datasets indicate that MD-BLSSVM not only possess faster learning speed, but also obtains better generalization than BLSSVMs and other methods.

An Ensemble of Classifiers Based on Positive and Unlabeled Data in One-Class Remote Sensing Classification

One-class remote sensing classification refers to the situations when users are only interested in one specific land type without considering other types. The positive and unlabeled learning (PUL) algorithm, which trains a binary classifier from positive and unlabeled data, has been shown to be promising in one-class classification. The implementation of PUL by a single classifier has been investigated. However, implementing PUL using multiple classifiers and creating classifier ensembles based on PUL have not been studied. In this research, we investigate the implementations of PUL using several classifiers, including generalized linear model, generalized additive model, multivariate adaptive regression splines, maximum entropy, backpropagation neural network, and support vector machine, as well as three ensemble methods based on majority vote, weighted average, and weighted vote combination rules. These methods are applied in classifying the urban areas from four remote sensing imagery of different spatial resolutions, including aerial photograph, Landsat 8, WorldView-3, and Gaofen-1. Experimental results show that classifiers can successfully extract the urban areas with high accuracies, and the ensemble methods based on weighted average and weighted vote generally outperform the individual classifiers on different datasets. We conclude that PUL is a promising method in one-class remote sensing classification, and the classifier ensemble based on PUL can significantly improve the accuracy.

Positive and Unlabeled Learning for User Behavior Analysis Based on Mobile Internet Traffic Data

With the rapid development of wireless communication and mobile Internet, mobile phone becomes ubiquitous and functions as a versatile and a smart system, on which people frequently interact with various mobile applications (Apps). Understanding human behaviors using mobile phone is significant for mobile system developers, for human-centered system optimization and better service provisioning. In this paper, we focus on mobile user behavior analysis and prediction based on mobile Internet traffic data. Real traffic flow data is collected from the public network of Internet service providers, by high-performance network traffic monitors. We construct a User-App bipartite network to represent the traffic interaction pattern between users and App servers. After mining the explicit and implicit features from the User-App bipartite network, we propose two positive and unlabeled (PU) learning methods, including Spy-based PU learning and K-means-based PU learning, for App usage prediction and mobile video traffic identification. We first use the traffic flow data of QQ, a very famous messaging and social media application possessing high market share in China, as the experimental data set for App usage prediction task. Then, we use the traffic flow data from six popular Apps, including video intensive Apps (Youku, Baofeng, LeTV, and Tudou) and other Apps (Meituan and Apple), as the experimental data set for mobile video traffic identification task. Experimental results show that our proposed PU learning methods perform well in both tasks.

Global and local learning from positive and unlabeled examples

In common binary classification scenarios, learning algorithms assume the presence of both positive and negative examples. Unfortunately, in many practical areas, only limited labeled positive examples and large amounts of unlabeled examples are available, but there are no negative examples. In such cases, the algorithm that only exploits positive and unlabeled examples is needed. Such learning is termed as positive and unlabeled (PU) learning. In this paper, a novel classifier called global and local learning classifier (GLLC) for PU learning is proposed. The advantages of GLLC are as follows: (1) both intrinsic geometric structure and accurate positive information of PU data are exploited from global learning. (2) The smoothness and manifold of data are reflected sufficiently from local learning. (3) The algorithm of GLLC has faster training speed because the linear equations are solved. (4) The experiments on both synthetic and real datasets verify the above opinions and show that the classification result of GLLC is much better than those popular methods, such as LUHC, Pulce, BSVM, NB and so on.