Unlabeled Examples Research Articles

Estimating the Class Prior in Positive and Unlabeled Data Through Decision Tree Induction

For tasks such as medical diagnosis and knowledge base completion, a classifier may only have access to positive and unlabeled examples, where the unlabeled data consists of both positive and negative examples. One way that enables learning from this type of data is knowing the true class prior. In this paper, we propose a simple yet effective method for estimating the class prior, by estimating the probability that a positive example is selected to be labeled. Our key insight is that subdomains of the data give a lower bound on this probability. This lower bound gets closer to the real probability as the ratio of labeled examples increases. Finding such subsets can naturally be done via top-down decision tree induction. Experiments show that our method makes estimates which are equivalently accurate as those of the state of the art methods, and is an order of magnitude faster.

Revisiting transductive support vector machines with margin distribution embedding

Transductive Support Vector Machine (TSVM) is one of the most successful classification methods for semi-supervised learning (SSL). One challenge of TSVMs is that the performance degeneration is caused by unlabeled examples that are obscure or misleading for the discovery of the underlying distribution. To address this problem, we disclose the underlying data distribution and describe the margin distribution of TSVMs as the first-order (margin mean) and second-order (margin variance) statistics of examples. Since the optimization problems of TSVMs are not convex, we utilize the concave-convex procedure and variation of stochastic variance reduced gradient methods to solve them. Particularly, we propose two specific algorithms to optimize the margin distribution of TSVM via maximizing the margin mean and minimizing the margin variance simultaneously, which the generalization ability is improved and being robust to the outliers and noise. In addition, we derive a bound on the expectation of error according to the leave-one-out cross-validation estimate, which is an unbiased estimate of the probability of test error. Finally, to validate the effectiveness of the proposed method, extensive experiments are conducted on diversity datasets. The experimental results demonstrate that the performance of proposed algorithms are superior to the existing TSVMs and other semi-supervised learning methods.

Semi-supervised trees for multi-target regression

The predictive performance of traditional supervised methods heavily depends on the amount of labeled data. However, obtaining labels is a difficult process in many real-life tasks, and only a small amount of labeled data is typically available for model learning. As an answer to this problem, the concept of semi-supervised learning has emerged. Semi-supervised methods use unlabeled data in addition to labeled data to improve the performance of supervised methods.It is even more difficult to get labeled data for data mining problems with structured outputs since several labels need to be determined for each example. Multi-target regression (MTR) is one type of a structured output prediction problem, where we need to simultaneously predict multiple continuous variables. Despite the apparent need for semi-supervised methods able to deal with MTR, only a few such methods are available and even those are difficult to use in practice and/or their advantages over supervised methods for MTR are not clear.This paper presents an extension of predictive clustering trees for MTR and ensembles thereof towards semi-supervised learning. The proposed method preserves the appealing characteristic of decision trees while enabling the use of unlabeled examples. In particular, the proposed semi-supervised trees for MTR are interpretable, easy to understand, fast to learn, and can handle both numeric and nominal descriptive features. We perform an extensive empirical evaluation in both an inductive and a transductive semi-supervised setting. The results show that the proposed method improves the performance of supervised predictive clustering trees and enhances their interpretability (due to reduced tree size), whereas, in the ensemble learning scenario, it outperforms its supervised counterpart in the transductive setting. The proposed methods have a mechanism for controlling the influence of unlabeled examples, which makes them highly useful in practice: This mechanism can protect them against a degradation of performance of their supervised counterparts – an inherent risk of semi-supervised learning. The proposed methods also outperform two existing semi-supervised methods for MTR.

Using Semi-Supervised Learning in Cellular Automata for Edge Detection of Image Segmentation

Semi-Supervised Learning is an area of increasing importance in Machine Learning techniques that make use of both labeled and unlabeled data. The goal of using both labeled and unlabeled data is to build better learners instead of using each one alone. Semi-supervised learning investigates how to use the information of both labeled and unlabeled examples to perform better than supervised learning. In this paper we present a new method for edge detection of image segmentation using cellular automata with modification for game of life rules and K-means algorithm. We use the semi-supervised clustering method, which can jointly learn to fusion by making use of the unlabeled data. The learning aim consists in distinguishing between edge and no edge for each pixel in image. We have applied the semi-supervised method for finding edge detection in natural image and measured its performance using the Berkeley Segmentation Dataset and Benchmark dataset. The results and experiments showed the accuracy and efficiency of the proposed method.

Clustering data with partial background information

Clustering with partial supervision background information or semi-supervised clustering, learning from a combination of both labeled and unlabeled data, has received a lot of attention over the last decade. The supervisory information is usually used as the constraints to bias clustering towards a good region of search space. This paper proposes a semi-supervised algorithm, called constrained non-negative matrix factorization (Constrained-NMF), with a few labeled examples as constraints to improve performance. The proposed algorithm is a matrix factorization algorithm, in which initialization of matrices is required at the beginning. Although the benefits of good initialization are well-known, randomized seeding of basis matrix and coefficient matrix is still the standard approach for many non-negative matrix factorization (NMF) algorithms. This work devises an algorithm called entropy-based weighted semi-supervised fuzzy c-means (EWSS-FCM) algorithm to initialize the seeds. The experimental results indicate that the proposed Constrained-NMF can benefit from the initialization obtained from EWSS-FCM, which emphasizes the role of labeled examples and automatically weights them during the course of clustering. This work considers labeled examples in the objective functions to devise the two algorithms, in which the labeled information is propagated to unlabeled examples iteratively. We further analyze the proposed Constrained-NMF and give convergence justifications. The experiments are conducted on five real data sets, and experimental results indicate that the proposed algorithm generally outperforms the other alternatives.

Evolutionary Strategy to Perform Batch-Mode Active Learning on Multi-Label Data

Multi-label learning has become an important area of research owing to the increasing number of real-world problems that contain multi-label data. Data labeling is an expensive process that requires expert handling. The annotation of multi-label data is laborious since a human expert needs to consider the presence/absence of each possible label. Consequently, numerous modern multi-label problems may involve a small number of labeled examples and plentiful unlabeled examples simultaneously. Active learning methods allow us to induce better classifiers by selecting the most useful unlabeled data, thus considerably reducing the labeling effort and the cost of training an accurate model. Batch-mode active learning methods focus on selecting a set of unlabeled examples in each iteration in such a way that the selected examples are informative and as diverse as possible. This article presents a strategy to perform batch-mode active learning on multi-label data. The batch-mode active learning is formulated as a multi-objective problem, and it is solved by means of an evolutionary algorithm. Extensive experiments were conducted in a large collection of datasets, and the experimental results confirmed the effectiveness of our proposal for better batch-mode multi-label active learning.

On the automation of gestalt perception in remotely sensed data

Gestalt perception, the laws of seeing, and perceptual grouping is rarely addressed in the context of remotely sensed imagery. The paper at hand reviews the corresponding state as well in machine vision as in remote sensing, in particular concerning urban areas. Automatic methods can be separated into three types: 1) knowledge-based inference, which needs machine-readable knowledge, 2) automatic learning methods, which require labeled or un-labeled example images, and 3) perceptual grouping along the lines of the laws of seeing, which should be pre-coded and should work on any kind of imagery, but in particular on urban aerial or satellite data. Perceptual grouping of parts into aggregates is a combinatorial problem. Exhaustive enumeration of all combinations is intractable. The paper at hand presents a constant-false-alarm-rate search rationale. An open problem is the choice of the extraction method for the primitive objects to start with. Here super-pixel-segmentation is used.

Collaborative Filtering Recommendation Based on All-Weighted Matrix Factorization and Fast Optimization

Collaborative filtering recommendation with implicit feedbacks (e.g., clicks, views, and plays) is regarded as one of the most challenging issues in both academia and industry. From implicit feedbacks, we can only get a small fraction of observed data (positive examples), and the massive unobserved data are the mixture of negative examples and unlabeled positive examples. However, most of the existing efforts either treat unobserved data equally by assigning a uniform weight or uniformly weight observed data while ignoring the hidden information (i.e., visit frequency) in implicit feedbacks. This assumption may not hold in real-life scenarios since they cannot distinguish the contributions of the whole data and it easily leads to prediction bias. Besides, those approaches still suffer from low-efficiency issue. To this end, we propose an all-weighted matrix factorization and fast optimization strategy for effective and efficient recommendation. We first design a frequency-aware weighting scheme for observed data and a user-oriented weighting scheme for unobserved data nonuniformly. Then, the weighting schemes of both observed and unobserved data are combined in a unified way to form an all-weighted matrix factorization model. Afterwards, we present a surrogate objective function and develop a fast optimization strategy to enhance the efficiency. Extensive experimental results on real-world datasets demonstrate that our method outperforms the competitive baselines on several evaluation metrics.

Ensemble Teaching for Hybrid Label Propagation.

Label propagation aims to iteratively diffuse the label information from labeled examples to unlabeled examples over a similarity graph. Current label propagation algorithms cannot consistently yield satisfactory performance due to two reasons: one is the instability of single propagation method in dealing with various practical data, and the other one is the improper propagation sequence ignoring the labeling difficulties of different examples. To remedy above defects, this paper proposes a novel propagation algorithm called hybrid diffusion under ensemble teaching (HyDEnT). Specifically, HyDEnT integrates multiple propagation methods as base "learners" to fully exploit their individual wisdom, which helps HyDEnT to be stable and obtain consistent encouraging results. More importantly, HyDEnT conducts propagation under the guidance of an ensemble of "teachers". That is to say, in every propagation round the simplest curriculum examples are wisely designated by a teaching algorithm, so that their labels can be reliably and accurately decided by the learners. To optimally choose these simplest examples, every teacher in the ensemble should comprehensively consider the examples' difficulties from its own viewpoint, as well as the common knowledge shared by all the teachers. This is accomplished by a designed optimization problem, which can be efficiently solved via the block coordinate descent method. Thanks to the efforts of the teachers, all the unlabeled examples are logically propagated from simple to difficult, leading to better propagation quality of HyDEnT than the existing methods. Experiments on six popular datasets reveal that HyDEnT achieves the highest classification accuracy when compared with six state-of-the-art propagation methodologies such as harmonic functions, Fick's law assisted propagation, linear neighborhood propagation, semisupervised ensemble learning, bipartite graph-based consensus maximization, and teaching-to-learn and learning-to-teach.

Global and local learning from positive and unlabeled examples

In common binary classification scenarios, learning algorithms assume the presence of both positive and negative examples. Unfortunately, in many practical areas, only limited labeled positive examples and large amounts of unlabeled examples are available, but there are no negative examples. In such cases, the algorithm that only exploits positive and unlabeled examples is needed. Such learning is termed as positive and unlabeled (PU) learning. In this paper, a novel classifier called global and local learning classifier (GLLC) for PU learning is proposed. The advantages of GLLC are as follows: (1) both intrinsic geometric structure and accurate positive information of PU data are exploited from global learning. (2) The smoothness and manifold of data are reflected sufficiently from local learning. (3) The algorithm of GLLC has faster training speed because the linear equations are solved. (4) The experiments on both synthetic and real datasets verify the above opinions and show that the classification result of GLLC is much better than those popular methods, such as LUHC, Pulce, BSVM, NB and so on.

Stopping Criterion for Active Learning with Model Stability

Active learning selectively labels the most informative instances, aiming to reduce the cost of data annotation. While much effort has been devoted to active sampling functions, relatively limited attention has been paid to when the learning process should stop. In this article, we focus on the stopping criterion of active learning and propose a model stability--based criterion, that is, when a model does not change with inclusion of additional training instances. The challenge lies in how to measure the model change without labeling additional instances and training new models. Inspired by the stochastic gradient update rule, we use the gradient of the loss function at each candidate example to measure its effect on model change. We propose to stop active learning when the model change brought by any of the remaining unlabeled examples is lower than a given threshold. We apply the proposed stopping criterion to two popular classifiers: logistic regression (LR) and support vector machines (SVMs). In addition, we theoretically analyze the stability and generalization ability of the model obtained by our stopping criterion. Substantial experiments on various UCI benchmark datasets and ImageNet datasets have demonstrated that the proposed approach is highly effective.

Two birds with one stone: Classifying positive and unlabeled examples on uncertain data streams

Abstract An important feature characteristic of the data streams in many of today's big data applications is the intrinsic uncertainty, which could happen for both item occurrence and attribute value. While this has already posed great challenges for fundamental data mining tasks such as classification, things are made even more complicated by the fact that completely-labeled examples are usually unavailable in such settings, leaving researchers the only option to learn classifiers on partially-labeled examples on uncertain data streams. Furthermore, there will be concept drift on evolving data streams. To address these challenges, this paper therefore focuses on the study of learning from positive and unlabeled examples (PU) on uncertain data streams. To the best of our knowledge, this paper is the first work to address the uncertainty issue in both item occurrence (occurrence level) and attribute value (attribute level) for the problem of PU learning over streaming data. Firstly, we propose an algorithm to classify positive and unlabeled examples with both uncertainties (PUU). The algorithm extracts reliable positive and negative examples by clustering-based method, and then trains the classifier with Weighted Extreme Learning Machine (Weighted ELM). Secondly, we propose an algorithm of PU learning over uncertain data streams (PUUS). It adopts ensemble model and trains base classification by PUU. In order to detect concept drift, PUUS uses cluster set similarities between the current data block and history data block. We propose different update strategies for different concept drift to adapt to evolving uncertain data streams. Experimental results show that PUUS can effectively classify uncertain data streams with just positive and unlabeled data, while achieving in the meantime good performance in detecting and handling concept drifts.

Effective active learning strategy for multi-label learning

Data labelling is commonly an expensive process that requires expert handling. In multi-label data, data labelling is further complicated owing to the experts must label several times each example, as each example belongs to various categories. Active learning is concerned with learning accurate classifiers by choosing which examples will be labelled, reducing the labelling effort and the cost of training an accurate model. The main challenge in performing multi-label active learning is designing effective strategies that measure the informative potential of unlabelled examples across all labels. This paper presents a new active learning strategy for working on multi-label data. Two uncertainty measures based on the base classifier predictions and the inconsistency of a predicted label set, respectively, were defined to select the most informative examples. The proposed strategy was compared to several state-of-the-art strategies on a large number of datasets. The experimental results showed the effectiveness of the proposal for better multi-label active learning.

Exploiting the value of class labels on high-dimensional feature spaces: topic models for semi-supervised document classification

We propose a class-based mixture of topic models for classifying documents using both labeled and unlabeled examples (i.e., in a semi-supervised fashion). Most topic models incorporate documents’ class labels by generating them after generating the words. In these models, the training class labels have small effect on the estimated topics, as they are effectively treated as just another word, amongst a huge set of word features. In this paper, we propose to increase the influence of class labels on topic models by generating the words in each document conditioned on the class label. We show that our specific generative process improves classification performance with small loss in test set log-likelihood. Within our framework, we provide a principled mechanism to control the contributions of the class labels and the word space to the likelihood function. Experiments show our approach achieves better classification accuracy compared to some standard semi-supervised and supervised topic models.

Instance labeling in semi-supervised learning with meaning values of words

In supervised learning systems; only labeled samples are used for building a classifier that is then used to predict the class labels of the unlabeled samples. However, obtaining labeled data is very expensive, time consuming and difficult in real-life practical situations as labeling a data set requires the effort of a human expert. On the other side, unlabeled data are often plentiful which makes it relatively inexpensive and easier to obtain. Semi-Supervised Learning methods strive to utilize this plentiful source of unlabeled examples to increase the learning capacity of the classifier particularly when amount of labeled examples are restricted. Since SSL techniques usually reach higher accuracy and require less human effort, they attract a substantial amount of attention both in practical applications and theoretical research. A novel semi-supervised methodology is offered in this study. This algorithm utilizes a new method to predict the class labels of unlabeled examples in a corpus and incorporate them into the training set to build a better classifier. The approach presented here depends on a meaning calculation, which computes the words’ meaning scores in the scope of classes. Meaning computation is constructed on the Helmholtz principle and utilized to various applications in the field of text mining like feature extraction, information retrieval and document summarization. Nevertheless, according to the literature, ILBOM is the first work which uses meaning calculation in a semi-supervised way to construct a semantic smoothing kernel for Support Vector Machines (SVM). Evaluation of the proposed methodology is done by performing various experiments on standard textual datasets. ILBOM's experimental results are compared with three baseline algorithms including SVM using linear kernel which is one of the most frequently used algorithms in text classification field. Experimental results show that labeling unlabeled instances based on meaning scores of words to augment the training set is valuable, and increases the classification accuracy on previously unseen test instances significantly.

An uncertainty and density based active semi-supervised learning scheme for positive unlabeled multivariate time series classification

In reality, the number of labeled time series data is often small and there is a huge number of unlabeled data. Manually labeling these unlabeled examples is time-consuming and expensive, and sometimes it is even impossible. In this paper, we combine active learning and semi-supervised learning to obtain a confident and sufficient labeled training data for multivariate time series classification. We first propose a sampling strategy by ranking the informativeness of unlabeled examples based on its uncertainty and its local data density. Next, an active semi-supervised learning framework is introduced to make best use of the advantage of active learning and semi-supervised learning for data annotation. Finally, we advance a valid stopping criterion of active learning to provide a sufficient and reliable labeled training dataset by costing human resources as less as possible. Our experimental results show that our approach can manually annotate examples as small as possible and simultaneously obtain a confident and informative labeled dataset, which is sufficient to learn an efficient classification.

Graph Structure Learning from Unlabeled Data for Early Outbreak Detection

Processes such as disease propagation and information diffusion often spread over some latent network structure that must be learned from observation. Given a set of unlabeled training examples representing occurrences of an event type of interest (such as a disease outbreak), the authors aim to learn a graph structure that can be used to accurately detect future events of that type. They propose a novel framework for learning graph structure from unlabeled data by comparing the most anomalous subsets detected with and without the graph constraints. Their framework uses the mean normalized log-likelihood ratio score to measure the quality of a graph structure, and it efficiently searches for the highest-scoring graph structure. Using simulated disease outbreaks injected into real-world Emergency Department data from Allegheny County, the authors show that their method learns a structure similar to the true underlying graph, but enables faster and more accurate detection.

Case-Based Statistical Learning: A Non-Parametric Implementation With a Conditional-Error Rate SVM

Machine learning has been successfully applied to many areas of science and engineering. Some examples include time series prediction, optical character recognition, signal and image classification in biomedical applications for diagnosis and prognosis and so on. In the theory of semi-supervised learning, we have a training set and an unlabeled data, that are employed to fit a prediction model or learner , with the help of an iterative algorithm, such as the expectation-maximization algorithm. In this paper, a novel non-parametric approach of the so-called case-based statistical learning is proposed in a low-dimensional classification problem. This supervised feature selection scheme analyzes the discrete set of outcomes in the classification problem by hypothesis-testing and makes assumptions on these outcome values to obtain the most likely prediction model at the training stage. A novel prediction model is described in terms of the output scores of a confidence-based support vector machine classifier under class-hypothesis testing. To have a more accurate prediction by considering the unlabeled points, the distribution of unlabeled examples must be relevant for the classification problem. The estimation of the error rates from a well-trained support vector machines allows us to propose a non-parametric approach avoiding the use of Gaussian density function-based models in the likelihood ratio test.

Boosted Web Named Entity Recognition via Tri-Training

Named entity extraction is a fundamental task for many natural language processing applications on the web. Existing studies rely on annotated training data, which is quite expensive to obtain large datasets, limiting the effectiveness of recognition. In this research, we propose a semisupervised learning approach for web named entity recognition (NER) model construction via automatic labeling and tri-training. The former utilizes structured resources containing known named entities for automatic labeling, while the latter makes use of unlabeled examples to improve the extraction performance. Since this automatically labeled training data may contain noise, a self-testing procedure is used as a follow-up to remove low-confidence annotation and prepare higher-quality training data. Furthermore, we modify tri-training for sequence labeling and derive a proper initialization for large dataset training to improve entity recognition. Finally, we apply this semisupervised learning framework for person name recognition, business organization name recognition, and location name extraction. In the task of Chinese NER, an F-measure of 0.911, 0.849, and 0.845 can be achieved, for person, business organization, and location NER, respectively. The same framework is also applied for English and Japanese business organization name recognition and obtains models with performance of a 0.832 and 0.803 F-measure.

Summit-Training: A hybrid Semi-Supervised technique and its application to classification tasks

With an explosion in the amount of data available for classification problems it becomes more and more complicated to obtain labels for the whole dataset. Because of this we are frequently presented with only a fraction of labeled data from the whole dataset. To leverage the presence of unlabeled examples Semi-Supervised methods could be used to increase the precision of used classifiers. Therefore, we propose a novel hybrid technique which extends the concept of Self-Training and Help-Training used in Semi-Supervised techniques by incorporating the Active Learning approach for determining the confidence of the classifier in the testing set samples. Specifically we employ the Query-by-Committee (QbC) approach and we call the final method Summit-Training. We apply this method to a range of classifiers (generative and discriminative) and evaluate it on several datasets and real world problems. The formulation of the Summit-Training method especially allows us to use Semi-Supervised approaches for purely discriminative classifiers in which no probabilistic representation of the evaluated classes exists. Compared to other Semi-Supervised techniques (Self-Training and Help-Training), the proposed new method achieves superior performance. It also has better generalization properties since it reduces the number of hyper-parameters and relaxes the conditions for classifiers on which it could be applied.