Traditional Co-training Research Articles

A reinforcement learning approach for single redundant view co-training text classification

We tackle the problem of learning classification models with very small amounts of labeled data (e.g., less than 10% of the dataset) by introducing a novel Single View Co-Training strategy supported by Reinforcement Learning (CoRL). CoRL is a novel semi-supervised learning framework that can be used with a single view (representation). Differently from traditional co-training that requires at least two sufficient and independent data views (e.g., modes), our solution is applicable to any kind of data. Our approach exploits a reinforcement learning (RL) paradigm as a strategy to relax the view independence assumption, using a stronger iterative agent that builds more precise combined decision class boundaries. Our experimental evaluation with four popular textual benchmarks demonstrates that CoRL can produce better classifiers than confidence-based co-training methods, while producing high effectiveness in comparison with the state-of-the-art in semi-supervised learning. In our experiments, CoRL reduced the labeling effort by more than 80% with no losses in classification effectiveness, outperforming state-of-the-art baselines, including methods based on neural networks, with gains of up to 96% against some of the best competitors.

A co-training approach for sequential three-way decisions

In recent years, three-way decisions have received much attention in uncertain decision and cost-sensitive learning communities. However, in many real applications, labeled samples are usually far from sufficient. In this case, it is a reasonable choice to defer the decision rather than make an immediate decision without sufficient supported information, thus it constructs a boundary region. In order to label more available samples, a traditional co-training method employs two classifiers on two complementary views to extend the existing training sets. However, the wrong predictions of new labels may lead to a high misclassification cost, especially when few labeled samples are available. To address this problem, a co-training method is incorporated into three-way decisions, which can label new samples with higher confidence. When we obtain sufficient labeled samples, the non-commitment decisions are directly decided to a positive or a negative region, which finally generates a two-way decisions result. Experiments on several face databases are conducted to validate the effectiveness of the proposed approach.

Development of semi-supervised multiple-output soft-sensors with Co-training and tri-training MPLS and MRVM

Soft sensors are the most commonly used tools to estimate the hard-to-measure variables in the chemical processes and other industries, mainly due to unknown mechanism, significant measurement delay and highly unacceptable costs. However, a small number of labeled data and a large number of unlabeled data are not fully investigated and coordination of them to train the models and to improve the prediction accuracy is even rare. Multiple tasks learning or multiple outputs learning adds more complexity to this problem. In this light, this paper proposed semi-supervised multiple-output learning soft sensor models with co-training MPLS (Multiple-output partial least squares), co-training MRVM (Multiple-output relevance vector machines), tri-training MPLS and tri-training MRVM. Co-training MPLS model is developed by extending the traditional co-training PLS model. Co-training MRVM is promoted by replacing MPLS with MRVM. Tri-training MPLS and tri-training MRVM are built by combining tri-training algorithm with MPLS and MRVM. The proposed four models can make full use of appropriate unlabeled data to optimize the regression model, and then to directly strengthen multiple-output variables prediction. These models were firstly demonstrated by a numerical example, then accessed by a wastewater plant (WWTP) simulated with well-established WWTP validation platform, Benchmark Simulation Model No. 1 (BSM1). The results proved that the proposed models were able to significantly improve the prediction performance and efficiency.

Arabic named entity recognition via deep co-learning

Named entity recognition (NER) is an important natural language processing (NLP) task with many applications. We tackle the problem of Arabic NER using deep learning based on Arabic word embeddings that capture syntactic and semantic relationships between words. Deep learning has been shown to perform significantly better than other approaches for various NLP tasks including NER. However, deep-learning models also require a significantly large amount of training data, which is highly lacking in the case of the Arabic language. To remedy this, we adopt the semi-supervised co-training approach to the realm of deep learning, which we refer to as deep co-learning. Our deep co-learning approach makes use of a small amount of labeled data, which is augmented with partially labeled data that is automatically generated from Wikipedia. Our approach relies only on word embeddings as features and does not involve any additional feature engineering. Nonetheless, when tested on three different Arabic NER benchmarks, our approach consistently outperforms state-of-the-art Arabic NER approaches, including ones that employ carefully-crafted NLP features. It also consistently outperforms various baselines including purely-supervised deep-learning approaches as well as semi-supervised ones that make use of only unlabeled data such as self-learning and the traditional co-training approach.

Hessian-regularized co-training for social activity recognition.

Co-training is a major multi-view learning paradigm that alternately trains two classifiers on two distinct views and maximizes the mutual agreement on the two-view unlabeled data. Traditional co-training algorithms usually train a learner on each view separately and then force the learners to be consistent across views. Although many co-trainings have been developed, it is quite possible that a learner will receive erroneous labels for unlabeled data when the other learner has only mediocre accuracy. This usually happens in the first rounds of co-training, when there are only a few labeled examples. As a result, co-training algorithms often have unstable performance. In this paper, Hessian-regularized co-training is proposed to overcome these limitations. Specifically, each Hessian is obtained from a particular view of examples; Hessian regularization is then integrated into the learner training process of each view by penalizing the regression function along the potential manifold. Hessian can properly exploit the local structure of the underlying data manifold. Hessian regularization significantly boosts the generalizability of a classifier, especially when there are a small number of labeled examples and a large number of unlabeled examples. To evaluate the proposed method, extensive experiments were conducted on the unstructured social activity attribute (USAA) dataset for social activity recognition. Our results demonstrate that the proposed method outperforms baseline methods, including the traditional co-training and LapCo algorithms.

An Active Co-Training Algorithm for Biomedical Named-Entity Recognition

Exploiting unlabeled text data with a relatively small labeled corpus has been an active and challenging research topic in text mining, due to the recent growth of the amount of biomedical literature. Biomedical named-entity recognition is an essential prerequisite task before effective text mining of biomedical literature can begin. This paper proposes an Active Co-Training (ACT) algorithm for biomedical named-entity recognition. ACT is a semi-supervised learning method in which two classifiers based on two different feature sets iteratively learn from informative examples that have been queried from the unlabeled data. We design a new classification problem to measure the informativeness of an example in unlabeled data. In this classification problem, the examples are classified based on a joint view of a feature set to be informative/non-informative to both classifiers. To form the training data for the classification problem, we adopt a query-by-committee method. Therefore, in the ACT, both classifiers are considered to be one committee, which is used on the labeled data to give the informativeness label to each example. The ACT method outperforms the traditional co-training algorithm in terms of f-measure as well as the number of training iterations performed to build a good classification model. The proposed method tends to efficiently exploit a large amount of unlabeled data by selecting a small number of examples having not only useful information but also a comprehensive pattern.