Transductive Approach Research Articles

Transductive Methods for the Distributed Ensemble Classification Problem

We consider ensemble classification for the case where there is no common labeled training data for jointly designing the individual classifiers and the function that aggregates their decisions. This problem, which we call distributed ensemble classification, applies when individual classifiers operate (perhaps remotely) on different sensing modalities and when combining proprietary or legacy classifiers. The conventional wisdom in this case is to apply fixed rules of combination such as voting methods or rules for aggregating probabilities. Alternatively, we take a transductive approach, optimizing the combining rule for an objective function measured on the unlabeled batch of test data. We propose maximum likelihood (ML) objectives that are shown to yield well-known forms of probabilistic aggregation, albeit with iterative, expectation-maximization-based adjustment to account for mismatch between class priors used by individual classifiers and those reflected in the new data batch. These methods are extensions, for the ensemble case, of the work of Saerens, Latinne, and Decaestecker (2002). We also propose an information-theoretic method that generally outperforms the ML methods, better handles classifier redundancies, and addresses some scenarios where the ML methods are not applicable. This method also well handles the case of missing classes in the test batch. On UC Irvine benchmark data, all our methods give improvements in classification accuracy over the use of fixed rules when there is prior mismatch.

Effective transductive learning via objective model selection

This paper is concerned with transductive learning. We study a recent transductive learning approach based on clustering. In this approach one constructs a diversity of unsupervised models of the unlabeled data using clustering algorithms. These models are then exploited to construct a number of hypotheses using the labeled data and the learner selects an hypothesis that minimizes a transductive error bound, which holds with high probability. Empirical examination of this approach, implemented with ‘spectral clustering’, on a suite of benchmark datasets from the UCI repository, indicates that the new approach is effective and comparable with one of the best known transductive learning algorithms to-date.

Transductive Machine Learning for Reliable Medical Diagnostics

In the past decades Machine Learning tools have been successfully used in several medical diagnostic problems. While they often significantly outperform expert physicians (in terms of diagnostic accuracy, sensitivity, and specificity), they are mostly not being used in practice. One reason for this is that it is difficult to obtain an unbiased estimation of diagnose's reliability. We discuss how reliability of diagnoses is assessed in medical decision making and propose a general framework for reliability estimation in Machine Learning, based on transductive inference. We compare our approach with a usual (Machine Learning) probabilistic approach as well as with classical stepwise diagnostic process where reliability of diagnose is presented as its posttest probability. The proposed transductive approach is evaluated on several medical data sets from the UCI (University of California, Irvine) repository as well as on a practical problem of clinical diagnosis of the coronary artery disease. In all cases significant improvements over existing techniques are achieved.

Transductive reliability estimation for medical diagnosis

In the past decades, machine learning (ML) tools have been successfully used in several medical diagnostic problems. While they often significantly outperform expert physicians (in terms of diagnostic accuracy, sensitivity, and specificity), they are mostly not being used in practice. One reason for this is that it is difficult to obtain an unbiased estimation of diagnose’s reliability. We discuss how reliability of diagnoses is assessed in medical decision-making and propose a general framework for reliability estimation in machine learning, based on transductive inference. We compare our approach with a usual (machine learning) probabilistic approach as well as with classical stepwise diagnostic process where reliability of diagnose is presented as its post-test probability. The proposed transductive approach is evaluated on several medical datasets from the University of California (UCI) repository as well as on a practical problem of clinical diagnosis of the coronary artery disease (CAD). In all cases, significant improvements over existing techniques are achieved.

Learning with progressive transductive support vector machine

Support vector machine (SVM) is a new learning method developed in recent years based on the foundations of statistical learning theory. By taking a transductive approach instead of an inductive one in support vector classifiers, the working set can be used as an additional source of information about margins. Compared with traditional inductive support vector machines, transductive support vector machine is often more powerful and can give better performance. In transduction, one estimates the classification function at points within the working set using information from both the training and the working set data. This will help to improve the generalization performance of SVMs, especially when training data is inadequate. Intuitively, we would expect transductive learning to yield improvements when the training sets are small or when there is a significant deviation between the training and working set subsamples of the total population. In this paper, a progressive transductive support vector machine is addressed to extend Joachims' transductive SVM to handle different class distributions. It solves the problem of having to estimate the ratio of positive/negative examples from the working set. The experimental results show the algorithm is very promising.