Consensus and Reliability: The Case of Two Binary Classifiers

Abstract

In this paper we consider the problem of estimating the probability of misclas-sification when consensus is achieved between two binary classifiers that are trained on the same training set. Firstly, it is shown that, under consensus, the probability of misclassification compares favourably with that of the best of the two classifiers. Secondly, we provide accurate, and yet simple to compute, estimates of the probability of consensus and the probability of misclassification under consensus. This paper provides a theoretical basis for these estimates and demonstrates their accuracy by simulation results on a synthetic data set and on a medical data set for breast cancer cell classification.