BA2Cs: Bounded abstaining with two constraints of reject rates in binary classification

Abstract

Abstaining classifiers refrain from uncertain classifications to decrease errors and have been widely applied in critical fields. There are two types of abstaining classifiers: Chow’s rule-based and receiver operating characteristic (ROC)-based. Chow’s rule is obtained by minimizing the overall error rate given a reject rate. Therefore, Chow’s rule-based classifiers do not consider the difference between false positives and false negatives. ROC-based abstaining classifiers minimize cost functions; therefore, costs of classification and rejection need to set during the training stage. However, in real-world applications, misclassification costs of different classes are usually unequal. Furthermore, costs of classification and rejection are not easy to obtain and they may vary over time. To overcome such drawbacks, we propose a bounded abstaining classifier with two constraints of reject rates (BA2Cs). BA2Cs aims at maximizing the area under the ROC curve (AUC) while constraining the reject rates of the positive and negative classes, respectively. Optimizing AUC allows the classifier independent from class distributions and misclassification costs. We propose an efficient algorithm, which has linear time complexity, to construct the BA2Cs classifier. The algorithm searches rejection thresholds according to an incremental moving rule until the two constraints are satisfied. Experimental results show that BA2Cs obtains higher AUC and lower cost than state-of-the-art abstaining classifiers.