Improvement of Waegeman–Baets–Boullart algorithms for ordered multi-class ROC analysis

Abstract

To accommodate multi-class scenarios, area under the receiver operating characteristic (ROC) curve (AUC) has been extended to volume under the ROC hyper-surface (VUHS) to measure the overall power of a model to classify objects belonging to various (more than two) categories. In their state-of-the-art work, Waegeman, Baets and Boullart proposed three algorithms, referred to as WBBA0, WBBA1 and WBBA2 in this work, to compute the point estimator of VUHS as well as its variance and covariance for two models. Unfortunately, WBBA1 and WBBA2 are only asymptotically unbiased, which therefore might be misleading in practice when the sample size is small. To overcome such drawbacks, in this work we develop unbiased versions of WBBA1 and WBBA2 by removing the redundant terms and modifying the coefficients contributing to the effect of bias within the graph-based framework of Waegeman et al. Theoretical analyses as well as Monte Carlo simulations verify that the revised versions of WBBA1 and WBBA2 proposed are not only strictly unbiased, but also have comparable computational loads with their original counterparts.