Optimal receiver operating characteristic manifolds

Abstract

To evaluate the overall discrimination capacity of a marker for multi-class classification tasks, the performance function is a natural assessment tool and fully provides the essential ingredients in receiver operating characteristic (ROC) analysis. The optimal ROC manifolds supply a geometric characterization of the magnitude of separation among multiple classes. It has been shown that the hypervolume under the optimal ROC manifold (HUM) is a well-defined and meaningful accuracy measure only in suitable ROC subspaces. In this article, we provided a rigorous proof for the equality of HUM and its alternative form, the correctness probability, which is directly related to an explicit U-estimator. In addition, extensive simulations are conducted to investigate the finite sample properties of the proposed estimators and the related inference procedures. Further, a rule of thumb is given in application to assess for the HUM. Conclusively, our theoretical framework allows more sophisticated modeling on the performance of markers and helps practitioners examine the optimality of applied classification procedures.