Constructing “Proper” ROCs from Ordinal Response Data Using Weighted Power Functions

Abstract

Receiver operating characteristic (ROC) analysis is the standard method for describing the accuracy of diagnostic systems where the decision task involves distinguishing between 2 mutually exclusive possibilities. The popular binormal curve-fitting model usually produces ROCs that are improper in that they do not have the ever-decreasing slope required by signal detection theory. Not infrequently, binormal ROCs have visible hooks that falsely imply worse-than-chance diagnostic differentiation where the curve lies below the no-information diagonal. In this article, we present and evaluate a 2-parameter, weighted power function (WPF) model that always results in a proper ROC curve with a positive, monotonically decreasing slope. We used a computer simulation study to compare results from binormal and WPF models. The WPF model produces ROC curves that are less biased and closer to the true values than are curves obtained using the binormal model. The better performance of the WPF model follows from its design constraint as a necessarily proper ROC. The WPF model fits a broader variety of data sets than previously published power function models while maintaining straightforward relationships among the original decision variable, specific operating points, ROC curve contours, and model parameters. Compared with other proper ROC models, the WPF model is distinctive in its simplicity, and it avoids the flaws of the conventional binormal ROC model.