Smooth non‐parametric receiver operating characteristic (ROC) curves for continuous diagnostic tests

Abstract

Receiver operating characteristic (ROC) curves have seen increasing use to assess the accuracy of diagnostic tests that yield continuous test results. We can calculate a fully non-parametric ROC curve from the empirical false positive rate (FPR) and true positive rate (TPR) at each possible decision threshold, but it may be quite jagged. We can obtain a smooth ROC curve by directly fitting a parametric model, for example, binormal or bilogistic, to the actual test results, but substantial lack-of-fit may result if the distributional assumptions are not valid. A recently proposed alternative algorithm ‘LABROC4’ is insensitive to such departures, but is not fully flexible in that the form of the resulting ROC curve is restricted to be binormal. We propose a smooth non-parametric ROC curve derived from kernel density estimates of the two test result distributions. We obtain pointwise standard errors for the estimated TPR at each FPR and use them to construct pointwise confidence intervals for the ROC curve, using a logit transformation. We also obtain standard errors for the estimated TPR and FPR at given thresholds and use them to construct confidence rectangles in (FPR, TPR)-space that correspond to a specific threshold. We adapt existing methods for the unsmoothed non-parametric ROC curve to obtain the area under the ROC curve and its standard error; we also give partial areas and corresponding standard errors. We have created a FORTRAN algorithm ‘ROC-&-ROL’, available upon request. We compare ROC curves and areas obtained by applying our methods and its competitors to two data sets, one of which is fit well by parametric methods and LABROC4, the other of which is not. © 1997 John Wiley & Sons, Ltd.