On the equivalence of classic ROC analysis and the loss-function model to set cut points in sequential testing.

Abstract

In an effort to reduce the cost of administration for objective structured clinical examinations (OSCEs), several authors have promoted the use of sequential testing in which all candidates take a short screening test and candidates who pass the screen are exempted from taking the full test. Traditionally, the determination of the optimally efficient cut point (passing score) for the screen has used ROC analysis to minimize false-positive and false-negative errors. Recently, Muijtjens et al. have questioned the appropriateness of the ROC method for these purposes and have promoted an alternative method that uses a "loss" formula. However, given certain theoretically derived conditions, it can be shown that the use of the loss formula is functionally identical to using ROC analysis, and the authors suggest that continued use of the ROC method is appropriate.