Screening test length for sequential testing with a standardized-patient examination: a receiver operating characteristic (ROC) analysis.

Abstract

The feasibility of using sequential testing (i.e., using a screening test) to reduce the length and expense of a performance-based examination with standardized-patient cases was demonstrated previously. In the present study, quantitative criteria rather than practical considerations were used to determine optimal values for the length of the screening test (i.e., number of cases) and the location of the screening pass-fail cutoff (i.e., its relation to the mean of the pass levels for the different cases). Data were derived from five classes of senior students at the Southern Illinois University School of Medicine, 1987-1991. Specifically, receiver operating characteristic (ROC) curves were plotted for screening tests of varying lengths, with the points on each ROC curve corresponding to different pass-fail cutoffs on the screening test. The results showed that good accuracy can be attained with a screening test that is only one-third the length of the full examination and that the cutoff for this screen should be set slightly above the mean of the case pass levels to maximize sensitivity and specificity. The authors conclude that their study demonstrates the value of an ROC analysis in evaluating the psychometric properties of a screening test in sequential testing.