Fundamental ROC Analysis

Abstract

Receiver operating characteristic (ROC) analysis is accepted widely as the most complete way of quantifying and reporting accuracy in two-group classification tasks. Based in statistical decision theory [1], ROC methodology was developed initially for evaluation of detectability in radar [2, 3] but soon was extended to applications in psychology and psychophysics [4-€“8] and in other fields [9-€“11]. The use of ROC analysis in medical decision making, medical diagnosis and medical imaging was first proposed by Lusted [12-€“15], who pointed out that a diagnostician or radiologist can achieve different combinations of sensitivity and specificity by consciously or unconsciously changing the €œthreshold of abnormality or €“critical confidence level which is used to distinguish nominally positive test results (or images) from nominally negative outcomes, and that ROC analysis is ideally suited to the task of separating such “decision threshold” effects from inherent differences in diagnostic accuracy. Subsequently, ROC techniques have been employed in the evaluation of a broad variety of diagnostic procedures [16-21], especially in medical imaging [22-€“30]. The medical applications of ROC analysis have fostered a number of methodological innovations, particularly in ROC curve fitting, in elucidating distinctions among several sources of variation in ROC estimates, and in testing the statistical significance of differences between such estimates. This chapter surveys conventional ROC methodology from a broad perspective, both to acquaint the reader with the current state of the art and to guide the reader to other literature that provides greater conceptual and/ˆ•or methodological detail. Variants of ROC analysis that take localization into account [31-€“38] are described in a subsequent chapter, whereas relationships between ROC analysis and cost-benefit analysis [22, 39-€“43] as well as the use of ROC analysis in predicting and quantifying the gains in accuracy obtained from multiple readings of each image [44, 45] are discussed elsewhere.