Receiver operating characteristic analysis and confidence–accuracy characteristic analysis in investigations of system variables and estimator variables that affect eyewitness memory

Abstract

Two graphical techniques, receiver operating characteristic (ROC) analysis and what might be termed “confidence–accuracy characteristic” (CAC) analysis, are important tools for investigating variables that affect the accuracy of eyewitness identifications (e.g., type of lineup, exposure duration, same-race vs. other-race identifications, etc.). CAC analysis (a close relative of calibration analysis) consists of simply plotting suspect identification accuracy for each level of confidence. Two parties interested in the results of such investigations include (1) legal policymakers (e.g., state legislators and police chiefs) and (2) triers of guilt and innocence (e.g., judges and jurors). Which type of analysis is the most relevant to which party? The answer is largely a matter of whether the variable in question is a system variable or an estimator variable. ROC analysis, which measures discriminability, is critical for understanding system variables that affect eyewitness accuracy (e.g., the best lineup procedures). Thus, policymakers should be particularly attuned to the results of ROC analysis when making decisions about those variables. CAC analysis, which directly measures the confidence–accuracy relationship for suspect IDs, is critical for understanding the effect of estimator variables on eyewitness accuracy (e.g., exposure duration). Thus, triers of guilt and innocence should be particularly attuned to the results of CAC analysis. The utility of both analyses to system and estimator variables is illustrated by examining both types of analyses on previously published experiments and new experiments.