Is ROC analysis a tool that should replace probative analysis in studying lineups?

Abstract

Purpose – The purpose of this paper is to explain why ROC analysis is an inappropriate replacement for probative analysis in lineup research. Design/methodology/approach – Taking as the medical example comparing two methods to detect the presence of a malignant tumor (Mickes et al., 2012), and operationally defining ROC analysis: radiologists are shown the results from two methods. Their confidence judgments create a graph of correct identifications by mistaken ones. The author can compare the methods on radiologists’ ability to differentiate sick from healthy. Lineup researchers create two distinct lineups. In target-present lineups, witnesses differentiate between the target and the foils, not the target and the innocent suspect. In target-absent lineups, witnesses cannot even differentiate between innocent suspects and foils, having seen none. Findings – Eyewitness ROC curves are similar to probative analysis, but provide less useful information. Research limitations/implications – Researchers ware warned against using ROC when conducting lineup research. Originality/value – Preventing inappropriate use of ROC analysis.