<title>Comparison of free-response and ROC analyses in modality comparison studies involving lesion localization</title>

Abstract

It is possible that neglect of location information in lesion detection studies analyzed by the Receiver Operating Characteristic (ROC) method can compromise power. The Alternative Free-response ROC (AFROC) analysis considers the location information but its usage has been discouraged, since it neglects intra-image correlations. This study compared the statistical power of ROC and AFROC methodologies using simulations. A model including intra-image correlations was developed to describe the decision variable sampling and was used to simulate data for ROC and AFROC analysis. Five readers and 200 cases, half of which contained one signal, were simulated for each trial. Two hundred trials were run, equally split between the Null Hypothesis (NH) and the Alternative Hypothesis (AH). The ratings were analyzed by the Dorfman-Berbaum-Metz (DBM) method and the separation of the NH/AH distributions was calculated. It was found that the AFROC method yielded higher power than ROC. The separation of the NH and AH distributions were larger by a factor of 1.6, irrespective of the presence or absence of intra-image correlations. The effect of the incorrect localizations occurring in ROC analysis of localization data is believed to be the major reason for the enhanced power of the AFROC method, and greater use of AFROC methodology is warranted.