Proposed solution to the FROC problem and an invitation to collaborate

Abstract

There is a need to develop observer performance methodologies that are not restricted to binary tasks, as is the widely used Receiver Operating Characteristic (ROC) method. Clinical tasks involving lesion localization do not fit the ROC paradigm. Alternative approaches namely Free-response ROC (FROC) and Localization ROC (LROC), have their own limitations. They neglect intra-image correlations, with consequent questionable statistical validity, the image scoring criterion is arbitrary and affects the results, and they do not account for the two sub-tasks, namely detection and localization, inherent to these experiments. The purpose of this work is to propose a new FROC model that deals with these issues, to propose a maximum likelihood method of estimating the model parameters, and to invite collaboration by others in implementing the solution. The model was used to generate simulated FROC data that illustrate the above-mentioned limitations. Measures of detection and localization performance are proposed that will allow FROC performance to be interpreted more quantitatively, and observer performance experiments to be conducted with greater statistical power.