Determining Roe and Metz model parameters for simulating multireader multicase confidence-of-disease rating data based on real-data or conjectured Obuchowski-Rockette parameter estimates.

Abstract

The most frequently used model for simulating MRMC data that emulate confidence-of-disease ratings from diagnostic imaging studies has been the Roe and Metz model, proposed in 1997. The RM model generates continuous confidence-of-diseases ratings based on an underlying equal-variance binormal model for each reader, with the separation between the normal and abnormal rating distributions varying across readers. A problem with the RM model is that the parameters are expressed in terms of the rating distributions, as opposed to the reader performance outcomes. Because MRMC analysis results are almost always expressed in terms of the reader performance outcomes, and not in terms of the rating data distributions, it has been difficult to assess how similar the simulated data are to MRMC data encountered in practice. To remedy this situation, recently Hillis (in 2018) derived formulas expressing parameters that describe the distribution of empirical AUC outcomes computed from RM simulated data as functions of the RM parameters. An examination of these values revealed several problems with the realism of the simulated data. This paper continues that work by providing the inverse mapping, i.e., by deriving an algorithm that expresses the RM parameters as functions of the AUC empirical distribution parameters. This result will enable the creation of a recalibrated RM model that more closely emulates real-data studies.