<title>Differential receiver operating characteristic (DROC) method: rationale and results of recent experiments</title>

Abstract

The Differential Receiver Operating Characteristic (DROC) method has been recently proposed as a method of more sensitively determining which of two modalities has a higher Az value. This method is unlike the Receiver Operating Characteristic (ROC) method, which employs a single image interpretation strategy. The DROC experiment requires the reader to interpret pairs of images of the same patient, one from each modality termed A and B. This study reports on methodological improvements that we have made as well as experiments that we have conducted since this method was last proposed. The original B-modality image set consisted of 60 digitized normal mammograms. We simulated abnormal images from these original images by superimposing speck-like patterns resembling clustered microcalcifications. Applying a wavelet compression-decompression program to these two sets of images yielded the corresponding A-modality image sets. An image creation and display program was written with a user-friendly interface. Five readers interpreted the images in alternating ROC and DROC sessions. In the ROC sessions the diagnosis (abnormal/normal) and associated confidence level (0 - 100) was indicated for each image. In the DROC-sessions, two decisions and associated confidence levels were indicated for each image pair: the diagnosis (abnormal/normal) and a preference (A/B) for the modality that yielded the higher confidence level for the diagnosis decision. It was found that both DROC and ROC showed that Az (A) less than Az(B), demonstrating that the observers could readily detect the degradation introduced by the compression. In addition, the DROC critical ratio was larger than the corresponding ROC critical ratio for all observers. This confirmed the earlier published results, which used a noise-addition processing and a non-clinical simulation. The combined experiments continue to indicate that DROC has the potential advantage over ROC of increased sensitivity to image quality differences. Issues of bias and their effect on ROC and DROC readings are discussed and suggestions are made for further improvements to DROC methodology. The significance of the increased sensitivity potentially offered by DROC may transform future observer performance studies. Fewer cases and readers may be needed to conduct DROC studies with equivalent power to ROC studies. Thus, DROC would enable more convenient testing of imaging modalities, allowing design engineers to more quickly optimize imaging variables.© (1999) COPYRIGHT SPIE--The International Society for Optical Engineering. Downloading of the abstract is permitted for personal use only.