Abstract
Purpose: Search involves identifying the locations of potential lesions. Classification involves determining if an identified region is a lesion. The area (AUC) under the ROC curve is affected by both search and classification performance. The purpose of this work is to suggest a method for quantifying the two distinct contributions. Methods: The search model for free‐response data involves sampling from noise and signal site unit variance normal distributions. It predicts ROC curves that extend from (0,0) to (x,y), where (x,y) is to the lower left of (1,1). The quantity (y‐x) is proposed as a measure of search performance. It equals the perpendicular distance from (x,y) to the chance diagonal multiplied by the square root of 2. The probability that a sample from the signal distribution is greater than a sample from the noise distribution is proposed as a measure of classification performance. AUC performance is the probability that the highest sample from an abnormal image is greater than the highest sample from a normal image. Search, classification and AUC performance were calculated for mammographer data from a FROC study in which 5 radiologists interpreted 96 normal and 89 disease‐present cases in breast tomosynthesis (BT) and digital mammography (DM) modalities. Results: Search performance ranged from zero to 0.65, classification performance ranged from 0.64 to 0.98 and AUC performance ranged from 0.68 to 0.88. AUC correlated positively with search performance while classification performance was inversely correlated with search performance. All three measures were greater for the BT modality. Conclusions: The method, which is applicable to FROC or ROC data, allows separation of physically distinct contributions to AUC, and consequent identification of the weak link. The preliminary results suggest that search is generally the weak link but that radiologists may learn to compensate for poor search performance with better classification performance.This work was supported by grants from the Department of Health and Human Services, National Institutes of Health, R01‐EB005243 and R01‐ EB008688.
Talk to us
Join us for a 30 min session where you can share your feedback and ask us any queries you have
Disclaimer: All third-party content on this website/platform is and will remain the property of their respective owners and is provided on "as is" basis without any warranties, express or implied. Use of third-party content does not indicate any affiliation, sponsorship with or endorsement by them. Any references to third-party content is to identify the corresponding services and shall be considered fair use under The CopyrightLaw.