Observer Study of a Prototype Clinical Decision Support System for Breast Cancer Diagnosis Using Dynamic Contrast-Enhanced MRI

Abstract

The purpose of this article is to evaluate the performance of radiologists using a prototype clinical decision support system to diagnose and manage patients with breast cancer based on dynamic contrast-enhanced MRI studies. The study was conducted with three breast radiologists and two breast imaging fellows who gave patient treatment recommendations and confidence ratings, both without and with computer aid. The computer aid presented similar cases from a retrieval database of 192 lesions (96 malignant and 96 benign) for a test set of 97 mass lesions (46 malignant and 51 benign). The performance of each observer was quantified by receiver operating characteristic analysis. The radiologists' confidence in their recommendations was analyzed with respect to the query case pathologic diagnosis, perceived usefulness of the similar cases, and the accuracy of the computer in retrieving cases of the correct diagnosis. The statistical significance in the performance measure differences was determined by using a two-tailed Student t test for paired data. For each observer, the area under the receiver operating characteristic curve did not change significantly with the use of the computer aid (from a mean of 0.8 to a mean of 0.8; p = 0.61). The average confidence of three of the five observers increased significantly with the computer aid (from 5.9 to 6.3 [p < 0.001], from 7.0 to 7.2 [p = 0.04], and from 4.4 to 5.4 [p < 0.001], respectively). The confidence change of the radiologists was more frequent and larger for malignant lesions where the computer was correct. However, for benign lesions, even when the computer was correct, the confidence of the radiologists did not necessarily change. The presentation of similar cases reinforced radiologists' confidence rating in the diagnosis of malignant lesions; however, it did not change their confidence rating for benign lesions or reduce the number of unnecessary biopsies in managing patients with breast cancer using dynamic contrast-enhanced MRI under the limited study conditions.