1000-Case Reader Study of Radiologists' Performance in Interpretation of Automated Breast Volume Scanner Images with a Computer-Aided Detection System

Abstract

The objective of our study was to assess, in a reader study, radiologists' performance in interpretation of automated breast volume scanner (ABVS) images with the aid of a computer-aided detection (CADe) system. Our study is a retrospective observer study with the purpose of investigating the effectiveness of using a CADe system as an aid for radiologists in interpretation of ABVS images. The multiple-reader, multiple-case study was designed to compare the diagnostic performance of radiologists with and without CADe. The study included 1000 cases selected from ABVS examinations in our institution in 2012. Among those cases were 206 malignant, 486 benign and 308 normal cases. The cancer cases were consecutive; the benign and normal cases were randomly selected. All malignant and benign cases were confirmed by biopsy or surgery, and normal cases were confirmed by 2-y follow-up. Reader performance was compared in terms of area under the receiver operating characteristic curve, sensitivity and specificity. Additionally, the reading time per case for each reader was recorded. Nine radiologists from our institution participated in the study. Three had more than 8 y of ultrasound experience and more than 4 y of ABVS experience (group A); 3 had more than 5 y of ultrasound experience (group B), and 3 had more than 1 y of ultrasound experience (group C). Both group B and group C had no ABVS experience. The CADe system used was the QVCAD System (QView Medical, Inc., Los Altos, CA, USA). It is designed to aid radiologists in searching for suspicious areas in ABVS images. CADe results are presented to the reader simultaneously with the ABVS images; that is, the radiologists read the ABVS images concurrently with the CADe results. The cases were randomly assigned for each reader into two equal-size groups, 1 and 2. Initially the readers read their group 1 cases with the aid of CADe and their group 2 cases without CADe. After a 1-mo washout period, they re-read their group 1 cases without CADe and their group 2 cases with CADe. The areas under the receiver operating characteristic curves of all readers were 0.784 for reading with CADe and 0.747 without CADe. Areas under the curves with and without CADe were 0.833 and 0.829 for group A, 0.757 and 0.696 for group B and 0.759 and 0.718 for group C. All differences in areas under the curve were statistically significant (p < 0.05), except that for group A. The average reading time was 9.3% (p < < 0.05) faster with CADe for all readers. In summary, CADe improves radiologist performance with respect to both accuracy and reading time for the detection of breast cancer using the ABVS, with the greater benefit for those inexperienced with ABVS.