Prospective Evaluation of Artificial Intelligence Triage of Incidental Pulmonary Emboli on Contrast-Enhanced CT Examinations of the Chest or Abdomen.

Abstract

BACKGROUND. Artificial intelligence (AI) algorithms improved detection of incidental pulmonary embolism (IPE) on contrast-enhanced CT (CECT) examinations in retrospective studies; however, prospective validation studies are lacking. OBJECTIVE. The purpose of this study was to assess the effect on radiologists' real-world diagnostic performance and report turnaround times of a radiology department's clinical implementation of an AI triage system for detecting IPE on CECT examinations of the chest or abdomen. METHODS. This prospective single-center study included consecutive adult patients who underwent CECT of the chest or abdomen for reasons other than pulmonary embolism (PE) detection from May 12, 2021, to June 30, 2021 (phase 1), or from September 30, 2021, to December 4, 2021 (phase 2). Before phase 1, the radiology department installed a commercially available AI triage algorithm for IPE detection that automatically processed CT examinations and notified radiologists of positive results through an interactive floating widget. In phase 1, the widget was inactive, and radiologists interpreted examinations without AI assistance. In phase 2, the widget was activated, and radiologists interpreted examinations with AI assistance. A review process involving a panel of radiologists was implemented to establish the reference standard for the presence of IPE. Diagnostic performance and report turnaround times were compared using the Pearson chi-square test and Wilcoxon rank sum test, respectively. RESULTS. Phase 1 included 1467 examinations in 1434 patients (mean age, 53.8 ± 18.5 [SD] years; 753 men, 681 women); phase 2 included 3182 examinations in 2886 patients (mean age, 55.4 ± 18.2 years; 1520 men, 1366 women). The frequency of IPE was 1.4% (20/1467) in phase 1 and 1.6% (52/3182) in phase 2. Radiologists without AI, in comparison to radiologists with AI, showed significantly lower sensitivity (80.0% vs 96.2%, respectively; p = .03), without a significant difference in specificity (99.9% vs 99.9%, p = .58), for the detection of IPE. The mean report turnaround time for IPE-positive examinations was not significantly different between radiologists without AI and radiologists with AI (78.3 vs 74.6 minutes, p = .26). CONCLUSION. An AI triage system improved radiologists' sensitivity for IPE detection on CECT examinations of the chest or abdomen without significant change in report turnaround times. CLINICAL IMPACT. This prospective real-world study supports the use of AI assistance for maximizing IPE detection.