Present Limitations of Artificial Intelligence in the Emergency Setting - Performance Study of a Commercial, Computer-Aided Detection Algorithm for Pulmonary Embolism.

Abstract

Since artificial intelligence is transitioning from an experimental stage to clinical implementation, the aim of our study was to evaluate the performance of a commercial, computer-aided detection algorithm of computed tomography pulmonary angiograms regarding the presence of pulmonary embolism in the emergency room. This retrospective study includes all pulmonary computed tomography angiogram studies performed in a large emergency department over a period of 36months that were analyzed by two radiologists experienced in emergency radiology to set a reference standard. Original reports and computer-aided detection results were compared regarding the detection of lobar, segmental, and subsegmental pulmonary embolism. All computer-aided detection findings were analyzed concerning the underlying pathology. False-positive findings were correlated to the contrast-to-noise ratio. Expert reading revealed pulmonary embolism in 182 of 1229 patients (49 % men, 10-97 years) with a total of 504emboli. The computer-aided detection algorithm reported 3331 findings, including 258 (8 %) true-positive findings and 3073 (92 %) false-positive findings. Computer-aided detection analysis showed a sensitivity of 47 % (95 %CI: 33-61 %) on the lobar level and 50 % (95 %CI 43-56 %) on the subsegmental level. On average, there were 2.25 false-positive findings per study (median 2, range 0-25). There was no significant correlation between the number of false-positive findings and the contrast-to-noise ratio (Spearman's Rank Correlation Coefficient = 0.09). Soft tissue (61.0 %) and pulmonary veins (24.1 %) were the most common underlying reasons for false-positive findings. Applied to a population at a large emergency room, the tested commercial computer-aided detection algorithm faced relevant performance challenges that need to be addressed in future development projects. · Computed tomography pulmonary angiograms are frequently acquired in emergency radiology.. · Computer-aided detection algorithms (CADs) can support image analysis.. · CADs face challenges regarding false-positive and false-negative findings.. · Radiologists using CADs need to be aware of these limitations.. · Further software improvements are necessary ahead of implementation in the daily routine.. · Müller-Peltzer K, Kretzschmar L, Negrão de Figueiredo G et al. Present Limitations of Artificial Intelligence in the Emergency Setting - Performance Study of a Commercial, Computer-Aided Detection Algorithm for Pulmonary Embolism. Fortschr Röntgenstr 2021; 193: 1436 - 1444.