Automated Radiology Alert System for Pneumothorax Detection on Chest Radiographs Improves Efficiency and Diagnostic Performance.

Cheng-Yi Kao,Yi-Sheng Liu,Yi-Shan Tsai,Chiao-Yun Lin,Chien-Kuo Wang,Cheng-Chen Chao,Kang Sung,Po-Chang Chiang,Chia-Ming Chang,Li-Ting Huang,Han-Sheng Huang,Jung-Hsien Chiang,Bow Wang,Ting-Rong Chen,Hsing-Yu Lee

doi:10.3390/diagnostics11071182

Abstract

We aimed to set up an Automated Radiology Alert System (ARAS) for the detection of pneumothorax in chest radiographs by a deep learning model, and to compare its efficiency and diagnostic performance with the existing Manual Radiology Alert System (MRAS) at the tertiary medical center. This study retrospectively collected 1235 chest radiographs with pneumothorax labeling from 2013 to 2019, and 337 chest radiographs with negative findings in 2019 were separated into training and validation datasets for the deep learning model of ARAS. The efficiency before and after using the model was compared in terms of alert time and report time. During parallel running of the two systems from September to October 2020, chest radiographs prospectively acquired in the emergency department with age more than 6 years served as the testing dataset for comparison of diagnostic performance. The efficiency was improved after using the model, with mean alert time improving from 8.45 min to 0.69 min and the mean report time from 2.81 days to 1.59 days. The comparison of the diagnostic performance of both systems using 3739 chest radiographs acquired during parallel running showed that the ARAS was better than the MRAS as assessed in terms of sensitivity (recall), area under receiver operating characteristic curve, and F1 score (0.837 vs. 0.256, 0.914 vs. 0.628, and 0.754 vs. 0.407, respectively), but worse in terms of positive predictive value (PPV) (precision) (0.686 vs. 1.000). This study had successfully designed a deep learning model for pneumothorax detection on chest radiographs and set up an ARAS with improved efficiency and overall diagnostic performance.

Highlights

Chest radiography is the most common first-line imaging examination for the screening of multiple thoracic diseases, [1] including pneumothorax, a potentially life-threatening condition requiring clinical attention [2]
A retrospective search of the Picture Archiving and Communication System (PACS) and the Manual Radiology Alert System (MRAS) for chest radiographs with pneumothorax alerts from 1 January 2015 to 31 December 2019, and another retrospective search of the PACS
A deep learning model based on U-Net with balanced feature pyramid modules for pneumothorax detection was built using the training and validation datasets stated above

Summary

Introduction

Chest radiography is the most common first-line imaging examination for the screening of multiple thoracic diseases, [1] including pneumothorax, a potentially life-threatening condition requiring clinical attention [2]. An accuracy of 86% for detecting pneumothorax has been reached in a study based on the “ChestX-ray8” database of frontal chest radiographs with disease labels [5]. A deep-learning algorithm for multiple thoracic diseases on chest radiographs derived from single-center data, the accuracy of detecting pneumothorax reached 95% [6]. An automated detection system based on machine learning may screen these examinations for critical findings, notify the referring primary care physicians, and flag these examinations to be read by radiologists as early as possible [7]. Current studies regarding the application of machine learning in radiology have focused on the accuracy of their machine learning algorithms, but no study has revealed the time an automated detection system based on machine learning could save in the real-world setting

Objectives

Methods

Results

Discussion

Conclusion