Abstract
Background: Performance of deep-learning based automated detection (DLAD) algorithms in systematic screening for active pulmonary tuberculosis is unknown. We aimed to validate DLAD algorithm for detection of active pulmonary tuberculosis and any radiologicallyidentifiable relevant abnormality on chest radiographs (CRs) in this setting. Methods: We performed out-of-sample testing of a trained DLAD algorithm, using CRs from 19,686 asymptomatic individuals (male: 19,475, female: 211; mean ± standard deviation: 21.3 ± 1.9 years) as part of systematic screening for tuberculosis between January 2013 and July 2018. Area under the receiver operating characteristic curves (AUC) of DLAD for diagnosis of tuberculosis and any relevant abnormalities were measured. Accuracy measures including sensitivities, specificities, positive predictive values (PPVs), negative predictive values (NPVs) were calculated at predefined operating thresholds (high sensitivity threshold, 0·16; high specificity threshold, 0·46). Findings: All five CRs with active pulmonary tuberculosis were correctly classified as having abnormal findings by DLAD with specificities of 0·959 and 0·997, PPVs of 0·006 and 0·068, and NPVs of both 1·000 at high sensitivity and high specificity thresholds, respectively. With high specificity thresholds, DLAD showed comparable diagnostic measures for tuberculosis to the pooled radiologists (P values > 0·05). For the detection of any radiologically-identifiable relevant abnormality (n=28), DLAD showed AUC value of 0·967 (95% confidence interval, 0·938-0·996) with sensitivities of 0·821 and 0·679, specificities of 0·960 and 0·997, PPVs of 0·028 and 0·257, and NPVs of both 1·000 at high sensitivity and high specificity thresholds, respectively. Interpretation: In systematic screening for tuberculosis in a lowprevalence setting, DLAD algorithm demonstrated excellent diagnostic performance, comparable to the radiologists in the detection of active pulmonary tuberculosis. Funding Statement: This study was supported by the Seoul Research & Business Development Program (grant number: FI170002), and Lunit Inc. provided technical supports for this study. Declaration of Interests: There is a major research agreement between Seoul National University Hospital and Lunit Inc, in which roles of researchers and Lunit Inc. were described. However, the funder and Lunit Inc. did not have any role either in the study design; in the collection, analysis, and interpretation of the data; in the writing of the report; and in the decision to submit the article for publication. Researchers (J.H.L., E.J.H., W.Y.L., S.L., J.R.A.) who controlled, manipulated and analyzed data, did not have any conflict of interest. Three authors (J.M.G., H.K., C.M.P.) received research grants from Lunit Inc. for outside of this study. Ethics Approval Statement: This retrospective study was approved by the institutional review board of The Armed Forces Medical Command of Korea, and the requirement for informed consent was waived.
Published Version
Talk to us
Join us for a 30 min session where you can share your feedback and ask us any queries you have
Schedule a call