Diagnostic accuracy of a commercially available, deep learning-based chest X-ray interpretation software for detecting culture-confirmed pulmonary tuberculosis

Abstract

Few evaluations of computer-aided detection (CAD) software for analyzing chest radiographs for tuberculosis have used mycobacterial culture as the reference standard. Using data from a prospective study of symptomatic adults and household contacts of persons with tuberculosis who were seeking care in Karachi, we evaluated the accuracy of LUNIT INSIGHT version 3.1.0.0 (LUNIT, South Korea) for detecting pulmonary tuberculosis in the triage use case. The reference standard was liquid culture. We estimated the diagnostic accuracy using three developer-recommended threshold scores for tuberculosis: 15, 30, and 45. A total 269 of 2190 (12%) participants had culture-confirmed pulmonary tuberculosis. LUNIT-reported abnormalities of nodule, consolidation, fibrosis, and pleural effusion were more common with culture-confirmed tuberculosis. At the tuberculosis threshold score of 30, sensitivity and specificity were, respectively, 87.7% [95% CI: 83.2-91.4%] and 64.3% [62.1-66.4%]. Sensitivity was similar at scores of 15, 88.1% [95% CI: 83.6-91.7%] and 45, 86.6% [82.0 - 90.5%]; and specificity was 57.9% [55.7-60.2%] and 69.9% [67.8-71.9%], respectively. Sensitivity was lower for smear-negative disease, and specificity was lower with increasing age, previous tuberculosis, and decreasing body mass index. Diabetes and tobacco smoking did not modify accuracy. In a population where most tuberculosis was smear-positive, LUNIT-reported radiographic abnormalities were associated with culture-confirmed disease. Manufacturer-recommended threshold scores had limited sensitivity.