Evaluating the performance of a deep learning-based computer-aided diagnosis (DL-CAD) system for detecting and characterizing lung nodules: Comparison with the performance of double reading by radiologists.

Abstract

BackgroundThe study was conducted to evaluate the performance of a state‐of‐the‐art commercial deep learning‐based computer‐aided diagnosis (DL‐CAD) system for detecting and characterizing pulmonary nodules.MethodsPulmonary nodules in 346 healthy subjects (male: female = 221:125, mean age 51 years) from a lung cancer screening program conducted from March to November 2017 were screened using a DL‐CAD system and double reading independently, and their performance in nodule detection and characterization were evaluated. An expert panel combined the results of the DL‐CAD system and double reading as the reference standard.ResultsThe DL‐CAD system showed a higher detection rate than double reading, regardless of nodule size (86.2% vs. 79.2%; P < 0.001): nodules ≥ 5 mm (96.5% vs. 88.0%; P = 0.008); nodules < 5 mm (84.3% vs. 77.5%; P < 0.001). However, the false positive rate (per computed tomography scan) of the DL‐CAD system (1.53, 529/346) was considerably higher than that of double reading (0.13, 44/346; P < 0.001). Regarding nodule characterization, the sensitivity and specificity of the DL‐CAD system for distinguishing solid nodules > 5 mm (90.3% and 100.0%, respectively) and ground‐glass nodules (100.0% and 96.1%, respectively) were close to that of double reading, but dropped to 55.5% and 93%, respectively, when discriminating part solid nodules.ConclusionOur DL‐CAD system detected significantly more nodules than double reading. In the future, false positive findings should be further reduced and characterization accuracy improved.