Estimation of malignancy of pulmonary nodules at CT scans: Effect of computer-aided diagnosis on diagnostic performance of radiologists.

Abstract

To develop a computer-aided diagnosis (CAD) system for distinguishing malignant from benign pulmonary nodules on computed tomography (CT) scans, and to assess whether the diagnostic performance of radiologists with different experiences can be improved with the assistant of CAD. A total of 857 malignant nodules from 601 patients and 426 benign nodules from 278 patients were retrospectively collected from four hospitals. In this study, we exploited convolutional neural network in the framework of deep learning to classify whether a nodule was benign or malignant. A total of 745 malignant nodules and 370 benign nodules were used as the training data of our CAD system. The remaining 112 malignant nodules and 56 benign nodules were used as the test data. The participants were two senior chest radiologists, two secondary chest radiologists, and two junior radiology residents. The readers estimated the likelihood of malignancy of pulmonary nodules first without and then with CAD output. Receiver-operating characteristic (ROC) curve was used to evaluate readers' diagnostic performance. When a threshold level of 58% was used to estimate the likelihood of malignancy, the sensitivity, specificity, and diagnostic accuracy values of our CAD scheme alone were 93.8%, 83.9%, and 90.5%, respectively. For all six readers, the mean area under the ROC curve (Az ) values without and with CAD system were 0.913 and 0.938, respectively. For each reader, there is a large difference in Az values that assessed without and with CAD system. With CAD output, the readers made correct changes an average of 15.7 times and incorrect changes an average of 2 times. Our CAD system significantly improved the diagnostic performance of readers regardless of their experience levels for assessment of the likelihood of malignancy of pulmonary nodules.