Improving the performance of lung nodule classification by fusing structured and unstructured data

Abstract

We investigated the issue of improving the classification performance for pulmonary nodules by learning the fusion features of structured and unstructured data. Current strategies for lung nodule classification, such as radiomics methods and deep learning approaches, all share the flaw of only using the unstructured data of patients, which is always a collection of medical images (e.g., computed tomography (CT) scans, X-rays, and pathological sections), while ignoring the structured data (e.g., baseline demographics, clinical characteristics, and laboratory examinations). However, from a clinical perspective, all of this information is required for accurate patient diagnosis. Therefore, to exploit all patient information, we addressed a more difficult problem: jointly modeling the multimodal patient data. Two models are proposed to combine structured and unstructured data. One employs deep learning with a softmax classifier (the structured and unstructured data fusion neural network (SUDFNN)), and the other implements an extreme gradient boosting (XGBoost) classifier (the structured and unstructured data fusion XGBoost (SUDFX)). The annotated structured data in the extensible markup language (XML) file from the Lung Image Database Consortium and Image Database Resource Initiative (LIDC-IDRI) database and the CT scans from the LUng Nodule Analysis 2016 (LUNA16) dataset were used to validate our model. The results show that the performance of the model is significantly improved when introducing the structured data, regardless of the nodule cube size and which classifier is used. The rationale for the improvement with the addition of structured features is provided. The optimal accuracy, sensitivity, specificity, and area under the receiver operating characteristic (ROC) curve (AUC) values reached 0.936, 0.919, 0.956, and 0.971, respectively. Consequently, fusing structured and unstructured data can uncover more patient information and provide better decision support for the clinical diagnosis and treatment process, providing good application value and promotion prospects.