Diagnostic performance of natural language processing in detection of acute cancer VTE.

Abstract

e19062 Background: The annual incidence of venous thromboembolism (VTE) is roughly 50-fold increased post allogeneic hematopoietic cell transplant (HSCT). Timely detection of VTE events has a significant impact on morbidity and mortality from these events. Manual chart review is commonly used for detection of VTE events; however, this is labor intensive and often not feasible. The International Classification of Diseases, 10th revision (ICD-10) has attempted to codify VTE more easily but has high false positive and false negative rates. Natural language processing (NLP) algorithms have been designed to improve detection of VTE events in electronic medical record systems over the past several years. This study aims to compare the accuracy of our institution’s NLP algorithm to the ICD-10 in detecting extremity VTE/pulmonary embolism (PE) events in hematopoietic stem cell transplant (HSCT) patients. Methods: This study was a retrospective analysis of adult patients who had allogeneic HSCT at MD Anderson between 2016-2020. Patient charts were assessed by three different methods for acute VTE events within 100 days of HSCT (index date). ICD-10 codes and our institution’s NLP algorithm were used separately to identify these VTE events. Afterwards, manual chart review was performed for confirmation. The sensitivity, specificity, positive predictive value (PPV), and negative predictive value (NPV) were calculated for the ICD-10, NLP, and the NLP+ICD combination. Results: A total of 1,391 patient charts were analyzed in our study. There were 129 (9.3%) VTE events detected via manual chart review. Three methodologies were used for comparison: the NLP algorithm alone, the ICD code alone, and detection by NLP or ICD. The NLP algorithm alone had the best specificity (98.9%) and PPV (89.4%) of the three methods. The ICD-10 code had a statistically significant decrease in ability to detect acute VTE events compared to the NLP (p<0.001) (Table). Of the 19 VTE events not identified by NLP, all were found in radiology or vascular laboratory reports. Conclusions: Our study shows the overall superior performance of the NLP algorithm, as it had higher sensitivity, specificity, PPV, and NPV compared to the ICD-10, as well as a better specificity and PPV compared to the combination of both methods. Future refinement of the NLP algorithm and combination with other detection methodologies will allow for more accurate detection of VTE in large cohorts. [Table: see text]