Abstract 16207: Deep Learning-Based Natural Language Processing of Discharge Summaries for Automated Identification of Heart Failure With Reduced Ejection Fraction

Abstract

Background: Implementation of a national program to improve guideline-directed care in heart failure with reduced ejection fraction (HFrEF) has been limited by the lack of automated tools to measure care quality. A key challenge for this approach is to identify patients with HFrEF at hospital discharge, an opportunity to evaluate and improve care. Methods: We developed a novel deep-learning language model to detect patients with HFrEF from discharge summaries using a semi-supervised learning framework. Hospitalizations with heart failure (HF) at Yale New Haven Hospital between 2015-2019 were labeled as HFrEF if left ventricular ejection fraction was under 40% on antecedent echocardiography. The model was internally validated with model-based net reclassification improvement (NRI) assessed against chart-based diagnosis codes. We externally validated the model on discharge summaries from hospitalizations with HF at Northwestern Medicine, community hospitals of Yale Health, and MIMIC-III database, confirmed with chart abstraction. Results: A total of 13,251 notes from 5,392 individuals (age 73 ± 14 years, 48% female), including 2,487 (46.1%) patients with HFrEF were used for model development (train/validation/test: 70/15/15%). The model achieved an AUROC 0.97 and AUPRC 0.97 in detecting HFrEF on the test set. In external validation, the model had high performance in identifying HFrEF with AUROC 0.91-0.95 and AUPRC 0.91-0.96 among 19,242 discharge summaries at Northwestern Medicine, 139 notes from Yale community hospitals, and 146 notes from MIMIC-III. Model predictions corresponded to an NRI of 28 ± 7% compared to chart diagnosis (p< 0.001) and identified significant gaps in the use of guideline-directed therapies (Figure). Conclusions: We developed and validated an NLP model that automates the identification of HFrEF from clinical notes with high precision and accuracy, representing a key element to automate quality-of-care measurements in HFrEF.