Abstract 13797: Enabling Advanced Real-world Evidence in Heart Failure: A Pilot Study Defining Preferred Approaches to Electronic Health Record Data Use

Abstract

Introduction: Real world evidence (RWE) is increasingly used for regulatory and market access decision-making. In heart failure (HF), typical structured datasets have limitations in data accuracy and identifying relevant patient characteristics. Understanding which characteristics require enhancement from unstructured data and how to validly apply extraction methods will improve the definition of complex patient cohorts. Hypothesis: Augmenting structured with unstructured electronic health record (EHR) data may overcome challenges in accurately identifying relevant HF patient characteristics. Methods: Using EHR data from 4,288 primary care encounters, 20 clinical concepts were defined a priori by 3 HF experts. A reference standard was generated through chart abstraction, with each record reviewed by at least two annotators. Inter-rater reliability (IRR) was measured by Cohen’s kappa. EHR structured data (EHR-S) extracted with traditional query techniques and EHR unstructured (EHR-U) data extracted with artificial intelligence (AI) technologies were tested for accuracy against the reference standard. Results: In EHR-S, recall ranged from 0-95.1% and precision from 52.9-100%. In EHR-U data processed using AI, recall ranged from 80.4-99.7% and precision from 82.3-100%. Results demonstrated a 45.1% absolute difference and 98.1% relative increase in F1-score (Table). Reference standard IRR was 95.3%. Conclusions: RWE credibility and applicability relies on accurate identification of a patient cohort. This study suggests that readily available data sources may not accurately identify patient phenotypes in HF. Novel means of using AI with EHR-U may improve such efforts, particularly for conditions and symptoms. This approach offers a pathway for defining highly accurate HF cohorts that may be useful in studies with narrowly defined or complex phenotypes, such as those where inclusion and exclusion criteria are specific and outcomes require validity.