Abstract P341: A Flexible, Interpretable Approach to Recalibrate Risk Prediction Models: Adapting the Pooled Cohorts Equation to a Local New England Contemporary Population

Abstract

Listen

Translate article

Introduction: Atherosclerotic cardiovascular disease (ASCVD) prognostic models have well-documented poor generalizability in contemporary populations. Sustained, accurate performance of risk prediction models is challenging given demographic shifts, changes in practice patterns, and geographic variation. We propose an approach to tailor known risk prediction models for local settings while preserving known disease risk mechanisms. Here, as a case study, we recalibrate the Pooled Cohort Equations (PCE) in a contemporary (1997-2018), local electronic health records (EHR) cohort in New England. Methods: We conducted a retrospective cohort study using EHR from large Boston-area hospitals. 61,808 patients 45+ years old, without ASCVD in 1997-2006 and complete data to calculate PCE model were split into training (50%), validation (10%), and test sets (40%). We used XGBoost to recalibrate PCE model (XGB-PCE) using PCE risk score, age, race/ethnicity, and sex (Figure). We constrained the XGB-PCE model to preserve known physiologic trends i.e. increase in ASCVD risk with increase in total cholesterol and systolic blood pressure. Models’ discrimination and calibration were calculated in a set aside test set using the Harrell’s c-index and modified Nam D’Agostino χ2 test. Results: In the test set, XGB-PCE exhibited improved calibration across ASCVD risk categories (p=0.37) and race/ethnicity populations (p=0.73). In contrast, even after recalibration in the large, PCE was poorly calibrated across risk categories (p<0.001) and race/ethnicity populations (p<0.001). For model discrimination, both models produced similar c-index (Δc-index of ±.01). Conclusions: Our recalibration approach maintained comparable discriminative performance and demonstrated improved calibration relative to the PCE while preserving known disease risk mechanisms. Methods that are flexible and preserve disease mechanisms have the potential to locally tailor existing models to be better calibrated for contemporary populations.