Dementia prediction in the general population: External validation of 12 prognostic models in the AGES‐Reykjavik Study

Abstract

AbstractBackgroundMultivariable prognostic models in the general population predict individual risk of developing Alzheimer’s disease (AD) or all‐cause dementia, which can aid selection of high‐risk individuals for clinical trials and prevention. Development of new models flourished across the past decades. However, external validation for the majority of models is lacking. We aimed to evaluate external performance of previously published prediction models for AD or all‐cause dementia.MethodBased on multiple systematic literature reviews, we identified 12 eligible models for external validation in the AGES‐Reykjavik Study, a prospective population‐based cohort study (n=5343 without dementia at baseline; 76.6±5.7 years). Models were developed to predict either AD or all‐cause dementia up to 5, 6, or 10 years of follow‐up, and were based on logistic, Cox, or competing risk regression. Predictive performance was assessed with C‐statistics (discrimination) and calibration plots (observed vs. predicted probabilities).ResultDuring 45,021 and 40,917 person‐years of follow‐up, 1099 participants developed dementia and 492 participants developed AD, respectively. The mean observed risk of dementia was 5.1% [CI:4.5‐5.7] for 5 years, 10.1% [CI:9.2‐11.0] for 6 years, and 22.8% [CI:21.5‐24.1] for 10 years; the risk for AD was 2.0% [CI:1.6‐2.4], 5.6% [CI:4.9‐6.3], and 12.1% [11.0‐13.2], respectively. Table 1 presents the validated models and their C‐statistics. Models ranged from 2‐11 predictors; all models with a cognitive test obtained C‐statistics ≥0.75, all models without ≤0.73. Information to perform calibration was provided for 5 models, and all but one showed overestimation of predicted risk (Figure 1).ConclusionThe majority of models showed acceptable discrimination (C‐statistic >0.70), but poor calibration with systematic overestimation. These models may be acceptable for exclusion of dementia or AD (if the predicted risk is low, the observed risk is even lower), but lack the ability to accurately identify individuals at higher risk. The one model that calibrated well was developed with logistic regression, a technique that ignores observation times and censoring, which can introduce bias into the prediction process. Updating existing models or development and external validation of new models aimed at identifying high‐risk individuals using Cox regression or competing risk regression is therefore warranted.