Identifying probable dementia in undiagnosed Black and White Americans using machine learning in Veterans Health Administration electronic health records

Abstract

AbstractBackgroundThe application of machine learning (ML) tools in electronic health records (EHRs) can help reduce the underdiagnosis of dementia, but models that are not designed to reflect minority population may perpetuate that underdiagnosis.MethodTo address the underdiagnosis of dementia in both Black Americans (BAs) and white Americans (WAs), we sought to develop and validate ML models that assign race‐specific risk scores. These scores were used to identify undiagnosed dementia in BA and WA Veterans in EHRs. More specifically, risk scores were generated separately for BAs (n = 10K) and WAs (n = 10K) in training samples of cases and controls by performing ML, equivalence mapping, topic modeling, and a support vector‐machine (SVM) in structured and unstructured EHR data. Scores were validated via blinded manual chart reviews (n = 1.2K) of controls from a separate sample (n = 20K). AUCs and negative and positive predictive values (NPVs and PPVs) were calculated to evaluate the models.ResultAmong Veterans with undiagnosed dementia who underwent chart review, those diagnosed by expert clinician reviewers with possible/probable dementia had higher ADRD ML risk scores compared to those diagnosed with no dementia (0.45 [0.38] vs. ‐0.02 [0.51] for BAs, and 0.38 [0.41] vs. ‐0.02 [0.47] for WAs; (see figure 1).Of the 1,200 Veterans who underwent chart review, 15.3% (n = 92) of BAs and 9.5% (n = 57) of WAs were identified with possible/probable dementia. However, adjusting for oversampling of Veterans with higher scores, the adjusted estimated prevalence was 4.1% for BA Veterans and 3.6% for WA Veterans. There was a strong positive relationship between risk scores and the prevalence of undiagnosed dementia (Figure 2). As anticipated, for Veterans with scores below the 90th percentile, the percentages of undiagnosed dementia were low: 3.9% for BA and 2.9% for WA Veterans. Among Veterans with scores above the 90th percentile, we found that a higher percentage of BA Veterans had undiagnosed dementia than WA Veterans: 25.6% vs. 15.3%.ConclusionOur findings suggest that race‐specific ML models can assist in the identification of undiagnosed dementia, particularly in BAs. Future studies should investigate implementing EHR‐based risk scores in clinics that serve both BA and WA Veterans.