Unstructured Electronic Health Record Data Research Articles

Natural Language Processing Versus Diagnosis Code-Based Methods for Postherpetic Neuralgia Identification: Algorithm Development and Validation.

Diagnosis codes and prescription data are used in algorithms to identify postherpetic neuralgia (PHN), a debilitating complication of herpes zoster (HZ). Because of the questionable accuracy of codes and prescription data, manual chart review is sometimes used to identify PHN in electronic health records (EHRs), which can be costly and time-consuming. This study aims to develop and validate a natural language processing (NLP) algorithm for automatically identifying PHN from unstructured EHR data and to compare its performance with that of code-based methods. This retrospective study used EHR data from Kaiser Permanente Southern California, a large integrated health care system that serves over 4.8 million members. The source population included members aged ≥50 years who received an incident HZ diagnosis and accompanying antiviral prescription between 2018 and 2020 and had ≥1 encounter within 90-180 days of the incident HZ diagnosis. The study team manually reviewed the EHR and identified PHN cases. For NLP development and validation, 500 and 800 random samples from the source population were selected, respectively. The sensitivity, specificity, positive predictive value (PPV), negative predictive value (NPV), F-score, and Matthews correlation coefficient (MCC) of NLP and the code-based methods were evaluated using chart-reviewed results as the reference standard. The NLP algorithm identified PHN cases with a 90.9% sensitivity, 98.5% specificity, 82% PPV, and 99.3% NPV. The composite scores of the NLP algorithm were 0.89 (F-score) and 0.85 (MCC). The prevalences of PHN in the validation data were 6.9% (reference standard), 7.6% (NLP), and 5.4%-13.1% (code-based). The code-based methods achieved a 52.7%-61.8% sensitivity, 89.8%-98.4% specificity, 27.6%-72.1% PPV, and 96.3%-97.1% NPV. The F-scores and MCCs ranged between 0.45 and 0.59 and between 0.32 and 0.61, respectively. The automated NLP-based approach identified PHN cases from the EHR with good accuracy. This method could be useful in population-based PHN research.

Machine Learning Models for Pancreatic Cancer Risk Prediction Using Electronic Health Record Data-A Systematic Review and Assessment.

Accurate risk prediction can facilitate screening and early detection of pancreatic cancer (PC). We conducted a systematic review to critically evaluate effectiveness of machine learning (ML) and artificial intelligence (AI) techniques applied to electronic health records (EHR) for PC risk prediction. Ovid MEDLINE(R), Ovid EMBASE, Ovid Cochrane Central Register of Controlled Trials, Ovid Cochrane Database of Systematic Reviews, Scopus, and Web of Science were searched for articles that utilized ML/AI techniques to predict PC, published between January 1, 2012, and February 1, 2024. Study selection and data extraction were conducted by 2 independent reviewers. Critical appraisal and data extraction were performed using the CHecklist for critical Appraisal and data extraction for systematic Reviews of prediction Modelling Studies checklist. Risk of bias and applicability were examined using prediction model risk of bias assessment tool. Thirty studies including 169,149 PC cases were identified. Logistic regression was the most frequent modeling method. Twenty studies utilized a curated set of known PC risk predictors or those identified by clinical experts. ML model discrimination performance (C-index) ranged from 0.57 to 1.0. Missing data were underreported, and most studies did not implement explainable-AI techniques or report exclusion time intervals. AI/ML models for PC risk prediction using known risk factors perform reasonably well and may have near-term applications in identifying cohorts for targeted PC screening if validated in real-world data sets. The combined use of structured and unstructured EHR data using emerging AI models while incorporating explainable-AI techniques has the potential to identify novel PC risk factors, and this approach merits further study.

Reducing diagnostic delays in acute hepatic porphyria using health records data and machine learning.

Acute hepatic porphyria (AHP) is a group of rare but treatable conditions associated with diagnostic delays of 15 years on average. The advent of electronic health records (EHR) data and machine learning (ML) may improve the timely recognition of rare diseases like AHP. However, prediction models can be difficult to train given the limited case numbers, unstructured EHR data, and selection biases intrinsic to healthcare delivery. We sought to train and characterize models for identifying patients with AHP. This diagnostic study used structured and notes-based EHR data from 2 centers at the University of California, UCSF (2012-2022) and UCLA (2019-2022). The data were split into 2 cohorts (referral and diagnosis) and used to develop models that predict (1) who will be referred for testing of acute porphyria, among those who presented with abdominal pain (a cardinal symptom of AHP), and (2) who will test positive, among those referred. The referral cohort consisted of 747 patients referred for testing and 99849 contemporaneous patients who were not. The diagnosis cohort consisted of 72 confirmed AHP cases and 347 patients who tested negative. The case cohort was 81% female and 6-75 years old at the time of diagnosis. Candidate models used a range of architectures. Feature selection was semi-automated and incorporated publicly available data from knowledge graphs. Our primary outcome was the F-score on an outcome-stratified test set. The best center-specific referral models achieved an F-score of 86%-91%. The best diagnosis model achieved an F-score of 92%. To further test our model, we contacted 372 current patients who lack an AHP diagnosis but were predicted by our models as potentially having it (≥10% probability of referral, ≥50% of testing positive). However, we were only able to recruit 10 of these patients for biochemical testing, all of whom were negative. Nonetheless, post hoc evaluations suggested that these models could identify 71% of cases earlier than their diagnosis date, saving 1.2 years. ML can reduce diagnostic delays in AHP and other rare diseases. Robust recruitment strategies and multicenter coordination will be needed to validate these models before they can be deployed.

Develop and validate a computable phenotype for the identification of Alzheimer's disease patients using electronic health record data.

Alzheimer's disease (AD) is often misclassified in electronic health records (EHRs) when relying solely on diagnosis codes. This study aimed to develop a more accurate, computable phenotype (CP) for identifying AD patients using structured and unstructured EHR data. We used EHRs from the University of Florida Health (UFHealth) system and created rule-based CPs iteratively through manual chart reviews. The CPs were then validated using data from the University of Texas Health Science Center at Houston (UTHealth) and the University of Minnesota (UMN). Our best-performing CP was "patient has at least 2 AD diagnoses and AD-related keywords in AD encounters," with an F1-score of 0.817 at UF, 0.961 at UTHealth, and 0.623 at UMN, respectively. We developed and validated rule-based CPs for AD identification with good performance, which will be crucial for studies that aim to use real-world data like EHRs. Developed a computable phenotype (CP) to identify Alzheimer's disease (AD) patients using EHR data.Utilized both structured and unstructured EHR data to enhance CP accuracy.Achieved a high F1-score of 0.817 at UFHealth, and 0.961 and 0.623 at UTHealth and UMN.Validated the CP across different demographics, ensuring robustness and fairness.

Enhancing the National Cancer Database content using natural language processing and electronic health record data

BackgroundThe prevalence of missing data in the National Cancer Database (NCDB) has marked implications on clinical care and research. The objective of this study was to enhance the NCDB by decreasing rates of missingness and adding new variables using automated statistical methodology. MethodsOne health system’s NCDB data from 2011–2021 was linked to electronic health record (EHR). Variables with frequent missingness and new clinically significant variables not yet included in the NCDB including patient Eastern Cooperative Oncology Group (ECOG) score, specific chemotherapy regimen, American Society of Anesthesiologists Physical Status Classification (ASA class), and discrete surgical procedure were identified in structured and unstructured EHR data. After automated incorporation of structured data from EHR, a natural language processing tool incorporating rule-based algorithms was designed to further extract variables from unstructured notes. Rates of missingness were compared between the original NCDB and the enhanced dataset, and example multivariable models were run to assess for altered model performance with reduced missingness and the addition of new clinically significant variables (chemotherapy regimen). ResultsA total of 6050 patients with NCDB records were linked to their EHR data. Prior to enhancement, rates of missingness for key variables ranged from 2.0% to 5.3%. Following dataset enhancement, missingness was significantly reduced, with relative missingness being reduced between 31.9% to 68.0%. Of the new variables added, 1367 (22.6%) of 6050 patients gained ECOG score, and 1099 (57.8%) of 1901 who received chemotherapy gained their chemotherapy regimen. Of 2989 who underwent surgery, 979 (32.8%) gained their procedure name and 621 (20.8%) gained ASA class. Comparison of the multivariable models demonstrated significant differences between the original NCDB and the enhanced dataset. Specifically, when replacing the binary predictor for chemotherapy in the original NCDB data with discrete regimens, the effect of ethnicity diminished, and the effect of radiation became significant. DiscussionWe applied statistical methodology to reduce rates of missingness in existing variables and add new variables to enrich the NCDB. While further refinement is needed to decrease missingness in new variables, this automated methodology can replace or augment manual chart review and improve the ability of to use the NCDB to study unanswered questions leading to clinical advancements in oncology.

Natural language processing to identify and characterize spondyloarthritis in clinical practice

ObjectiveThis study aims to use a novel technology based on natural language processing (NLP) to extract clinical information from electronic health records (EHRs) to characterise the clinical profile of patients...

Natural language processing systems for extracting information from electronic health records about activities of daily living. A systematic review.

Natural language processing (NLP) can enhance research on activities of daily living (ADL) by extracting structured information from unstructured electronic health records (EHRs) notes. This review aims to give insight into the state-of-the-art, usability, and performance of NLP systems to extract information on ADL from EHRs. A systematic review was conducted based on searches in Pubmed, Embase, Cinahl, Web of Science, and Scopus. Studies published between 2017 and 2022 were selected based on predefined eligibility criteria. The review identified 22 studies. Most studies (65%) used NLP for classifying unstructured EHR data on 1 or 2 ADL. Deep learning, combined with a ruled-based method or machine learning, was the approach most commonly used. NLP systems varied widely in terms of the pre-processing and algorithms. Common performance evaluation methods were cross-validation and train/test datasets, with F1, precision, and sensitivity as the most frequently reported evaluation metrics. Most studies reported relativity high overall scores on the evaluation metrics. NLP systems are valuable for the extraction of unstructured EHR data on ADL. However, comparing the performance of NLP systems is difficult due to the diversity of the studies and challenges related to the dataset, including restricted access to EHR data, inadequate documentation, lack of granularity, and small datasets. This systematic review indicates that NLP is promising for deriving information on ADL from unstructured EHR notes. However, what the best-performing NLP system is, depends on characteristics of the dataset, research question, and type of ADL.

Performance and clinical utility of a new supervised machine-learning pipeline in detecting rare ciliopathy patients based on deep phenotyping from electronic health records and semantic similarity

BackgroundRare diseases affect approximately 400 million people worldwide. Many of them suffer from delayed diagnosis. Among them, NPHP1-related renal ciliopathies need to be diagnosed as early as possible as potential treatments have been recently investigated with promising results. Our objective was to develop a supervised machine learning pipeline for the detection of NPHP1 ciliopathy patients from a large number of nephrology patients using electronic health records (EHRs).Methods and resultsWe designed a pipeline combining a phenotyping module re-using unstructured EHR data, a semantic similarity module to address the phenotype dependence, a feature selection step to deal with high dimensionality, an undersampling step to address the class imbalance, and a classification step with multiple train-test split for the small number of rare cases. The pipeline was applied to thirty NPHP1 patients and 7231 controls and achieved good performances (sensitivity 86% with specificity 90%). A qualitative review of the EHRs of 40 misclassified controls showed that 25% had phenotypes belonging to the ciliopathy spectrum, which demonstrates the ability of our system to detect patients with similar conditions.ConclusionsOur pipeline reached very encouraging performance scores for pre-diagnosing ciliopathy patients. The identified patients could then undergo genetic testing. The same data-driven approach can be adapted to other rare diseases facing underdiagnosis challenges.

An Algorithm to Assess Guideline Concordance of Antibiotic Choice in Community-Acquired Pneumonia.

This study aimed to develop and evaluate an algorithm to reduce the chart review burden of improvement efforts by automatically labeling antibiotic selection as either guideline-concordant or -discordant based on electronic health record data for patients with community-acquired pneumonia (CAP). We developed a 3-part algorithm using structured and unstructured data to assess adherence to an institutional CAP clinical practice guideline. The algorithm was applied to retrospective data for patients seen with CAP from 2017 to 2019 at a tertiary children's hospital. Performance metrics included positive predictive value (precision), sensitivity (recall), and F1 score (harmonized mean), with macro-weighted averages. Two physician reviewers independently assigned "actual" labels based on manual chart review. Of 1345 patients with CAP, 893 were included in the training cohort and 452 in the validation cohort. Overall, the model correctly labeled 435 of 452 (96%) patients. Of the 286 patients who met guideline inclusion criteria, 193 (68%) were labeled as having received guideline-concordant antibiotics, 48 (17%) were labeled as likely in a scenario in which deviation from the clinical practice guideline was appropriate, and 45 (16%) were given the final label of "possibly discordant, needs review." The sensitivity was 0.96, the positive predictive value was 0.97, and the F1 was 0.96. An automated algorithm that uses structured and unstructured electronic health record data can accurately assess the guideline concordance of antibiotic selection for CAP. This tool has the potential to improve the efficiency of improvement efforts by reducing the manual chart review needed for quality measurement.

Unveiling the dynamics of antimicrobial utilization and resistance in a large hospital network over five years: Insights from health record data analysis.

Antimicrobial Resistance (AMR) presents a pressing public health challenge globally which has been compounded by the COVID-19 pandemic. Elucidation of the impact of the pandemic on AMR evolution using population-level data that integrates clinical, laboratory and prescription data remains lacking. Data was extracted from the centralized electronic platform which captures the health records of 60,551 patients with a confirmed infection across the network of public healthcare facilities in Dubai, United Arab Emirates. For all inpatients and outpatients diagnosed with bacterial infection between 01/01/2017 and 31/05/2022, structured and unstructured Electronic Health Record data, microbiological laboratory data including antibiogram, molecular typing and COVID-19 testing information as well as antibiotic prescribing data were extracted curated and linked. Various analytical methods, including time-series analysis, natural language processing (NLP) and unsupervised clustering algorithms, were employed to investigate the trends of antimicrobial usage and resistance over time, assess the impact of prescription practices on resistance rates, and explore the effects of COVID-19 on antimicrobial usage and resistance. Our findings identified a significant impact of COVID-19 on antimicrobial prescription practices, with short-term and long-lasting over-prescription of these drugs. Resistance to antimicrobials increased the odds ratio of all mortality to an average of 2.18 (95% CI: 1.87-2.49) for the most commonly prescribed antimicrobials. Moreover, the effects of antimicrobial prescription practices on resistance were observed within one week of initiation. Significant trends in antimicrobial resistance, exhibiting fluctuations for various drugs and organisms, with an overall increasing trend in resistance levels, particularly post-COVID-19 were identified. This study provides a population-level insight into the evolution of AMR in the context of COVID-19 pandemic. The findings emphasize the impact of COVID-19 on the AMR crisis, which remained evident even two years after the onset of the pandemic. This underscores the necessity for enhanced antimicrobial stewardship to address the evolution of AMR.

Developing a computable phenotype for glioblastoma.

Glioblastoma is the most common malignant brain tumor, and thus it is important to be able to identify patients with this diagnosis for population studies. However, this can be challenging as diagnostic codes are nonspecific. The aim of this study was to create a computable phenotype (CP) for glioblastoma multiforme (GBM) from structured and unstructured data to identify patients with this condition in a large electronic health record (EHR). We used the University of Florida (UF) Health Integrated Data Repository, a centralized clinical data warehouse that stores clinical and research data from various sources within the UF Health system, including the EHR system. We performed multiple iterations to refine the GBM-relevant diagnosis codes, procedure codes, medication codes, and keywords through manual chart review of patient data. We then evaluated the performances of various possible proposed CPs constructed from the relevant codes and keywords. We underwent six rounds of manual chart reviews to refine the CP elements. The final CP algorithm for identifying GBM patients was selected based on the best F1-score. Overall, the CP rule "if the patient had at least 1 relevant diagnosis code and at least 1 relevant keyword" demonstrated the highest F1-score using both structured and unstructured data. Thus, it was selected as the best-performing CP rule. We developed and validated a CP algorithm for identifying patients with GBM using both structured and unstructured EHR data from a large tertiary care center. The final algorithm achieved an F1-score of 0.817, indicating a high performance, which minimizes possible biases from misclassification errors.

Identifying probable dementia in undiagnosed Black and White Americans using machine learning in Veterans Health Administration electronic health records

AbstractBackgroundThe application of machine learning (ML) tools in electronic health records (EHRs) can help reduce the underdiagnosis of dementia, but models that are not designed to reflect minority population may perpetuate that underdiagnosis.MethodTo address the underdiagnosis of dementia in both Black Americans (BAs) and white Americans (WAs), we sought to develop and validate ML models that assign race‐specific risk scores. These scores were used to identify undiagnosed dementia in BA and WA Veterans in EHRs. More specifically, risk scores were generated separately for BAs (n = 10K) and WAs (n = 10K) in training samples of cases and controls by performing ML, equivalence mapping, topic modeling, and a support vector‐machine (SVM) in structured and unstructured EHR data. Scores were validated via blinded manual chart reviews (n = 1.2K) of controls from a separate sample (n = 20K). AUCs and negative and positive predictive values (NPVs and PPVs) were calculated to evaluate the models.ResultAmong Veterans with undiagnosed dementia who underwent chart review, those diagnosed by expert clinician reviewers with possible/probable dementia had higher ADRD ML risk scores compared to those diagnosed with no dementia (0.45 [0.38] vs. ‐0.02 [0.51] for BAs, and 0.38 [0.41] vs. ‐0.02 [0.47] for WAs; (see figure 1).Of the 1,200 Veterans who underwent chart review, 15.3% (n = 92) of BAs and 9.5% (n = 57) of WAs were identified with possible/probable dementia. However, adjusting for oversampling of Veterans with higher scores, the adjusted estimated prevalence was 4.1% for BA Veterans and 3.6% for WA Veterans. There was a strong positive relationship between risk scores and the prevalence of undiagnosed dementia (Figure 2). As anticipated, for Veterans with scores below the 90th percentile, the percentages of undiagnosed dementia were low: 3.9% for BA and 2.9% for WA Veterans. Among Veterans with scores above the 90th percentile, we found that a higher percentage of BA Veterans had undiagnosed dementia than WA Veterans: 25.6% vs. 15.3%.ConclusionOur findings suggest that race‐specific ML models can assist in the identification of undiagnosed dementia, particularly in BAs. Future studies should investigate implementing EHR‐based risk scores in clinics that serve both BA and WA Veterans.

A Rule‐Based Framework to Identify Severity of Dementia from Unstructured Electronic Health Record Data

AbstractBackgroundThe severity of Alzheimer’s disease and related dementias (ADRD) is mostly documented in unstructured textual data in electronic health records (EHR). This information is important for clinical decision‐making yet is often “hidden” in free text fields and then not as readily available as information in the structured fields for clinicians to act upon. This study assessed the feasibility and potential bias in using keywords and rules‐based matching for obtaining information about severity of ADRD from EHR.MethodWe used EHR data from a large academic healthcare system that included patients with a primary discharge diagnosis of ADRD based on ICD‐9/10 codes between 2014 and 2020. The severity of ADRD was determined using clinicians’ notes based on (1) scores from the Mini Mental State Examination and Montreal Cognitive Assessment, and (2) explicit terms for ADRD severity (e.g., “mild dementia”, “advanced AD”). A list of common ADRD symptoms, cognitive test names and diagnosis stage terms was compiled and iteratively refined based on prior literature and clinical expertise. We used the list together with rule‐based pattern matching to identify the context in which the word/phrase was mentioned. The algorithm was developed in python 3.8 using spaCy and pandas library. We assessed the prevalence of the documented ADRD severity and used logistic regression to examine whether the severity varies by patient characteristics.ResultA total of 9,115 patients with over 65k providers’ notes were evaluated. Overall, 16.85% (N = 1,536) of patients were documented with mild ADRD, 17.95% (N = 1,636) were documented with moderate or severe ADRD, and 65.20% (N = 5,942) did not have any documentation of the severity of their ADRD. Compared with patients with mild ADRD, those documented with more advanced ADRD were older, more likely to be female, black, and receive their diagnoses in a primary care or in‐hospital setting. Relative to patients with undocumented ADRD severity, those documented with ADRD severity had a similar distribution regarding sex, race, and rural/urban living environment.ConclusionThis study demonstrated the value of unstructured EHR data and the feasibility of using pattern matching algorithm in identifying severity of ADRD. Still, differences in the documentation may introduce bias in the algorithm.

Clinical Prediction Models for Hospital-Induced Delirium Using Structured and Unstructured Electronic Health Record Data: Protocol for a Development and Validation Study.

Hospital-induced delirium is one of the most common and costly iatrogenic conditions, and its incidence is predicted to increase as the population of the United States ages. An academic and clinical interdisciplinary systems approach is needed to reduce the frequency and impact of hospital-induced delirium. The long-term goal of our research is to enhance the safety of hospitalized older adults by reducing iatrogenic conditions through an effective learning health system. In this study, we will develop models for predicting hospital-induced delirium. In order to accomplish this objective, we will create a computable phenotype for our outcome (hospital-induced delirium), design an expert-based traditional logistic regression model, leverage machine learning techniques to generate a model using structured data, and use machine learning and natural language processing to produce an integrated model with components from both structured data and text data. This study will explore text-based data, such as nursing notes, to improve the predictive capability of prognostic models for hospital-induced delirium. By using supervised and unsupervised text mining in addition to structured data, we will examine multiple types of information in electronic health record data to predict medical-surgical patient risk of developing delirium. Development and validation will be compliant to the Transparent Reporting of a multivariable prediction model for Individual Prognosis Or Diagnosis (TRIPOD) statement. Work on this project will take place through March 2024. For this study, we will use data from approximately 332,230 encounters that occurred between January 2012 to May 2021. Findings from this project will be disseminated at scientific conferences and in peer-reviewed journals. Success in this study will yield a durable, high-performing research-data infrastructure that will process, extract, and analyze clinical text data in near real time. This model has the potential to be integrated into the electronic health record and provide point-of-care decision support to prevent harm and improve quality of care. DERR1-10.2196/48521.

A cross-modal clinical prediction system for intensive care unit patient outcome

Electronic Health Records (EHRs) are becoming an encyclopedia of clinical concepts, encompassing diagnoses, medications and clinical descriptions to describe patients’ health conditions. The abundance of EHR data presents unprecedented opportunities for leveraging machine learning in clinical predictive tasks. However, the complex process of acquiring healthcare data in some settings like Intensive Care Unit (ICU) gives rise to various challenges in mining EHR, such as modality diversity, heterogeneity, data silo and data quality problems, etc. In this paper, to overcome the data heterogeneity and data quality issues, we propose a Code-Text cross-modal Contrastive Learning (CTCL) framework specifically designed for predicting ICU patient outcomes. This framework leverages both structured (multi-view medical codes) and unstructured (clinical notes) EHR data to extract patient representations. To capture the complicated intra and inter interactions among medical codes, CTCL employs a multi-view graph convolution network to extract code encodings and a cross-view contrastive loss to regularize the model parameters. To enhance patient representations across different modalities, a cross-modal contrastive learning module is introduced. Experimental evaluations on the eICU-CRD database demonstrate that CTCL outperforms state-of-the-art competitors on the ICU mortality prediction task and in-hospital readmission prediction task. Furthermore, ablation studies and discussions demonstrate the effectiveness of each module of the framework.

Forms or Free-Text? Measuring Advance Care Planning Activity Using Electronic Health Records.

Advance care planning (ACP) discussions seek to guide future serious illness care. These discussions may be recorded in the electronic health record by documentation in clinical notes, structured forms and directives, and physician orders. Yet, most studies of ACP prevalence have only examined structured electronic health record elements and ignored data existing in notes. We sought to investigate the relative comprehensiveness and accuracy of ACP documentation from structured and unstructured electronic health record data sources. We evaluated structured and unstructured ACP documentation present in the electronic health records of 435 patients with cancer drawn from three separate healthcare systems. We extracted structured ACP documentation by manually annotating written documents and forms scanned into the electronic health record. We coded unstructured ACP documentation using a rule-based natural language processing software that identified ACP keywords within clinical notes and was subsequently reviewed for accuracy. The unstructured approach identified more instances of ACP documentation (238, 54.7% of patients) than the structured ACP approach (187, 42.9% of patients). Additionally, 16.6% of all patients with structured ACP documentation only had documents that were judged as misclassified, incomplete, blank, unavailable, or a duplicate of a previously entered erroneous document. ACP documents scanned into electronic health records represent a limited view of ACP activity. Research and measures of clinical practice with ACP should incorporate information from unstructured data.

Identifying Probable Dementia in Undiagnosed Black and White Americans Using Machine Learning in Veterans Health Administration Electronic Health Records

The application of natural language processing and machine learning (ML) in electronic health records (EHRs) may help reduce dementia underdiagnosis, but models that are not designed to reflect minority populations may instead perpetuate underdiagnosis. To improve the identification of undiagnosed dementia, particularly in Black Americans (BAs), we developed support vector machine (SVM) ML models to assign dementia risk scores based on features identified in unstructured EHR data (via latent Dirichlet allocation and stable topic extraction in n = 1 M notes) and structured EHR data. We hypothesized that separate models would show differentiation between racial groups, so the models were fit separately for BAs (n = 5 K with dementia ICD codes, n = 5 K without) and White Americans (WAs; n = 5 K with codes, n = 5 K without). To validate our method, scores were generated for separate samples of BAs (n = 10 K) and WAs (n = 10 K) without dementia codes, and the EHRs of 1.2 K of these patients were reviewed by dementia experts. All subjects were age 65+ and drawn from the VA, which meant that the samples were disproportionately male. A strong positive relationship was observed between SVM-generated risk scores and undiagnosed dementia. BAs were more likely than WAs to have undiagnosed dementia per chart review, both overall (15.3% vs. 9.5%) and among Veterans with >90th percentile cutoff scores (25.6% vs. 15.3%). With chart reviews as the reference standard and varied cutoff scores, the BA model performed slightly better than the WA model (AUC = 0.86 with negative predictive value [NPV] = 0.98, positive predictive value [PPV] = 0.26, sensitivity = 0.61, specificity = 0.92 and accuracy = 0.91 at >90th percentile cutoff vs. AUC = 0.77 with NPV = 0.98, PPV = 0.15, sensitivity = 0.43, specificity = 0.91 and accuracy = 0.89 at >90th). Our findings suggest that race-specific ML models can help identify BAs who may have undiagnosed dementia. Future studies should examine model generalizability in settings with more females and test whether incorporating these models into clinical settings increases the referral of undiagnosed BAs to specialists.

Development and external validation of multimodal postoperative acute kidney injury risk machine learning models.

To develop and externally validate machine learning models using structured and unstructured electronic health record data to predict postoperative acute kidney injury (AKI) across inpatient settings. Data for adult postoperative admissions to the Loyola University Medical Center (2009-2017) were used for model development and admissions to the University of Wisconsin-Madison (2009-2020) were used for validation. Structured features included demographics, vital signs, laboratory results, and nurse-documented scores. Unstructured text from clinical notes were converted into concept unique identifiers (CUIs) using the clinical Text Analysis and Knowledge Extraction System. The primary outcome was the development of Kidney Disease Improvement Global Outcomes stage 2 AKI within 7 days after leaving the operating room. We derived unimodal extreme gradient boosting machines (XGBoost) and elastic net logistic regression (GLMNET) models using structured-only data and multimodal models combining structured data with CUI features. Model comparison was performed using the receiver operating characteristic curve (AUROC), with Delong's test for statistical differences. The study cohort included 138 389 adult patient admissions (mean [SD] age 58 [16] years; 11 506 [8%] African-American; and 70 826 [51%] female) across the 2 sites. Of those, 2959 (2.1%) developed stage 2 AKI or higher. Across all data types, XGBoost outperformed GLMNET (mean AUROC 0.81 [95% confidence interval (CI), 0.80-0.82] vs 0.78 [95% CI, 0.77-0.79]). The multimodal XGBoost model incorporating CUIs parameterized as term frequency-inverse document frequency (TF-IDF) showed the highest discrimination performance (AUROC 0.82 [95% CI, 0.81-0.83]) over unimodal models (AUROC 0.79 [95% CI, 0.78-0.80]). A multimodality approach with structured data and TF-IDF weighting of CUIs increased model performance over structured data-only models. These findings highlight the predictive power of CUIs when merged with structured data for clinical prediction models, which may improve the detection of postoperative AKI.

Development and application of pharmacological statin-associated muscle symptoms phenotyping algorithms using structured and unstructured electronic health records data.

Statins are widely prescribed cholesterol-lowering medications in the United States, but their clinical benefits can be diminished by statin-associated muscle symptoms (SAMS), leading to discontinuation. In this study, we aimed to develop and validate a pharmacological SAMS clinical phenotyping algorithm using electronic health records (EHRs) data from Minnesota Fairview. We retrieved structured and unstructured EHR data of statin users and manually ascertained a gold standard set of SAMS cases and controls using the published SAMS-Clinical Index tool from clinical notes in 200 patients. We developed machine learning algorithms and rule-based algorithms that incorporated various criteria, including ICD codes, statin allergy, creatine kinase elevation, and keyword mentions in clinical notes. We applied the best-performing algorithm to the statin cohort to identify SAMS. We identified 16 889 patients who started statins in the Fairview EHR system from 2010 to 2020. The combined rule-based (CRB) algorithm, which utilized both clinical notes and structured data criteria, achieved similar performance compared to machine learning algorithms with a precision of 0.85, recall of 0.71, and F1 score of 0.77 against the gold standard set. Applying the CRB algorithm to the statin cohort, we identified the pharmacological SAMS prevalence to be 1.9% and selective risk factors which included female gender, coronary artery disease, hypothyroidism, and use of immunosuppressants or fibrates. Our study developed and validated a simple pharmacological SAMS phenotyping algorithm that can be used to create SAMS case/control cohort to enable further analysis which can lead to the development of a SAMS risk prediction model.

Developing Clinical Risk Prediction Models for Worsening Heart Failure Events and Death by Left Ventricular Ejection Fraction.

Background There is a need to develop electronic health record-based predictive models for worsening heart failure (WHF) events across clinical settings and across the spectrum of left ventricular ejection fraction (LVEF). Methods and Results We studied adults with heart failure (HF) from 2011 to 2019 within an integrated health care delivery system. WHF encounters were ascertained using natural language processing and structured data. We conducted boosted decision tree ensemble models to predict 1-year hospitalizations, emergency department visits/observation stays, and outpatient encounters for WHF and all-cause death within each LVEF category: HF with reduced ejection fraction (EF) (LVEF <40%), HF with mildly reduced EF (LVEF 40%-49%), and HF with preserved EF (LVEF ≥50%). Model discrimination was evaluated using area under the curve and calibration using mean squared error. We identified 338 426 adults with HF: 61 045 (18.0%) had HF with reduced EF, 49 618 (14.7%) had HF with mildly reduced EF, and 227 763 (67.3%) had HF with preserved EF. The 1-year risks of any WHF event and death were, respectively, 22.3% and 13.0% for HF with reduced EF, 17.0% and 10.1% for HF with mildly reduced EF, and 16.3% and 10.3% for HF with preserved EF. The WHF model displayed an area under the curve of 0.76 and mean squared error of 0.13, whereas the model for death displayed an area under the curve of 0.83 and mean squared error of 0.076. Performance and predictors were similar across WHF encounter types and LVEF categories. Conclusions We developed risk prediction models for 1-year WHF events and death across the LVEF spectrum using structured and unstructured electronic health record data and observed no substantial differences in model performance or predictors except for death, despite differences in underlying HF cause.