Unstructured Electronic Health Record Research Articles

Identifying probable dementia in undiagnosed Black and White Americans using machine learning in Veterans Health Administration electronic health records

AbstractBackgroundThe application of machine learning (ML) tools in electronic health records (EHRs) can help reduce the underdiagnosis of dementia, but models that are not designed to reflect minority population may perpetuate that underdiagnosis.MethodTo address the underdiagnosis of dementia in both Black Americans (BAs) and white Americans (WAs), we sought to develop and validate ML models that assign race‐specific risk scores. These scores were used to identify undiagnosed dementia in BA and WA Veterans in EHRs. More specifically, risk scores were generated separately for BAs (n = 10K) and WAs (n = 10K) in training samples of cases and controls by performing ML, equivalence mapping, topic modeling, and a support vector‐machine (SVM) in structured and unstructured EHR data. Scores were validated via blinded manual chart reviews (n = 1.2K) of controls from a separate sample (n = 20K). AUCs and negative and positive predictive values (NPVs and PPVs) were calculated to evaluate the models.ResultAmong Veterans with undiagnosed dementia who underwent chart review, those diagnosed by expert clinician reviewers with possible/probable dementia had higher ADRD ML risk scores compared to those diagnosed with no dementia (0.45 [0.38] vs. ‐0.02 [0.51] for BAs, and 0.38 [0.41] vs. ‐0.02 [0.47] for WAs; (see figure 1).Of the 1,200 Veterans who underwent chart review, 15.3% (n = 92) of BAs and 9.5% (n = 57) of WAs were identified with possible/probable dementia. However, adjusting for oversampling of Veterans with higher scores, the adjusted estimated prevalence was 4.1% for BA Veterans and 3.6% for WA Veterans. There was a strong positive relationship between risk scores and the prevalence of undiagnosed dementia (Figure 2). As anticipated, for Veterans with scores below the 90th percentile, the percentages of undiagnosed dementia were low: 3.9% for BA and 2.9% for WA Veterans. Among Veterans with scores above the 90th percentile, we found that a higher percentage of BA Veterans had undiagnosed dementia than WA Veterans: 25.6% vs. 15.3%.ConclusionOur findings suggest that race‐specific ML models can assist in the identification of undiagnosed dementia, particularly in BAs. Future studies should investigate implementing EHR‐based risk scores in clinics that serve both BA and WA Veterans.

Enhanced Cardiovascular Risk Assessment Incorporating Natural Language Processing and Qrisk-3 Score Determination in Patients with Essential Thrombocythaemia

Introduction: Fundamental to management of patients with essential thrombocythaemia (ET) is assessment, and reduction of thrombotic risk. We present a machine learning approach to summarise patient electronic health records (EHR) to determine prevalence of cardiovascular comorbidities and risk factors. We then review use of the QRISK-3 score to assess cardiovascular risk. Methods: We used a natural language processing (NLP) pipeline to identify mentions of hypertension (HTN), hypercholesterolaemia (HC), diabetes mellitus (DM), smoking (SM), unspecified thrombosis (VTE), deep vein thrombosis (DVT), pulmonary embolism (PE), portal vein thrombosis (PVT), myocardial infarction (MI) and stroke (CVA) in EHR. CogStack is an information retrieval and extraction architecture incorporating structured and unstructured EHR components. Data extracted from CogStack was processed by a medical concept annotation toolkit (MedCAT). MedCAT was used to disambiguate and capture synonyms and acronyms for Systematized Nomenclature of Medicine-Clinical Terms (SNOMED-CT) concepts. Using deep learning MedCAT determined linguistic and grammatical context such as negation, subject, and temporality. The base MedCAT model was trained in an unsupervised manner on >18 million EHR documents and this was further fine-tuned through 500 clinician annotated haematology documents. MedCAT mapped mentions of relevant concepts to respective SNOMED-CT codes and total counts were aggregated and grouped by individual patient. Manual validations were performed and an optimizer was applied to convert counts to a binary state by applying a threshold, above which a patient's condition was inferred to be present (Fig. i). QRISK-3 is an advanced validated score incorporating age, ethnicity, body mass index and other cardiovascular risk factors to determine 10-year cardiovascular risk in people aged 25-84. Results: 12905 documents from 560 ET patients were reviewed (median 20 per patient, IQR 8-34). In the manual validation dataset (n=120), MedCAT achieved excellent real-world F1 scores (model accuracy) for most concepts (HTN 0.91, HC 0.81, DM 1.0, VTE 0.73, CVA 0.87 and MI 0.67). Using a threshold of >2 mentions to define a positive population; HTN was identified in 21.3% (119) of patients, DM in 4.6% (26), MI in 3.6% (20), CVA in 7.7% (43), VTE in 8% (45), DVT in 1.4% (8), PE in 1.8% (10), PVT in 1.3% (7) and positive smoking status in 6.6% (37). HC was identified in 9.6% (54) using a threshold >1. 52% (28) of patients with HC and 69.2% (18) of those with DM also had HTN. Obesity was not identified in any patients using this approach. Patients with a diagnosis of HTN were more likely to have CVA than those without (15:104 vs 28:413, p=0.03). Patients with HTN were also more likely to have VTE (13:106 vs 19:422, p=0.01). Of patients with CVA/MI; 58.1% (25) /55% (11) had this event pre or at diagnosis and 30.2% (13)/ 10% (2) while receiving cytoreductive therapy. QRISK-3 analysis was performed in 32 patients with prior thrombosis and baseline criteria to evaluate predictive value; then 137 patients classified as low or intermediate (LIM) risk and not receiving cytoreductive therapy. Mean QRISK-3 was 8 in the thrombosis group, validating its relevance, and 2.5 (p<0.0001, Fig. ii) in the LIM cohort. Using the recognised QRISK-3 score threshold of >7.5 to define a high-risk population, 5.1% (7) patients from the LIM group were reclassified as high-risk due to additional comorbidities relevant to QRISK-3 including HTN in 8% (11), migraine 7.3% (10), DM 2.2% (3), severe mental illness 2.9% (4) and antipsychotic medication 0.7% (1). Discussion: We describe a novel approach to cardiovascular risk assessment in patients with ET, incorporating machine learning, allowing large volume data analysis, and detailed risk assessment using QRISK-3 scoring. We provide a rare ‘real-world’ report on the prevalence of comorbidities in this group, confirming increased CVA and VTE in patients with HTN. A previous report of 891 patients with ET showed prevalence of 5% for CVA, 2% for MI and 4% for VTE, suggesting that detection rate using our approach is within acceptable limits (Carobbio et al., Blood, 2011). Finally, as a novel finding, we show that QRISK-3 scoring is predictive of increased thrombotic risk and identifies a small group of patients who should be considered high-risk and may benefit from cytoreductive therapy, that are not detected using standard approaches.

Clinical Prediction Models for Hospital-Induced Delirium Using Structured and Unstructured Electronic Health Record Data: Protocol for a Development and Validation Study.

Hospital-induced delirium is one of the most common and costly iatrogenic conditions, and its incidence is predicted to increase as the population of the United States ages. An academic and clinical interdisciplinary systems approach is needed to reduce the frequency and impact of hospital-induced delirium. The long-term goal of our research is to enhance the safety of hospitalized older adults by reducing iatrogenic conditions through an effective learning health system. In this study, we will develop models for predicting hospital-induced delirium. In order to accomplish this objective, we will create a computable phenotype for our outcome (hospital-induced delirium), design an expert-based traditional logistic regression model, leverage machine learning techniques to generate a model using structured data, and use machine learning and natural language processing to produce an integrated model with components from both structured data and text data. This study will explore text-based data, such as nursing notes, to improve the predictive capability of prognostic models for hospital-induced delirium. By using supervised and unsupervised text mining in addition to structured data, we will examine multiple types of information in electronic health record data to predict medical-surgical patient risk of developing delirium. Development and validation will be compliant to the Transparent Reporting of a multivariable prediction model for Individual Prognosis Or Diagnosis (TRIPOD) statement. Work on this project will take place through March 2024. For this study, we will use data from approximately 332,230 encounters that occurred between January 2012 to May 2021. Findings from this project will be disseminated at scientific conferences and in peer-reviewed journals. Success in this study will yield a durable, high-performing research-data infrastructure that will process, extract, and analyze clinical text data in near real time. This model has the potential to be integrated into the electronic health record and provide point-of-care decision support to prevent harm and improve quality of care. DERR1-10.2196/48521.

A cross-modal clinical prediction system for intensive care unit patient outcome

Electronic Health Records (EHRs) are becoming an encyclopedia of clinical concepts, encompassing diagnoses, medications and clinical descriptions to describe patients’ health conditions. The abundance of EHR data presents unprecedented opportunities for leveraging machine learning in clinical predictive tasks. However, the complex process of acquiring healthcare data in some settings like Intensive Care Unit (ICU) gives rise to various challenges in mining EHR, such as modality diversity, heterogeneity, data silo and data quality problems, etc. In this paper, to overcome the data heterogeneity and data quality issues, we propose a Code-Text cross-modal Contrastive Learning (CTCL) framework specifically designed for predicting ICU patient outcomes. This framework leverages both structured (multi-view medical codes) and unstructured (clinical notes) EHR data to extract patient representations. To capture the complicated intra and inter interactions among medical codes, CTCL employs a multi-view graph convolution network to extract code encodings and a cross-view contrastive loss to regularize the model parameters. To enhance patient representations across different modalities, a cross-modal contrastive learning module is introduced. Experimental evaluations on the eICU-CRD database demonstrate that CTCL outperforms state-of-the-art competitors on the ICU mortality prediction task and in-hospital readmission prediction task. Furthermore, ablation studies and discussions demonstrate the effectiveness of each module of the framework.

Trustworthy Data and AI Environments for Clinical Prediction: Application to Crisis-Risk in People with Depression.

Depression is a common mental health condition that often occurs in association with other chronic illnesses, and varies considerably in severity. Electronic Health Records (EHRs) contain rich information about a patient's medical history and can be used to train, test and maintain predictive models to support and improve patient care. This work evaluated the feasibility of implementing an environment for predicting mental health crisis among people living with depression based on both structured and unstructured EHRs. A large EHR from a mental health provider, Mersey Care, was pseudonymised and ingested into the Natural Language Processing (NLP) platform CogStack, allowing text content in binary clinical notes to be extracted. All unstructured clinical notes and summaries were semantically annotated by MedCAT and BioYODIE NLP services. Cases of crisis in patients with depression were then identified. Random forest models, gradient boosting trees, and Long Short-Term Memory (LSTM) networks, with varying feature arrangement, were trained to predict the occurrence of crisis. The results showed that all the prediction models can use a combination of structured and unstructured EHR information to predict crisis in patients with depression with good and useful accuracy. The LSTM network that was trained on a modified dataset with only 1000 most-important features from the random forest model with temporality showed the best performance with a mean AUC of 0.901 and a standard deviation of 0.006 using a training dataset and a mean AUC of 0.810 and 0.01 using a hold-out test dataset. Comparing the results from the technical evaluation with the views of psychiatrists shows that there are now opportunities to refine and integrate such prediction models into pragmatic point-of-care clinical decision support tools for supporting mental healthcare delivery.

Forms or Free-Text? Measuring Advance Care Planning Activity Using Electronic Health Records.

Advance care planning (ACP) discussions seek to guide future serious illness care. These discussions may be recorded in the electronic health record by documentation in clinical notes, structured forms and directives, and physician orders. Yet, most studies of ACP prevalence have only examined structured electronic health record elements and ignored data existing in notes. We sought to investigate the relative comprehensiveness and accuracy of ACP documentation from structured and unstructured electronic health record data sources. We evaluated structured and unstructured ACP documentation present in the electronic health records of 435 patients with cancer drawn from three separate healthcare systems. We extracted structured ACP documentation by manually annotating written documents and forms scanned into the electronic health record. We coded unstructured ACP documentation using a rule-based natural language processing software that identified ACP keywords within clinical notes and was subsequently reviewed for accuracy. The unstructured approach identified more instances of ACP documentation (238, 54.7% of patients) than the structured ACP approach (187, 42.9% of patients). Additionally, 16.6% of all patients with structured ACP documentation only had documents that were judged as misclassified, incomplete, blank, unavailable, or a duplicate of a previously entered erroneous document. ACP documents scanned into electronic health records represent a limited view of ACP activity. Research and measures of clinical practice with ACP should incorporate information from unstructured data.

Combining clinical notes with structured electronic health records enhances the prediction of mental health crises

An automatic prediction of mental health crises can improve caseload prioritization and enable preventative interventions, improving patient outcomes and reducing costs. We combine structured electronic health records (EHRs) with clinical notes from 59,750 de-identified patients to predict the risk of mental health crisis relapse within the next 28days. The results suggest that an ensemble machine learning model that relies on structured EHRs and clinical notes when available, and relying solely on structured data when the notes are unavailable, offers superior performance over models trained with either of the two data streams alone. Furthermore, the study provides key takeaways related to the required amount of clinical notes to add value in predictive analytics. This study sheds light on the untapped potential of clinical notes in the prediction of mental health crises and highlights the importance of choosing an appropriate machine learning method to combine structured and unstructured EHRs.

Identifying Probable Dementia in Undiagnosed Black and White Americans Using Machine Learning in Veterans Health Administration Electronic Health Records

The application of natural language processing and machine learning (ML) in electronic health records (EHRs) may help reduce dementia underdiagnosis, but models that are not designed to reflect minority populations may instead perpetuate underdiagnosis. To improve the identification of undiagnosed dementia, particularly in Black Americans (BAs), we developed support vector machine (SVM) ML models to assign dementia risk scores based on features identified in unstructured EHR data (via latent Dirichlet allocation and stable topic extraction in n = 1 M notes) and structured EHR data. We hypothesized that separate models would show differentiation between racial groups, so the models were fit separately for BAs (n = 5 K with dementia ICD codes, n = 5 K without) and White Americans (WAs; n = 5 K with codes, n = 5 K without). To validate our method, scores were generated for separate samples of BAs (n = 10 K) and WAs (n = 10 K) without dementia codes, and the EHRs of 1.2 K of these patients were reviewed by dementia experts. All subjects were age 65+ and drawn from the VA, which meant that the samples were disproportionately male. A strong positive relationship was observed between SVM-generated risk scores and undiagnosed dementia. BAs were more likely than WAs to have undiagnosed dementia per chart review, both overall (15.3% vs. 9.5%) and among Veterans with >90th percentile cutoff scores (25.6% vs. 15.3%). With chart reviews as the reference standard and varied cutoff scores, the BA model performed slightly better than the WA model (AUC = 0.86 with negative predictive value [NPV] = 0.98, positive predictive value [PPV] = 0.26, sensitivity = 0.61, specificity = 0.92 and accuracy = 0.91 at >90th percentile cutoff vs. AUC = 0.77 with NPV = 0.98, PPV = 0.15, sensitivity = 0.43, specificity = 0.91 and accuracy = 0.89 at >90th). Our findings suggest that race-specific ML models can help identify BAs who may have undiagnosed dementia. Future studies should examine model generalizability in settings with more females and test whether incorporating these models into clinical settings increases the referral of undiagnosed BAs to specialists.

Development and application of pharmacological statin-associated muscle symptoms phenotyping algorithms using structured and unstructured electronic health records data.

Statins are widely prescribed cholesterol-lowering medications in the United States, but their clinical benefits can be diminished by statin-associated muscle symptoms (SAMS), leading to discontinuation. In this study, we aimed to develop and validate a pharmacological SAMS clinical phenotyping algorithm using electronic health records (EHRs) data from Minnesota Fairview. We retrieved structured and unstructured EHR data of statin users and manually ascertained a gold standard set of SAMS cases and controls using the published SAMS-Clinical Index tool from clinical notes in 200 patients. We developed machine learning algorithms and rule-based algorithms that incorporated various criteria, including ICD codes, statin allergy, creatine kinase elevation, and keyword mentions in clinical notes. We applied the best-performing algorithm to the statin cohort to identify SAMS. We identified 16 889 patients who started statins in the Fairview EHR system from 2010 to 2020. The combined rule-based (CRB) algorithm, which utilized both clinical notes and structured data criteria, achieved similar performance compared to machine learning algorithms with a precision of 0.85, recall of 0.71, and F1 score of 0.77 against the gold standard set. Applying the CRB algorithm to the statin cohort, we identified the pharmacological SAMS prevalence to be 1.9% and selective risk factors which included female gender, coronary artery disease, hypothyroidism, and use of immunosuppressants or fibrates. Our study developed and validated a simple pharmacological SAMS phenotyping algorithm that can be used to create SAMS case/control cohort to enable further analysis which can lead to the development of a SAMS risk prediction model.

Development and external validation of multimodal postoperative acute kidney injury risk machine learning models.

To develop and externally validate machine learning models using structured and unstructured electronic health record data to predict postoperative acute kidney injury (AKI) across inpatient settings. Data for adult postoperative admissions to the Loyola University Medical Center (2009-2017) were used for model development and admissions to the University of Wisconsin-Madison (2009-2020) were used for validation. Structured features included demographics, vital signs, laboratory results, and nurse-documented scores. Unstructured text from clinical notes were converted into concept unique identifiers (CUIs) using the clinical Text Analysis and Knowledge Extraction System. The primary outcome was the development of Kidney Disease Improvement Global Outcomes stage 2 AKI within 7 days after leaving the operating room. We derived unimodal extreme gradient boosting machines (XGBoost) and elastic net logistic regression (GLMNET) models using structured-only data and multimodal models combining structured data with CUI features. Model comparison was performed using the receiver operating characteristic curve (AUROC), with Delong's test for statistical differences. The study cohort included 138 389 adult patient admissions (mean [SD] age 58 [16] years; 11 506 [8%] African-American; and 70 826 [51%] female) across the 2 sites. Of those, 2959 (2.1%) developed stage 2 AKI or higher. Across all data types, XGBoost outperformed GLMNET (mean AUROC 0.81 [95% confidence interval (CI), 0.80-0.82] vs 0.78 [95% CI, 0.77-0.79]). The multimodal XGBoost model incorporating CUIs parameterized as term frequency-inverse document frequency (TF-IDF) showed the highest discrimination performance (AUROC 0.82 [95% CI, 0.81-0.83]) over unimodal models (AUROC 0.79 [95% CI, 0.78-0.80]). A multimodality approach with structured data and TF-IDF weighting of CUIs increased model performance over structured data-only models. These findings highlight the predictive power of CUIs when merged with structured data for clinical prediction models, which may improve the detection of postoperative AKI.

Developing Clinical Risk Prediction Models for Worsening Heart Failure Events and Death by Left Ventricular Ejection Fraction.

Background There is a need to develop electronic health record-based predictive models for worsening heart failure (WHF) events across clinical settings and across the spectrum of left ventricular ejection fraction (LVEF). Methods and Results We studied adults with heart failure (HF) from 2011 to 2019 within an integrated health care delivery system. WHF encounters were ascertained using natural language processing and structured data. We conducted boosted decision tree ensemble models to predict 1-year hospitalizations, emergency department visits/observation stays, and outpatient encounters for WHF and all-cause death within each LVEF category: HF with reduced ejection fraction (EF) (LVEF <40%), HF with mildly reduced EF (LVEF 40%-49%), and HF with preserved EF (LVEF ≥50%). Model discrimination was evaluated using area under the curve and calibration using mean squared error. We identified 338 426 adults with HF: 61 045 (18.0%) had HF with reduced EF, 49 618 (14.7%) had HF with mildly reduced EF, and 227 763 (67.3%) had HF with preserved EF. The 1-year risks of any WHF event and death were, respectively, 22.3% and 13.0% for HF with reduced EF, 17.0% and 10.1% for HF with mildly reduced EF, and 16.3% and 10.3% for HF with preserved EF. The WHF model displayed an area under the curve of 0.76 and mean squared error of 0.13, whereas the model for death displayed an area under the curve of 0.83 and mean squared error of 0.076. Performance and predictors were similar across WHF encounter types and LVEF categories. Conclusions We developed risk prediction models for 1-year WHF events and death across the LVEF spectrum using structured and unstructured electronic health record data and observed no substantial differences in model performance or predictors except for death, despite differences in underlying HF cause.

Approach to machine learning for extraction of real-world data variables from electronic health records.

Background: As artificial intelligence (AI) continues to advance with breakthroughs in natural language processing (NLP) and machine learning (ML), such as the development of models like OpenAI's ChatGPT, new opportunities are emerging for efficient curation of electronic health records (EHR) into real-world data (RWD) for evidence generation in oncology. Our objective is to describe the research and development of industry methods to promote transparency and explainability. Methods: We applied NLP with ML techniques to train, validate, and test the extraction of information from unstructured documents (e.g., clinician notes, radiology reports, lab reports, etc.) to output a set of structured variables required for RWD analysis. This research used a nationwide electronic health record (EHR)-derived database. Models were selected based on performance. Variables curated with an approach using ML extraction are those where the value is determined solely based on an ML model (i.e. not confirmed by abstraction), which identifies key information from visit notes and documents. These models do not predict future events or infer missing information. Results: We developed an approach using NLP and ML for extraction of clinically meaningful information from unstructured EHR documents and found high performance of output variables compared with variables curated by manually abstracted data. These extraction methods resulted in research-ready variables including initial cancer diagnosis with date, advanced/metastatic diagnosis with date, disease stage, histology, smoking status, surgery status with date, biomarker test results with dates, and oral treatments with dates. Conclusion: NLP and ML enable the extraction of retrospective clinical data in EHR with speed and scalability to help researchers learn from the experience of every person with cancer.

Identification of opioid use disorder using electronic health records: Beyond diagnostic codes

BackgroundWe used structured and unstructured electronic health record (EHR) data to develop and validate an approach to identify moderate/severe opioid use disorder (OUD) that includes individuals without prescription opioid use or chronic pain, an underrepresented population. MethodsUsing electronic diagnosis grouper text from EHRs of ~1 million patients (2012–2020), we created indicators of OUD—with “tiers” indicating OUD likelihood—combined with OUD medication (MOUD) orders. We developed six sub-algorithms with varying criteria (multiple vs single MOUD orders, multiple vs single tier 1 indicators, tier 2 indicators, tier 3 and 4 indicators). Positive predictive values (PPVs) were calculated based on chart review to determine OUD status and severity. We compared demographic and clinical characteristics of cases identified by the sub-algorithms. ResultsIn total, 14,852 patients met criteria for one of the sub-algorithms. Five sub-algorithms had PPVs ≥0.90 for any severity OUD; four had PPVs ≥0.90 for moderate/severe OUD. Demographic and clinical characteristics differed substantially between groups. Of identified OUD cases, 31.3% had no past opioid analgesic orders, 79.7% lacked evidence of chronic prescription opioid use, and 43.5% lacked a chronic pain diagnosis. DiscussionIncorporating unstructured data with MOUD orders yielded an approach that adequately identified moderate/severe OUD, identified unique demographic and clinical sub-groups, and included individuals without prescription opioid use or chronic pain, whose OUD may stem from illicit opioids. Findings show that incorporating unstructured data strengthens EHR algorithms for identifying OUD and suggests approaches limited to populations with prescription opioid use or chronic pain exclude many individuals with OUD.

Retrospective study of propionic acidemia using natural language processing in Mayo Clinic electronic health record data

BackgroundPropionic acidemia (PA) is a rare autosomal recessive organic acidemia that classically presents within the first days of life with a metabolic crisis or via newborn screening and is confirmed with laboratory tests. Limited data exist on the natural history of patients with PA describing presentation, treatments, and clinical outcomes. ObjectiveTo retrospectively describe the natural history of patients with PA in a clinical setting from a real-world database using both structured and unstructured electronic health record (EHR) data using novel data extraction techniques in a unique care setting. Design/MethodsThis retrospective study used EHR data to identify patients with PA seen at the Mayo Clinic. Unstructured clinical text (medical notes, pathology reports) were analyzed using augmented curation natural language processing models to enhance analysis of data extracted by structured data fields (International Classification of Diseases 9th or 10th revision [ICD-9/−10] codes, Current Procedural Terminology [CPT] codes, and medication orders). De-identified health records were also manually reviewed by clinical scientists to ensure data accuracy and completeness. The index date was defined as the patient's date of PA diagnosis at the Mayo Clinic. Results were reported as aggregate descriptive statistics relative to patients' index dates. Complications, therapeutic interventions, laboratory tests, procedures, and hospitalization encounters related to PA were described at and within 6 months of the patient's index date, and from medical history available before the index date. ResultsIn total, 13 patients with PA were identified, with visits occurring from 1998 to 2022. Age at diagnosis ranged from birth to 3 years; age at initial evaluation at the Mayo Clinic ranged from 3 days to 28 years. The mean number of Mayo Clinic outpatient visits was 31 (median duration of care, 2 years). PA-related complications were documented in 85% of patients and included nutritional difficulties (46%), metabolic decompensation events (MDEs; 38%), neurologic abnormalities (38%), and cardiomyopathy (7%). One pair of affected siblings had mild symptoms and no complications or MDEs. All 5 patients with a history of MDEs presented with developmental delays. Among patients with MDEs, the mean frequency of outpatient clinical care visits was 10 per year, and 3 patients required inpatient hospitalization (mean duration, 16 days). The incidence of severe complications was higher among patients with MDEs than those without MDEs. Of the patients with MDEs, 2 experienced crises while receiving treatment at the Mayo Clinic, with 9 total MDEs occurring between the 2 patients. Symptoms at presentation included hyperammonemia (78%), fever and/or decreased nutritional intake (67%), hyperglycemia/hypoglycemia (56%), intercurrent upper respiratory infection and/or lethargy (44%), constipation (33%), altered mental status (33%), and cough (33%). ConclusionsThis study highlights the range and frequency of clinical outcomes experienced by patients with PA and demonstrates the clinical burden of MDEs.

The role of natural language processing techniques versus conventional methods to gain ICI safety insights from unstructured EHR data.

136 Background: Studies using International Classification of Diseases (ICD) codes have aimed to characterize the safety profile of cancer therapies in the Real-World Data setting, but this approach has limitations including an inability to illustrate patient journey and offer insights into drug-Adverse Event (AE) causality. Using data from a cohort of ~9000 patients treated with immune checkpoint inhibitors (ICI) at Mayo Clinic, we recently demonstrated the feasibility of applying Augmented Curation (AC), a natural language processing (NLP)-based innovation, on unstructured electronic health record (EHR) data to detect and characterize immune related AEs (irAEs). While the ability to identify and extract clinical details from EHRs has been previously demonstrated, our approach focuses on applying these techniques to identify implied textual causality of drug-event pairs. This study aims to build upon our prior findings and compare the accuracy, speed, and resource savings enabled by AC analysis versus manual review of patient notes. Methods: We compared the prevalence of adrenal insufficiency, colitis, and hypophysitis (A.C.H) among ICI patients at Mayo Clinic using ICD codes and AC which leverages SciBERT to perform scientific NLP tasks. AC models trained to identify drug-adverse event relationships were used to assess irAEs and subsequent treatments and demonstrated strong performance with F1 scores of 0.85. Manual curation of relevant clinical data, i.e. medications and adverse events, was employed to create a “gold standard” A.C.H. cohort. Time required for manual review compared to AC was documented for overall comparison. Corticosteroid/immunosuppressant use and ICI discontinuations were used as proxies of severity. Results: In our cohort, 540 unique patients experienced at least one of the A.C.H. irAEs. A.C.H patients were found to receive corticosteroids for their respective irAE 79% of the time and discontinued the ICI due to the irAE 7.7% of the time. 6.1% of Colitis patients were treated with a 2L immunosuppressant. Similar to the previous study, AC showed higher sensitivity, PPV and NPV compared to ICD codes alone within the A.C.H. cohort. Sentence extraction and entity classification using AC models on all 9,000-patients took ~10 minutes, whereas manual review of this cohort would take roughly 9 weeks (~10 minutes per patient). Conclusions: The results from this study reinforce that AC models are a valuable tool to comprehensively extract information and to impute relationships between entities, such as therapies and AEs. The AC models demonstrated better accuracy at a much greater speed compared to manual review of patient EHRs. Leveraging AC models provides the ability to manipulate unstructured clinical text married to structured EHR data at scale while minimizing need for manual curation.

Enriching Real-world Data with Social Determinants of Health for Health Outcomes and Health Equity: Successes, Challenges, and Opportunities.

To summarize the recent methods and applications that leverage real-world data such as electronic health records (EHRs) with social determinants of health (SDoH) for public and population health and health equity and identify successes, challenges, and possible solutions. In this opinion review, grounded on a social-ecological-model-based conceptual framework, we surveyed data sources and recent informatics approaches that enable leveraging SDoH along with real-world data to support public health and clinical health applications including helping design public health intervention, enhancing risk stratification, and enabling the prediction of unmet social needs. Besides summarizing data sources, we identified gaps in capturing SDoH data in existing EHR systems and opportunities to leverage informatics approaches to collect SDoH information either from structured and unstructured EHR data or through linking with public surveys and environmental data. We also surveyed recently developed ontologies for standardizing SDoH information and approaches that incorporate SDoH for disease risk stratification, public health crisis prediction, and development of tailored interventions. To enable effective public health and clinical applications using real-world data with SDoH, it is necessary to develop both non-technical solutions involving incentives, policies, and training as well as technical solutions such as novel social risk management tools that are integrated into clinical workflow. Ultimately, SDoH-powered social risk management, disease risk prediction, and development of SDoH tailored interventions for disease prevention and management have the potential to improve population health, reduce disparities, and improve health equity.

Retrospective comparison of traditional and artificial intelligence-based heart failure phenotyping in a US health system to enable real-world evidence

ObjectiveQuantitatively evaluate the quality of data underlying real-world evidence (RWE) in heart failure (HF).DesignRetrospective comparison of accuracy in identifying patients with HF and phenotypic information was made using traditional (ie,...

Using artificial intelligence to identify patients with migraine and associated symptoms and conditions within electronic health records

BackgroundReal-world evidence (RWE)—based on information obtained from sources such as electronic health records (EHRs), claims and billing databases, product and disease registries, and personal devices and health applications—is increasingly used to support healthcare decision making. There is variability in the collection of EHR data, which includes “structured data” in predefined fields (e.g., problem list, open claims, medication list, etc.) and “unstructured data” as free text or narrative. Healthcare providers are likely to provide more complete information as free text, but extracting meaning from these fields requires newer technologies and a rigorous methodology to generate higher-quality evidence. Herein, an approach to identify concepts associated with the presence and progression of migraine was developed and validated using the complete patient record in EHR data, including both the structured and unstructured portions.Methods“Traditional RWE” approaches (i.e., capture from structured EHR fields and extraction using structured queries) and “Advanced RWE” approaches (i.e., capture from unstructured EHR data and processing by artificial intelligence [AI] technology, including natural language processing and AI-based inference) were evaluated against a manual chart abstraction reference standard for data collected from a tertiary care setting. The primary endpoint was recall; differences were compared using chi square.ResultsCompared with manual chart abstraction, recall for migraine and headache were 66.6% and 29.6%, respectively, for Traditional RWE, and 96.8% and 92.9% for Advanced RWE; differences were statistically significant (absolute differences, 30.2% and 63.3%; P < 0.001). Recall of 6 migraine-associated symptoms favored Advanced RWE over Traditional RWE to a greater extent (absolute differences, 71.5–88.8%; P < 0.001). The difference between traditional and advanced techniques for recall of migraine medications was less pronounced, approximately 80% for Traditional RWE and ≥ 98% for Advanced RWE (P < 0.001).ConclusionUnstructured EHR data, processed using AI technologies, provides a more credible approach to enable RWE in migraine than using structured EHR and claims data alone. An algorithm was developed that could be used to further study and validate the use of RWE to support diagnosis and management of patients with migraine.

Leveraging Natural Language Processing to Improve Electronic Health Record Suicide Risk Prediction for Veterans Health Administration Users

Background: Suicide risk prediction models frequently rely on structured electronic health record (EHR) data, including patient demographics and health care usage variables. Unstructured EHR data, such as clinical notes, may improve predictive accuracy by allowing access to detailed information that does not exist in structured data fields. To assess comparative benefits of including unstructured data, we developed a large case-control dataset matched on a state-of-the-art structured EHR suicide risk algorithm, utilized natural language processing (NLP) to derive a clinical note predictive model, and evaluated to what extent this model provided predictive accuracy over and above existing predictive thresholds.Methods: We developed a matched case-control sample of Veterans Health Administration (VHA) patients in 2017 and 2018. Each case (all patients that died by suicide in that interval, n = 4,584) was matched with 5 controls (patients who remained alive during treatment year) who shared the same suicide risk percentile. All sample EHR notes were selected and abstracted using NLP methods. We applied machine-learning classification algorithms to NLP output to develop predictive models. We calculated area under the curve (AUC) and suicide risk concentration to evaluate predictive accuracy overall and for high-risk patients.Results: The best performing NLP-derived models provided 19% overall additional predictive accuracy (AUC = 0.69; 95% CI, 0.67, 0.72) and 6-fold additional risk concentration for patients at the highest risk tier (top 0.1%), relative to the structured EHR model.Conclusions: The NLP-supplemented predictive models provided considerable benefit when compared to conventional structured EHR models. Results support future structured and unstructured EHR risk model integrations.

Machine learning outcome prediction using stress perfusion cardiac magnetic resonance and electronic health records

Abstract Funding Acknowledgements Type of funding sources: Public grant(s) – National budget only. Main funding source(s): Wellcome Trust. Background Stress perfusion cardiac magnetic resonance (SP-CMR) is a well validated and guidelines backed diagnostic test for non-invasive assessment of patients with known or suspected coronary artery disease (CAD)(1). The prognostic value of SP-CMR is enhanced by adding clinical information, however, most electronic health record (EHR) is stored in an unstructured format, therefore limiting access to the information stored when large datasets are concerned. Furthermore, current outcome prediction models rely on linear models, but recent literature has shown stronger prediction using non-linear models (2). Purpose We aimed to address these challenges by using artificial intelligence (AI) tools to extract unstructured data from SP-CMR database and improve outcome prediction. Methods SP-CMR cases from 2011 to 2021 were screened. Data extraction was performed using Cogstack (3), an information retrieval and extraction platform for unstructured data based on natural language processing (NLP). Data were analysed for survival using Log Normal curves. Multivariate analysis was performed using survival Cox model. Different machine learning models were trained to predict survival, and compared using area under the receiver operating characteristic curve (AUC) and the DeLong test. 80% of dataset was used for training and 20% for testing. P value of &amp;lt;0.05 was considered statistically significant. Results 4,188 cases were included in the analysis. Total number of events (deaths) was 252 (6%). Ischaemia on stress perfusion imaging or the presence of myocardial scar on late gadolinium enhancement were significantly associated with events (Logrank p&amp;lt;0.001). Independent predictors from Cox survival analysis used to train machine learning models were: age, chronic kidney disease, hypertension, male gender, smoking, heart failure, positive ischaemic myocardial scar, positive stress perfusion, and ventricular ejection fraction. Considering clinical parameters only, support vector machine survival prediction performed best (AUC 0.76, F1 score 0.24), followed by XGBoost, random forest and ensemble classifier. After adding CMR predictors (stress perfusion, ischaemic myocardial scar, and left ventricular ejection fraction), prediction power improved for all machine learning models, with support vector machine having the best performance (AUC 0.82, F1 score 0.30). After comparing AUCs, non-linear models were better than multilinear regression (p&amp;lt;0.001, Z score −19). Conclusion NLP-based data extraction enables immediate access to unstructured EHR and reveals plausible results, which can be used within an integrated pipeline for data analysis and prediction, and has advantages over structured data due to access to large datasets. Machine learning-based prediction is superior to conventional linear models due to the added ability to model non-linear relationships. CMR predictors improve the survival prediction, compared to clinical variables alone.