Structured Electronic Health Record Data Research Articles

Automated Identification of Postoperative Infections to Allow Prediction and Surveillance Based on Electronic Health Record Data: Scoping Review.

Postoperative infections remain a crucial challenge in health care, resulting in high morbidity, mortality, and costs. Accurate identification and labeling of patients with postoperative bacterial infections is crucial for developing prediction models, validating biomarkers, and implementing surveillance systems in clinical practice. This scoping review aimed to explore methods for identifying patients with postoperative infections using electronic health record (EHR) data to go beyond the reference standard of manual chart review. We performed a systematic search strategy across PubMed, Embase, Web of Science (Core Collection), the Cochrane Library, and Emcare (Ovid), targeting studies addressing the prediction and fully automated surveillance (ie, without manual check) of diverse bacterial infections in the postoperative setting. For prediction modeling studies, we assessed the labeling methods used, categorizing them as either manual or automated. We evaluated the different types of EHR data needed for the surveillance and labeling of postoperative infections, as well as the performance of fully automated surveillance systems compared with manual chart review. We identified 75 different methods and definitions used to identify patients with postoperative infections in studies published between 2003 and 2023. Manual labeling was the predominant method in prediction modeling research, 65% (49/75) of the identified methods use structured data, and 45% (34/75) use free text and clinical notes as one of their data sources. Fully automated surveillance systems should be used with caution because the reported positive predictive values are between 0.31 and 0.76. There is currently no evidence to support fully automated labeling and identification of patients with infections based solely on structured EHR data. Future research should focus on defining uniform definitions, as well as prioritizing the development of more scalable, automated methods for infection detection using structured EHR data.

Feasibility of using real-world data to emulate substance use disorder clinical trials: a cross-sectional study

IntroductionReal-world evidence is receiving considerable attention as a way to evaluate the efficacy and safety of medical products for substance use disorders (SUDs). However, the feasibility of using real-world data (RWD) to emulate clinical trials evaluating treatments for SUDs is uncertain. The aim of this study is to identify the number of clinical trials evaluating treatments for SUDs with reported results that could be feasibly emulated using observational data from contemporary insurance claims and/or electronic health record (EHR) data.MethodsIn this cross-sectional study, all phase 2–4 trials evaluating treatments for SUDs registered on ClinicalTrials.gov with reported results were identified. Each trial was evaluated to determine if the indications, interventions, at least 80% of eligibility criteria, comparators, and primary end points could be ascertained using contemporarily available administrative claims and/or structured EHR data.ResultsThere were 272 SUD trials on ClinicalTrials.gov with reported results. Of these, when examining feasibility using contemporarily available administrative claims and/or structured EHR data, 262 (96.3%) had indications that were ascertainable; 194 (71.3%) had interventions that were ascertainable; 21 (7.7%) had at least 80% of eligibility criteria that were ascertainable; 17 (6.3%) had active comparators that were ascertainable; and 61 (22.4%) had primary end points that were ascertainable. In total, there were no trials for which all 5 characteristics were ascertainable using contemporarily available administrative claims and/or structured EHR data. When considering placebo comparators as ascertainable, there were 6 (2.2%) trials that had all 5 key characteristics classified as ascertainable from contemporarily available administrative claims and/or structured EHR data.ConclusionsNo trials evaluating treatments for SUDs could be feasibly emulated using contemporarily available RWD, demonstrating a need for an increase in the resolution of data capture within a public health system to facilitate trial emulation.

External Validation of an Electronic Phenotyping Algorithm Detecting Attention to High Body Mass Index in Pediatric Primary Care.

The lack of feasible and meaningful measures of clinicians' behavior hinders efforts to assess and improve obesity management in pediatric primary care. In this study, we examined the external validity of a novel algorithm, previously validated in a single geographic region, using structured electronic health record (EHR) data to identify phenotypes of clinicians' attention to elevated body mass index (BMI) and weight-related comorbidities. We extracted structured EHR data for 300 randomly selected 6- to 12-year-old children with elevated BMI seen for well-child visits from June 2018 to May 2019 at pediatric primary care practices affiliated with Yale. Using diagnosis codes, laboratory orders, referrals, and medications adapted from the original algorithm, we categorized encounters as having evidence of attention to BMI only, weight-related comorbidities only, or both BMI and comorbidities. We evaluated the algorithm's sensitivity and specificity for detecting any attention to BMI and/or comorbidities using chart review as the reference standard. The adapted algorithm yielded a sensitivity of 79.2% and specificity of 94.0% for identifying any attention to high BMI/comorbidities in clinical documentation. Of 86 encounters labeled as "no attention" by the algorithm, 83% had evidence of attention in free-text components of the progress note. The likelihood of classification as "any attention" by both chart review and the algorithm varied by BMI category and by clinician type (p < 0.001). The electronic phenotyping algorithm had high specificity for detecting attention to high BMI and/or comorbidities in structured EHR inputs. The algorithm's performance may be improved by incorporating unstructured data from clinical notes.

Ascertaining accurate exposure to aspirin and other antithrombotic medications using structured electronic health record data

BackgroundAscertaining accurately the exposure to antithrombotic medications for both research and quality initiatives has been challenging due to a multitude of reasons: aspirin, the most commonly used antithrombotic, is available over the counter in the United States. Additionally, antithrombotic medications are frequently interrupted for bleeding and procedures. ObjectivesWe aimed to develop and validate an algorithm to capture accurately the longitudinal exposure to antithrombotic medications including aspirin using the electronic health record. MethodsWe used the Medical Inpatient Thrombosis and Hemostasis cohort, which consists of primary care patients at a university medical center followed for a median of 6.2 years. Exposure to antithrombotic medications was captured using the medication reconciliation data linked to each ambulatory encounter. We developed an algorithm that used the taking “yes” or “no” tab as well as start and stop dates to define the duration of exposure for each medication. Eighty charts were reviewed and compared with results of the algorithm for validation. We estimated the sensitivity, specificity, and positive and negative predictive values. ResultsThe algorithm was 97% (95% CI, 94%-100%) sensitive and 95% (95% CI, 90%-100%) specific in identifying exposure to any antithrombotic medication. This translated to a 93% (95% CI, 85%-100%) positive predictive value and 98% (95% CI, 96%-100%) negative predictive value. When looking at aspirin alone, the sensitivity and the positive predictive value were 95% (95% CI, 88%-100%) and 87% (95% CI, 71%-100%). ConclusionThis current algorithm provides a new and easily adaptable strategy to capture accurately exposure to aspirin and other antithrombotic medications.

Use of Electronic Health Record-Based Measures to Assess Quality of Care for Pediatric Agitation.

Acute agitation during pediatric mental health emergency department (ED) visits presents safety risks to patients and staff. We previously convened multidisciplinary stakeholders who prioritized 20 proposed quality measures for pediatric acute agitation management. Our objectives were to assess feasibility of evaluating performance on these quality measures using electronic health record (EHR) data and to examine performance variation across 3 EDs. At a children's hospital and 2 nonchildren's hospitals, we assessed feasibility of evaluating quality measures for pediatric acute agitation management using structured EHR data elements. We retrospectively evaluated measure performance during ED visits by children 5 to 17 years old who presented for a mental health condition, received medication for agitation, or received physical restraints from July 2020 to June 2021. Bivariate and multivariable regression were used to examine measure performance by patient characteristics and hospital. We identified 2785 mental health ED visits, 275 visits with medication given for agitation, and 35 visits with physical restraints. Performance was feasible to measure using EHR data for 10 measures. Nine measures varied by patient characteristics, including 4.87 times higher adjusted odds (95% confidence interval 1.28-18.54) of physical restraint use among children with versus without autism spectrum disorder. Four measures varied by hospital, with physical restraint use varying from 0.5% to 3.3% of mental health ED visits across hospitals. Quality of care for pediatric acute agitation management was feasible to evaluate using EHR-derived quality measures. Variation in performance across patient characteristics and hospitals highlights opportunities to improve care quality.

Prediction of Effectiveness and Toxicities of Immune Checkpoint Inhibitors Using Real-World Patient Data.

Although immune checkpoint inhibitors (ICIs) have improved outcomes in certain patients with cancer, they can also cause life-threatening immunotoxicities. Predicting immunotoxicity risks alongside response could provide a personalized risk-benefit profile, inform therapeutic decision making, and improve clinical trial cohort selection. We aimed to build a machine learning (ML) framework using routine electronic health record (EHR) data to predict hepatitis, colitis, pneumonitis, and 1-year overall survival. Real-world EHR data of more than 2,200 patients treated with ICI through December 31, 2018, were used to develop predictive models. Using a prediction time point of ICI initiation, a 1-year prediction time window was applied to create binary labels for the four outcomes for each patient. Feature engineering involved aggregating laboratory measurements over appropriate time windows (60-365 days). Patients were randomly partitioned into training (80%) and test (20%) sets. Random forest classifiers were developed using a rigorous model development framework. The patient cohort had a median age of 63 years and was 61.8% male. Patients predominantly had melanoma (37.8%), lung cancer (27.3%), or genitourinary cancer (16.4%). They were treated with PD-1 (60.4%), PD-L1 (9.0%), and CTLA-4 (19.7%) ICIs. Our models demonstrate reasonably strong performance, with AUCs of 0.739, 0.729, 0.755, and 0.752 for the pneumonitis, hepatitis, colitis, and 1-year overall survival models, respectively. Each model relies on an outcome-specific feature set, though some features are shared among models. To our knowledge, this is the first ML solution that assesses individual ICI risk-benefit profiles based predominantly on routine structured EHR data. As such, use of our ML solution will not require additional data collection or documentation in the clinic.

Development and validation of a rule-based algorithm to identify periodontal diagnosis using structured electronic health record data.

To develop and validate an automated electronic health record (EHR)-based algorithm to suggest a periodontal diagnosis based on the 2017 World Workshop on the Classification of Periodontal Diseases and Conditions. Using material published from the 2017 World Workshop, a tool was iteratively developed to suggest a periodontal diagnosis based on clinical data within the EHR. Pertinent clinical data included clinical attachment level (CAL), gingival margin to cemento-enamel junction distance, probing depth, furcation involvement (if present) and mobility. Chart reviews were conducted to confirm the algorithm's ability to accurately extract clinical data from the EHR, and then to test its ability to suggest an accurate diagnosis. Subsequently, refinements were made to address limitations of the data and specific clinical situations. Each refinement was evaluated through chart reviews by expert periodontists at the study sites. Three-hundred and twenty-three charts were manually reviewed, and a periodontal diagnosis (healthy, gingivitis or periodontitis including stage and grade) was made by expert periodontists for each case. After developing the initial version of the algorithm using the unmodified 2017 World Workshop criteria, accuracy was 71.8% for stage alone and 64.7% for stage and grade. Subsequently, 16 modifications to the algorithm were proposed and 14 were accepted. This refined version of the algorithm had 79.6% accuracy for stage alone and 68.8% for stage and grade together. Our findings suggest that a rule-based algorithm for suggesting a periodontal diagnosis using EHR data can be implemented with moderate accuracy in support of chairside clinical diagnostic decision making, especially for inexperienced clinicians. Grey-zone cases still exist, where clinical judgement will be required. Future applications of similar algorithms with improved performance will depend upon the quality (completeness/accuracy) of EHR data.

Bridging information gaps in menopause status classification through natural language processing.

To use natural language processing (NLP) of clinical notes to augment existing structured electronic health record (EHR) data for classification of a patient's menopausal status. A rule-based NLP system was designed to capture evidence of a patient's menopause status including dates of a patient's last menstrual period, reproductive surgeries, and postmenopause diagnosis as well as their use of birth control and menstrual interruptions. NLP-derived output was used in combination with structured EHR data to classify a patient's menopausal status. NLP processing and patient classification were performed on a cohort of 307512 female Veterans receiving healthcare at the US Department of Veterans Affairs (VA). NLP was validated at 99.6% precision. Including the NLP-derived data into a menopause phenotype increased the number of patients with data relevant to their menopausal status by 118%. Using structured codes alone, 81173 (27.0%) are able to be classified as postmenopausal or premenopausal. However, with the inclusion of NLP, this number increased 167804 (54.6%) patients. The premenopausal category grew by 532.7% with the inclusion of NLP data. By employing NLP, it became possible to identify documented data elements that predate VA care, originate outside VA networks, or have no corresponding structured field in the VA EHR that would be otherwise inaccessible for further analysis. NLP can be used to identify concepts relevant to a patient's menopausal status in clinical notes. Adding NLP-derived data to an algorithm classifying a patient's menopausal status significantly increases the number of patients classified using EHR data, ultimately enabling more detailed assessments of the impact of menopause on health outcomes.

Prediction of Venous Thromboembolism in Diverse Populations Using Machine Learning and Structured Electronic Health Records.

Venous thromboembolism (VTE) is a major cause of morbidity and mortality worldwide. Current risk assessment tools, such as the Caprini and Padua scores and Wells criteria, have limitations in their applicability and accuracy. This study aimed to develop machine learning models using structured electronic health record data to predict diagnosis and 1-year risk of VTE. We trained and validated models on data from 159 001 participants in the Mount Sinai Data Warehouse. We then externally tested them on 401 723 participants in the UK Biobank and 123 039 participants in All of Us. All data sets contain populations of diverse ancestries and clinical histories. We used these data sets to develop small, medium, and large models with increasing features on a range of optimizing portability to maximizing performance. We make trained models publicly available in click-and-run format at https://doi.org/10.17632/tkwzysr4y6.6. In the holdout and external test sets, respectively, models achieved areas under the receiver operating characteristic curve of 0.80 to 0.83 and 0.72 to 0.82 for VTE diagnosis prediction and 0.76 to 0.78 and 0.64 to 0.69 for 1-year risk prediction, significantly outperforming the Padua score. Models also demonstrated robust performance across different VTE types and patient subsets, including ethnicity, age, and surgical and hospitalization status. Models identified both established and novel clinical features contributing to VTE risk, offering valuable insights into its underlying pathophysiology. Machine learning models using structured electronic health record data can significantly improve VTE diagnosis and 1-year risk prediction in diverse populations. Model probability scores exist on a continuum, affecting mortality risk in both healthy individuals and VTE cases. Integrating these models into electronic health record systems to generate real-time predictions may enhance VTE risk assessment, early detection, and preventative measures, ultimately reducing the morbidity and mortality associated with VTE.

Prediction Models for Glaucoma in a Multicenter Electronic Health Records Consortium: The Sight Outcomes Research Collaborative

ObjectiveAdvances in artificial intelligence have enabled the development of predictive models for glaucoma. However, most work is single-center and uncertainty exists regarding the generalizability of such models. The purpose of this study was to build and evaluate machine learning (ML) approaches to predict glaucoma progression requiring surgery using data from a large multicenter consortium of electronic health records (EHR). DesignCohort study. Participants36,548 patients with glaucoma, as identified by ICD codes from six academic eye centers participating in the Sight OUtcomes Research Collaborative (SOURCE). MethodsWe developed machine learning models to predict whether glaucoma patients would progress to glaucoma surgery in the coming year (identified by CPT codes) using the following modeling approaches: 1) penalized logistic regression (lasso, ridge, and elastic net); 2) tree-based models (random forest, gradient boosted machines, XGBoost), and 3) deep learning models. Model input features included demographics, diagnosis codes, medications, and clinical information (intraocular pressure, visual acuity, refractive status, and central corneal thickness) available from structured EHR data. One site was reserved as an “external site” test set (N=1550); of the patients from the remaining sites, 10% each were randomly selected to be in development and test sets, with the remaining 27999 reserved for model training. Main Outcome and MeasuresEvaluation metrics included area under the receiver operating characteristic curve (AUROC) on the test set and the external site. Results6019 (16.5%) of 36,548 patients underwent glaucoma surgery. Overall, the AUROC ranged from 0.735-0.771 on the random test set and from 0.706-0.754 on the external test site, with the XGBoost and random forest model performing best, respectively. There was greatest performance decrease from the random test set to the external test site for the penalized regression models. ConclusionsML models developed using structured EHR data can reasonably predict whether glaucoma patients will need surgery, with reasonable generalizability to an external site. Additional research is needed to investigate the impact of protected class characteristics such as race or gender on model performance and fairness.

Predicting multiple sclerosis severity with multimodal deep neural networks

Multiple Sclerosis (MS) is a chronic disease developed in the human brain and spinal cord, which can cause permanent damage or deterioration of the nerves. The severity of MS disease is monitored by the Expanded Disability Status Scale, composed of several functional sub-scores. Early and accurate classification of MS disease severity is critical for slowing down or preventing disease progression via applying early therapeutic intervention strategies. Recent advances in deep learning and the wide use of Electronic Health Records (EHR) create opportunities to apply data-driven and predictive modeling tools for this goal. Previous studies focusing on using single-modal machine learning and deep learning algorithms were limited in terms of prediction accuracy due to data insufficiency or model simplicity. In this paper, we proposed the idea of using patients’ multimodal longitudinal and longitudinal EHR data to predict multiple sclerosis disease severity in the future. Our contribution has two main facets. First, we describe a pioneering effort to integrate structured EHR data, neuroimaging data and clinical notes to build a multi-modal deep learning framework to predict patient’s MS severity. The proposed pipeline demonstrates up to 19% increase in terms of the area under the Area Under the Receiver Operating Characteristic curve (AUROC) compared to models using single-modal data. Second, the study also provides valuable insights regarding the amount useful signal embedded in each data modality with respect to MS disease prediction, which may improve data collection processes.

A cross-modal clinical prediction system for intensive care unit patient outcome

Electronic Health Records (EHRs) are becoming an encyclopedia of clinical concepts, encompassing diagnoses, medications and clinical descriptions to describe patients’ health conditions. The abundance of EHR data presents unprecedented opportunities for leveraging machine learning in clinical predictive tasks. However, the complex process of acquiring healthcare data in some settings like Intensive Care Unit (ICU) gives rise to various challenges in mining EHR, such as modality diversity, heterogeneity, data silo and data quality problems, etc. In this paper, to overcome the data heterogeneity and data quality issues, we propose a Code-Text cross-modal Contrastive Learning (CTCL) framework specifically designed for predicting ICU patient outcomes. This framework leverages both structured (multi-view medical codes) and unstructured (clinical notes) EHR data to extract patient representations. To capture the complicated intra and inter interactions among medical codes, CTCL employs a multi-view graph convolution network to extract code encodings and a cross-view contrastive loss to regularize the model parameters. To enhance patient representations across different modalities, a cross-modal contrastive learning module is introduced. Experimental evaluations on the eICU-CRD database demonstrate that CTCL outperforms state-of-the-art competitors on the ICU mortality prediction task and in-hospital readmission prediction task. Furthermore, ablation studies and discussions demonstrate the effectiveness of each module of the framework.

Quality metrics at scale: Deriving time to first cancer treatment from electronic health records.

580 Background: Measurements of cancer care quality are critical for internal administration and participation in value-based care programs, towards the goal of improving patient outcomes. Data from electronic health records (EHRs) have the potential to allow for automatic computation of quality measures at a larger scale than manual chart review. This requires the creation of algorithms that process data and apply custom logic to define a clinically meaningful measurement. In this work, we describe the development and performance of an algorithm for deriving time from first primary oncologist visit to first cancer treatment using EHR data. Methods: This EHR-derived measure of time to first treatment (TTT) was implemented at an academic medical center for all patients who had a visit since 01/01/2011 and a cancer-related diagnosis code. We defined a patient’s primary oncologist as the first medical oncologist who they visited at least 5 times. These criteria were chosen to find the first provider at the site who the patient likely established care with and received treatment from. Then the patient’s TTT is computed as the number of days between their first visit date with this oncologist and the date of their first antineoplastic drug administration. Data on other treatment modalities, such as surgery and radiation therapy, were not available at the time of algorithm development. We allow for patients to receive their first treatment up to 60 days before their first visit to account for inpatient treatments before a primary outpatient oncologist is established. This algorithm’s performance was assessed through manual chart review among a sample of 38 patients. Results: Our algorithm correctly assigned 37 out of 38 patients (97.4%) to their primary oncologist, and computed the correct TTT for 28 out of 38 patients (73.4%). Incorrect TTT computations were mostly due to missingness of systemic therapy administrations in the structured EHR data; these were primarily for oral therapies. Conclusions: This work demonstrates the feasibility of measuring a patient’s TTT from structured EHR data alone. In particular, a patient’s initial visit with their primary oncologist can be derived with high accuracy. From this index date, the accuracy of the metric is dependent on the availability of structured treatment data. To further improve the EHR-derived metric, future work should incorporate other treatment modalities through structured data sources such as billing codes, and improve completeness of systemic therapy data in a scalable way, such as from natural language processing techniques.[Table: see text]

Implementing a BRAF biomarker electronic clinical quality measure in CancerLinQ.

418 Background: Achieving the benefits of precision oncology is dependent on timely and appropriate biomarker testing. ASCO received funding from the Gordon &amp; Betty Moore Foundation’s Diagnostic Excellence Initiative to develop and test a biomarker electronic Clinical Quality Measure (eCQM) in colorectal cancer for implementation in SmartLinQ, CancerLinQ’s quality platform. Methods: Working collaboratively with a Technical Expert Panel (TEP), ASCO staff successfully developed an eCQM, BRAF Mutational Analysis Results Received for Patients Treated for Metastatic Colorectal Cancer. CancerLinQ tested the implementation of three versions of the measure's technical specifications against data from 61 organizations and multiple electronic health record (EHR) systems. We analyzed the availability of required data elements in native and curated data, and variability and frequency of BRAF mutation testing documentation. The measure was also tested for validity and reliability among volunteer oncology practices. Results: Implementation efforts revealed that BRAF information is infrequently found in structured data fields and often lacks specificity for the specific variant tested. Ultimately, implementation of the measure in CancerLinQ was infeasible due to a lack of BRAF testing data and variant specificity across its active organizations. Reliability testing among oncology practices uncovered a lack of substantial performance gap with an average measure score of 87.4, 95% CI [86.1, 88.8], making the measure unsuitable for Consensus-Based Entity (CBE) endorsement or use in federal reporting programs. Conclusions: While ASCO’s implementation efforts yielded clear benefits, including identification of future CancerLinQ platform enhancements, gaps in the capture of BRAF testing data and inconsistent documentation of BRAF variants across EHR systems prevented successful implementation of the eCQM at this time. These findings underscore the current difficulty in ascertaining the specificity of key molecular variants in structured EHR data alone without human curation or access to unstructured data, as well as an ongoing need for measure stakeholders to advance eCQM standards necessary for supporting future eCQM deployment.

Impact of integrating standardized venous thromboembolism (VTE) prophylaxis orders for patients with multiple myeloma into an electronic health record (EHR).

469 Background: VTE is a serious complication in multiple myeloma patients, who have a 9-fold increased risk compared to the general population. Prophylaxis for VTE is recommended by international guidelines in multiple myeloma patients based on VTE risk assessment, but published studies have demonstrated prescribing may be suboptimal. Our aim was to evaluate the impact of integrating standardized VTE prophylaxis orders for multiple myeloma patients on active treatment into our EHR, which services over 2300 providers within 500 community cancer treatment locations across the United States. Methods: National Comprehensive Cancer Network Guideline-recommended VTE prophylaxis agents were integrated as optional orders into multiple myeloma EHR regimen templates from June to July 2022. Optional informational orders were also included: a “VTE Prophylaxis Reminder” order served as a trigger for the care team if prophylaxis was not prescribed and a “VTE Prophylaxis Exception” order served as documentation if the patient did not qualify for anticoagulation. An educational guide about VTE risk stratification and prophylaxis in multiple myeloma was developed and embedded in regimen templates. To evaluate the impact of this intervention, a structured EHR data export was used to identify multiple myeloma patients with regimen templates ordered during the prespecified time frames. A retrospective chart review was conducted to identify documentation of anticoagulation or reason for exception on or prior to cycle 1 day 1 of the multiple myeloma regimen. Data were compared using the Fisher’s exact test. We also evaluated ordering of VTE prophylaxis and use of informational orders from regimens during the post-implementation phase. Results: A total of 297 patients were identified, with 128 during the pre-implementation phase (May 2022) and 169 during the post-implementation phase (September 2022). Documentation of anticoagulation or reason for exception on or prior to cycle 1 day 1 of the multiple myeloma regimen increased from 70% (89/128) during the pre-implementation phase to 81% (137/169) during the post-implementation phase (two-sided p=0.0276). Aspirin was the most common agent prescribed in 73% and 64% of patients, respectively. During the post-implementation phase, VTE prophylaxis was prescribed directly from the regimen in 16% of patients. The “VTE Prophylaxis Reminder” order was utilized for 89% of patients and the “VTE Prophylaxis Exception” order was utilized for 1 patient. Conclusions: Integration of standardized orders into EHR regimen templates may facilitate prescribing and serve as reminders for the care team. We saw an 11% increase in documentation of VTE prophylaxis, which may potentially reduce risk of VTE in this patient population. Efforts are ongoing to highlight the need for anticoagulation and importance of documentation.

Identifying Probable Dementia in Undiagnosed Black and White Americans Using Machine Learning in Veterans Health Administration Electronic Health Records

The application of natural language processing and machine learning (ML) in electronic health records (EHRs) may help reduce dementia underdiagnosis, but models that are not designed to reflect minority populations may instead perpetuate underdiagnosis. To improve the identification of undiagnosed dementia, particularly in Black Americans (BAs), we developed support vector machine (SVM) ML models to assign dementia risk scores based on features identified in unstructured EHR data (via latent Dirichlet allocation and stable topic extraction in n = 1 M notes) and structured EHR data. We hypothesized that separate models would show differentiation between racial groups, so the models were fit separately for BAs (n = 5 K with dementia ICD codes, n = 5 K without) and White Americans (WAs; n = 5 K with codes, n = 5 K without). To validate our method, scores were generated for separate samples of BAs (n = 10 K) and WAs (n = 10 K) without dementia codes, and the EHRs of 1.2 K of these patients were reviewed by dementia experts. All subjects were age 65+ and drawn from the VA, which meant that the samples were disproportionately male. A strong positive relationship was observed between SVM-generated risk scores and undiagnosed dementia. BAs were more likely than WAs to have undiagnosed dementia per chart review, both overall (15.3% vs. 9.5%) and among Veterans with &gt;90th percentile cutoff scores (25.6% vs. 15.3%). With chart reviews as the reference standard and varied cutoff scores, the BA model performed slightly better than the WA model (AUC = 0.86 with negative predictive value [NPV] = 0.98, positive predictive value [PPV] = 0.26, sensitivity = 0.61, specificity = 0.92 and accuracy = 0.91 at &gt;90th percentile cutoff vs. AUC = 0.77 with NPV = 0.98, PPV = 0.15, sensitivity = 0.43, specificity = 0.91 and accuracy = 0.89 at &gt;90th). Our findings suggest that race-specific ML models can help identify BAs who may have undiagnosed dementia. Future studies should examine model generalizability in settings with more females and test whether incorporating these models into clinical settings increases the referral of undiagnosed BAs to specialists.

Associations between blood pressure control and clinical events suggestive of nutrition care documented in electronic health records of patients with hypertension

BackgroundClinical events suggestive of nutrition care found in electronic health records (EHRs) are rarely explored for their associations with hypertension outcomes.MethodsLongitudinal analysis using structured EHR data from primary care visits at a health system in the US from December 2017—December 2020 of adult patients with hypertension (n = 4,237) tested for associations between last visit blood pressure (BP) control (≤ 140 Systolic BP and ≤ 90 Diastolic BP) and ≥ 1 nutrition care clinical event operationalized as (overweight or obesity (BMI > 25 or 30, respectively) diagnoses, preventive care visits, or provision of patient education materials (PEM)). Descriptive statistics and longitudinal targeted maximum likelihood estimation (LTMLE) models were conducted to explore average treatment effects (ATE) of timing and dose response from these clinical events on blood pressure control overall and by race.ResultsThe median age was 62 years, 29% were male, 52% were Black, 25% were from rural areas and 50% had controlled BP at baseline. Annual documentation of overweight/obesity diagnoses ranged 3.0–7.8%, preventive care visits ranged 6.2–15.7%, and PEM with dietary and hypertension content were distributed to 8.5–28.8% patients. LTMLE models stratified by race showed differences in timing, dose, and type of nutrition care. Black patients who had nutrition care in Year 3 only compared to none had lower odds for BP control (ATE -0.23, 95% CI: -0.38,-0.08, p = 0.003), preventive visits in the last 2 years high higher odds for BP control (ATE 0.31, 95% CI: 0.07,0.54, p = 0.01), and early or late PEMs had lower odds for BP control (ATE -0.08, 95% CI: -0.15,-0.01, p = 0.03 and ATE -0.23, 95% CI: -0.41,-0.05, p = 0.01, respectively).ConclusionsIn this study, clinical events suggestive of nutrition care are significantly associated with BP control, but are infrequent and effects differ by type, timing, and patient race. Preventive visits appear to have the most effect; additional research should include examining clinical notes for evidence of nutrition care among different populations, which may uncover areas for improving nutrition care for patients with chronic disease.

Comparing Natural Language Processing and Structured Medical Data to Develop a Computable Phenotype for Patients Hospitalized Due to COVID-19: Retrospective Analysis.

Throughout the COVID-19 pandemic, many hospitals conducted routine testing of hospitalized patients for SARS-CoV-2 infection upon admission. Some of these patients are admitted for reasons unrelated to COVID-19, and incidentally test positive for the virus. Because COVID-19-related hospitalizations have become a critical public health indicator, it is important to identify patients who are hospitalized because of COVID-19 as opposed to those who were admitted for other indications. We compared the performance of different computable phenotype definitions for COVID-19 hospitalizations that use different types of data from the electronic health records (EHR), including structured EHR data elements, provider notes, or a combination of both data types. We conducted a retrospective data analysis utilizing clinician chart review-based validation at a large academic medical center. We reviewed and analyzed the charts of 586 hospitalized individuals who tested positive for SARS-CoV-2 during January 2022. We used LASSO regression and Random Forests to fit classification algorithms that incorporated structured EHR data elements, provider notes, or a combination of structured data and provider notes. We used natural language processing to incorporate data from provider notes. The performance of each model was evaluated based on Area Under the Receiver Operator Characteristic (AUROC) and an associated decision rule based on sensitivity and positive predictive value. We also identified top words and clinical indicators of COVID-19-specific hospitalization and assessed the impact of different phenotyping strategies on estimated hospital outcome metrics. Based on a chart review, 38% of 586 patients were determined to be hospitalized for reasons other than COVID-19 despite having tested positive for SARS-CoV-2. A computable phenotype that used provider notes had significantly better discrimination than one that used structured EHR data elements (AUROC: 0.894 vs 0.841, P < .001), and performed similarly to a model that combined provider notes with structured data elements (AUROC: 0.894 vs 0.893). Assessments of hospital outcome metrics significantly differed based on whether the population included all hospitalized patients who tested positive for SARS-CoV-2 versus those who were determined to have been hospitalized due to COVID-19. These findings highlight the importance of cause-specific phenotyping for COVID-19 hospitalizations. More generally, this work demonstrates the utility of natural language processing approaches to derive information related to patient hospitalizations in cases where there may be multiple conditions that could serve as the primary indication for hospitalization.

The role of natural language processing techniques versus conventional methods to gain ICI safety insights from unstructured EHR data.

136 Background: Studies using International Classification of Diseases (ICD) codes have aimed to characterize the safety profile of cancer therapies in the Real-World Data setting, but this approach has limitations including an inability to illustrate patient journey and offer insights into drug-Adverse Event (AE) causality. Using data from a cohort of ~9000 patients treated with immune checkpoint inhibitors (ICI) at Mayo Clinic, we recently demonstrated the feasibility of applying Augmented Curation (AC), a natural language processing (NLP)-based innovation, on unstructured electronic health record (EHR) data to detect and characterize immune related AEs (irAEs). While the ability to identify and extract clinical details from EHRs has been previously demonstrated, our approach focuses on applying these techniques to identify implied textual causality of drug-event pairs. This study aims to build upon our prior findings and compare the accuracy, speed, and resource savings enabled by AC analysis versus manual review of patient notes. Methods: We compared the prevalence of adrenal insufficiency, colitis, and hypophysitis (A.C.H) among ICI patients at Mayo Clinic using ICD codes and AC which leverages SciBERT to perform scientific NLP tasks. AC models trained to identify drug-adverse event relationships were used to assess irAEs and subsequent treatments and demonstrated strong performance with F1 scores of 0.85. Manual curation of relevant clinical data, i.e. medications and adverse events, was employed to create a “gold standard” A.C.H. cohort. Time required for manual review compared to AC was documented for overall comparison. Corticosteroid/immunosuppressant use and ICI discontinuations were used as proxies of severity. Results: In our cohort, 540 unique patients experienced at least one of the A.C.H. irAEs. A.C.H patients were found to receive corticosteroids for their respective irAE 79% of the time and discontinued the ICI due to the irAE 7.7% of the time. 6.1% of Colitis patients were treated with a 2L immunosuppressant. Similar to the previous study, AC showed higher sensitivity, PPV and NPV compared to ICD codes alone within the A.C.H. cohort. Sentence extraction and entity classification using AC models on all 9,000-patients took ~10 minutes, whereas manual review of this cohort would take roughly 9 weeks (~10 minutes per patient). Conclusions: The results from this study reinforce that AC models are a valuable tool to comprehensively extract information and to impute relationships between entities, such as therapies and AEs. The AC models demonstrated better accuracy at a much greater speed compared to manual review of patient EHRs. Leveraging AC models provides the ability to manipulate unstructured clinical text married to structured EHR data at scale while minimizing need for manual curation.

Using artificial intelligence to identify patients with migraine and associated symptoms and conditions within electronic health records

BackgroundReal-world evidence (RWE)—based on information obtained from sources such as electronic health records (EHRs), claims and billing databases, product and disease registries, and personal devices and health applications—is increasingly used to support healthcare decision making. There is variability in the collection of EHR data, which includes “structured data” in predefined fields (e.g., problem list, open claims, medication list, etc.) and “unstructured data” as free text or narrative. Healthcare providers are likely to provide more complete information as free text, but extracting meaning from these fields requires newer technologies and a rigorous methodology to generate higher-quality evidence. Herein, an approach to identify concepts associated with the presence and progression of migraine was developed and validated using the complete patient record in EHR data, including both the structured and unstructured portions.Methods“Traditional RWE” approaches (i.e., capture from structured EHR fields and extraction using structured queries) and “Advanced RWE” approaches (i.e., capture from unstructured EHR data and processing by artificial intelligence [AI] technology, including natural language processing and AI-based inference) were evaluated against a manual chart abstraction reference standard for data collected from a tertiary care setting. The primary endpoint was recall; differences were compared using chi square.ResultsCompared with manual chart abstraction, recall for migraine and headache were 66.6% and 29.6%, respectively, for Traditional RWE, and 96.8% and 92.9% for Advanced RWE; differences were statistically significant (absolute differences, 30.2% and 63.3%; P < 0.001). Recall of 6 migraine-associated symptoms favored Advanced RWE over Traditional RWE to a greater extent (absolute differences, 71.5–88.8%; P < 0.001). The difference between traditional and advanced techniques for recall of migraine medications was less pronounced, approximately 80% for Traditional RWE and ≥ 98% for Advanced RWE (P < 0.001).ConclusionUnstructured EHR data, processed using AI technologies, provides a more credible approach to enable RWE in migraine than using structured EHR and claims data alone. An algorithm was developed that could be used to further study and validate the use of RWE to support diagnosis and management of patients with migraine.