Real-world Electronic Health Record Data Research Articles

Applying interpretable deep learning models to identify chronic cough patients using EHR data

Background and ObjectiveChronic cough (CC) affects approximately 10% of adults. Many disease states are associated with chronic cough, such as asthma, upper airway cough syndrome, bronchitis, and gastroesophageal reflux disease. The lack of an ICD code specific for chronic cough makes it challenging to identify such patients from electronic health records (EHRs). For clinical and research purposes, computational methods using EHR data are urgently needed to identify chronic cough cases. This research aims to investigate the data representations and deep learning algorithms for chronic cough prediction. MethodsUtilizing real-world EHR data from a large academic healthcare system from October 2005 to September 2015, we investigated Natural Language Representation of the EHR data and systematically evaluated deep learning and traditional machine learning models to predict chronic cough patients. We built these machine learning models using structured data (medication and diagnosis) and unstructured data (clinical notes). ResultsThe sensitivity and specificity of a transformer-based deep learning algorithm, specifically BERT with attention model, was 0.856 and 0.866, respectively, using structured data (medication and diagnosis). Sensitivity and specificity improved to 0.952 and 0.930 when we combined structured data with symptoms extracted from clinical notes. We further found that the attention mechanism of deep learning models can be used to extract important features that drive the prediction decisions. Compared with our previously published rule-based algorithm, the deep learning algorithm can identify more chronic cough patients with structured data. ConclusionsBy applying deep learning models, chronic cough patients can be reliably identified for prospective or retrospective research through medication and diagnosis data, widely available in EHR and electronic claims data, thus improving the generalizability of the patient identification algorithm. Deep learning models can identify chronic cough patients with even higher sensitivity and specificity when structured and unstructured EHR data are utilized. We anticipate language-based data representation and deep learning models developed in this research could also be productively used for other disease prediction and case identification.

Clinical Impact of Adherence to NCCN Guidelines for Biomarker Testing and First-Line Treatment in Advanced Non-Small Cell Lung Cancer (aNSCLC) Using Real-World Electronic Health Record Data.

IntroductionAlthough clinical guidelines are broadly available, the relationship between adherence and outcomes is not well studied. This study aimed to assess the association between adherence to National Comprehensive Cancer Network (NCCN) guidelines and clinical outcomes for adult patients with advanced non-small-cell lung cancer (aNSCLC).MethodsThis was a retrospective cohort study of adult patients with aNSCLC (stages IIIB, IIIC, and IV) from a de-identified real-world database. The objective was accomplished in a two-step analysis process. We first assessed adherence to NCCN recommendations for biomarker testing and overall survival (OS). Next, we assessed adherence to NCCN-recommended first-line therapy and time to treatment discontinuation (TTD). Multivariable Cox regression analyses were conducted to evaluate the association between guideline adherence and patient outcomes. Kaplan–Meier analyses were used to assess median OS and TTD.ResultsA total of 28,784 patients with a diagnosis for aNSCLC between January 1, 2011 and July 31, 2019 met the inclusion criteria for the analysis of NCCN-recommended biomarker testing adherence. Two-thirds of these patients (n = 19,787) had evidence of biomarker testing (adherent). Multivariable Cox models found that testing-adherent patients had a significantly lower risk of mortality [hazard ratio (HR) = 0.89, 95% confidence interval (CI) 0.86, 0.92; p < 0.01]. Median OS was modestly longer in the testing-adherent group compared to the testing-non-adherent group (15.4 vs. 14.2 months; p < 0.01). For the first-line therapy analysis, 15,898 patients met the inclusion criteria, of which 69.9% had evidence of appropriate first-line therapy (first-line-adherent). The multivariable Cox model found that adherent patients had significantly lower risk of treatment discontinuation versus non-adherent patients (HR = 0.60, 95% CI 0.57, 0.62; p < 0.01). First-line-adherent patients had a modest, yet significantly longer median TTD compared to first-line-non-adherent patients (3.45 vs. 2.40 months; p < 0.01).ConclusionsImproved clinical outcomes were observed in patients who were adherent to NCCN-recommended biomarker testing and first-line therapy. This study demonstrated the value of following NCCN guideline recommendations and the need to prioritize timely access to biomarker testing and individualized treatment.Supplementary InformationThe online version contains supplementary material available at 10.1007/s12325-020-01617-2.

Assessment of a Clinical Trial–Derived Survival Model in Patients With Metastatic Castration-Resistant Prostate Cancer

Randomized clinical trials (RCTs) are considered the criterion standard for clinical evidence. Despite their many benefits, RCTs have limitations, such as costliness, that may reduce the generalizability of their findings among diverse populations and routine care settings. To assess the performance of an RCT-derived prognostic model that predicts survival among patients with metastatic castration-resistant prostate cancer (CRPC) when the model is applied to real-world data from electronic health records (EHRs). The RCT-trained model and patient data from the RCTs were obtained from the Dialogue for Reverse Engineering Assessments and Methods (DREAM) challenge for prostate cancer, which occurred from March 16 to July 27, 2015. This challenge included 4 phase 3 clinical trials of patients with metastatic CRPC. Real-world data were obtained from the EHRs of a tertiary care academic medical center that includes a comprehensive cancer center. In this study, the DREAM challenge RCT-trained model was applied to real-world data from January 1, 2008, to December 31, 2019; the model was then retrained using EHR data with optimized feature selection. Patients with metastatic CRPC were divided into RCT and EHR cohorts based on data source. Data were analyzed from March 23, 2018, to October 22, 2020. Patients who received treatment for metastatic CRPC. The primary outcome was the performance of an RCT-derived prognostic model that predicts survival among patients with metastatic CRPC when the model is applied to real-world data. Model performance was compared using 10-fold cross-validation according to time-dependent integrated area under the curve (iAUC) statistics. Among 2113 participants with metastatic CRPC, 1600 participants were included in the RCT cohort, and 513 participants were included in the EHR cohort. The RCT cohort comprised a larger proportion of White participants (1390 patients [86.9%] vs 337 patients [65.7%]) and a smaller proportion of Hispanic participants (14 patients [0.9%] vs 42 patients [8.2%]), Asian participants (41 patients [2.6%] vs 88 patients [17.2%]), and participants older than 75 years (388 patients [24.3%] vs 191 patients [37.2%]) compared with the EHR cohort. Participants in the RCT cohort also had fewer comorbidities (mean [SD], 1.6 [1.8] comorbidities vs 2.5 [2.6] comorbidities, respectively) compared with those in the EHR cohort. Of the 101 variables used in the RCT-derived model, 10 were not available in the EHR data set, 3 of which were among the top 10 features in the DREAM challenge RCT model. The best-performing EHR-trained model included only 25 of the 101 variables included in the RCT-trained model. The performance of the RCT-trained and EHR-trained models was adequate in the EHR cohort (mean [SD] iAUC, 0.722 [0.118] and 0.762 [0.106], respectively); model optimization was associated with improved performance of the best-performing EHR model (mean [SD] iAUC, 0.792 [0.097]). The EHR-trained model classified 256 patients as having a high risk of mortality and 256 patients as having a low risk of mortality (hazard ratio, 2.7; 95% CI, 2.0-3.7; log-rank P < .001). In this study, although the RCT-trained models did not perform well when applied to real-world EHR data, retraining the models using real-world EHR data and optimizing variable selection was beneficial for model performance. As clinical evidence evolves to include more real-world data, both industry and academia will likely search for ways to balance model optimization with generalizability. This study provides a pragmatic approach to applying RCT-trained models to real-world data.

Abstract 16450: Early Identification of High Risk Cardiac Decompensation Phenotypes via Real-time Electronic Health Record Data

Introduction: Early identification of cardiac decompensation remains critical for improved patient outcomes. Digital phenotypes using real-time electronic health record (EHR) data offer an unbiased method to detect decompensation in at-risk individuals. Methods: Phenotypes designed to detect cardiac decompensation and its sequelae were retrospectively evaluated in 108,697 adult patient hospitalizations at a single center from October 2015-August 2018. The 6 phenotypes included hypotension, end organ dysfunction (EOD), hypoperfusion (concomitant hypotension and EOD), escalating vasoactive medication use (vasoactive meds), respiratory decline, and respiratory intervention. Median time from admission to phenotype development was measured in hours. In-hospital mortality and unanticipated ICU transfers were determined across all phenotypes and phenotype combinations. Results: Prevalence and time to detection varied across all six phenotypes (Table 1), with EOD found most frequently (35.7%) and detected earliest (3.4h, IQR 0.9-26.2h). Among individual phenotypes, patients with hypoperfusion had the highest rates of unanticipated ICU transfer (20.62%) and in-hospital mortality (20.99%). Patients meeting at least one phenotype had a 5.90% ICU transfer rate and 5.04% in-hospital mortality rate, compared to 0.62% mortality and 2.19% ICU transfer rates for patients meeting zero phenotypes. Among the 41 measured phenotype combinations, patients meeting all 6 phenotypes had the highest rates of unanticipated ICU transfer (28.75%) and in-hospital mortality (36.45%). Conclusions: Digital phenotypes of decompensation using real-world EHR data identify patients at higher risk of unexpected ICU transfer and in-hospital mortality at early times points in the hospitalization. Further studies will evaluate if implementation of a digital phenotype detection tool can improve care pathways and outcomes.

Utilizing time series data embedded in electronic health records to develop continuous mortality risk prediction models using hidden Markov models: A sepsis case study.

Continuous mortality risk monitoring is instrumental to manage a patient's care and to efficiently utilize the limited hospital resources. Due to incompleteness and irregularities of electronic health records (EHR), developing continuous mortality risk prediction using EHR data is a challenge. In this study, we propose a framework to continuously monitor mortality risk, and apply it to the real-world EHR data. The proposed method employs hidden Markov models (temporal technique) that take account of both the previous state of patient's health and the current value of clinical signs. Following the Sepsis-3 definition, we selected 3898 encounters of patients with suspected infection to compare the performance of temporal and non-temporal methods (Decision Tree (DT), Logistic Regression (LR), Naive Bayes (NB), Random Forest (RF), and Support Vector Machine (SVM)). The area under receiver operating characteristics (AUROC) curve, sensitivity, specificity and G-mean were used as performance measures. On the selected data, the AUROC of the proposed temporal framework (0.87) is 9-12% greater than the nontemporal methods (DT: 0.78, NB: 0.79, SVM: 0.79, LR: 0.80 and RF: 0.80). The results also show that our model (G-mean=0.78) provides a better balance between sensitivity and specificity compared to clinically acceptable bed-side criteria (G-mean=0.71). The proposed framework leverages the longitudinal data available in EHR and performs better than the non-temporal methods. The proposed method facilitates information related to the time of change of the patient's health that may help practitioners to plan early and develop effective treatment strategies.

Explainable artificial intelligence models using real-world electronic healthrecord data: a systematic scoping review.

To conduct a systematic scoping review of explainable artificial intelligence (XAI) models that use real-world electronic health record data, categorize these techniques according to different biomedical applications, identify gaps of current studies, and suggest future research directions. We searched MEDLINE, IEEE Xplore, and the Association for Computing Machinery (ACM) Digital Library to identify relevant papers published between January 1, 2009 and May 1, 2019. We summarized these studies based on the year of publication, prediction tasks, machine learning algorithm, dataset(s) used to build the models, the scope, category, and evaluation of the XAI methods. We further assessed the reproducibility of the studies in terms of the availability of data and code and discussed open issues and challenges. Forty-two articles were included in this review. We reported the research trend and most-studied diseases. We grouped XAI methods into 5 categories: knowledge distillation and rule extraction (N = 13), intrinsically interpretable models (N = 9), data dimensionality reduction (N = 8), attention mechanism (N = 7), and feature interaction and importance (N = 5). XAI evaluation is an open issue that requiresa deeper focus in the case of medical applications. We also discuss the importance of reproducibility of research work in this field, as well as the challenges and opportunities of XAI from 2 medical professionals' point of view. Based on our review, we found that XAI evaluation in medicine has not been adequately and formally practiced. Reproducibility remains a critical concern. Ample opportunities exist to advance XAI research in medicine.

The application of unsupervised deep learning in predictive models using electronic health records

BackgroundThe main goal of this study is to explore the use of features representing patient-level electronic health record (EHR) data, generated by the unsupervised deep learning algorithm autoencoder, in predictive modeling. Since autoencoder features are unsupervised, this paper focuses on their general lower-dimensional representation of EHR information in a wide variety of predictive tasks.MethodsWe compare the model with autoencoder features to traditional models: logistic model with least absolute shrinkage and selection operator (LASSO) and Random Forest algorithm. In addition, we include a predictive model using a small subset of response-specific variables (Simple Reg) and a model combining these variables with features from autoencoder (Enhanced Reg). We performed the study first on simulated data that mimics real world EHR data and then on actual EHR data from eight Advocate hospitals.ResultsOn simulated data with incorrect categories and missing data, the precision for autoencoder is 24.16% when fixing recall at 0.7, which is higher than Random Forest (23.61%) and lower than LASSO (25.32%). The precision is 20.92% in Simple Reg and improves to 24.89% in Enhanced Reg. When using real EHR data to predict the 30-day readmission rate, the precision of autoencoder is 19.04%, which again is higher than Random Forest (18.48%) and lower than LASSO (19.70%). The precisions for Simple Reg and Enhanced Reg are 18.70 and 19.69% respectively. That is, Enhanced Reg can have competitive prediction performance compared to LASSO. In addition, results show that Enhanced Reg usually relies on fewer features under the setting of simulations of this paper.ConclusionsWe conclude that autoencoder can create useful features representing the entire space of EHR data and which are applicable to a wide array of predictive tasks. Together with important response-specific predictors, we can derive efficient and robust predictive models with less labor in data extraction and model training.

Exploring uniformity of gestational diabetes screening and diagnosis using real-world electronic health record data

In the United States, stillbirths (fetal deaths ≥ 20 weeks' gestation) are now more common than infant deaths. Nationally available data are limited, and little is known about women's experiences around the time of a loss. The Pregnancy Risk Assessment Monitoring System (PRAMS), a state-based survey of women with a recent live birth, could be expanded to include women who experienced a stillbirth. We aimed to determine whether women with a recent stillbirth would be amenable to a PRAMS-like survey.Eligible women were Georgia residents aged ≥18 years with a reported stillbirth from December 1, 2012–February 28, 2013 identified through fetal death certificates. Women received a handwritten sympathy card, followed by a mailed questionnaire about their health and experiences around the time of the loss. Nonresponders received two additional mailings and up to three phone calls.During the study period, 149 eligible women had a reported stillbirth. Forty-nine (33%) women responded. Excluding women with invalid contact information (n = 26) yields an adjusted response rate of 40%. Response differed by race and/or ethnicity, but not by fetal, delivery, or other maternal characteristics.Women appear willing to respond to a survey regarding a recent stillbirth. Further studies of the expansion of PRAMS to include stillbirth are warranted.

A temporal model in Electronic Health Record search

Electronic Health Records (EHRs) refer to a collection of patient data, including diagnosis, medical history, medication, allergies, etc., mostly contained in the form of unstructured text. EHRs are designed to capture the state of a patient over time, thus the temporal information is crucial. Most previous works processing time in EHRs narrative focused on temporal expression extraction, using textual dimension to embody the temporal dimension. In this paper, we propose to model the textual and the temporal dimension of EHRs narrative jointly. To meet the challenge, we propose to model the EHRs narrative as temporal sequential data. A novel representation framework is designed to model the clinical narrative text as document sequence, where the textual and temporal dimension are modeled simultaneously. In the framework, a dynamic time warping based measure is proposed to quantify the similarity between EHRs of different patients. To verify the effectiveness of the model, the proposed model is applied in EHRs search via clustering algorithm. Experiments on real-world EHRs data set demonstrate that the proposed model sufficiently expresses the temporal feature of the EHRs and provides an effective solution for measuring the temporal similarity between EHRs of different patients.

TaGiTeD: Predictive Task Guided Tensor Decomposition for Representation Learning from Electronic Health Records

With the better availability of healthcare data, such as Electronic Health Records (EHR), more and more data analytics methodologies are developed aiming at digging insights from them to improve the quality of care delivery. There are many challenges on analyzing EHR, such as high dimensionality and event sparsity. Moreover, different from other application domains, the EHR analysis algorithms need to be highly interpretable to make them clinically useful. This makes representation learning from EHRs of key importance. In this paper, we propose an algorithm called Predictive Task Guided Tensor Decomposition (TaGiTeD), to analyze EHRs. Specifically, TaGiTeD learns event interaction patterns that are highly predictive for certain tasks from EHRs with supervised tensor decomposition. Compared with unsupervised methods, TaGiTeD can learn effective EHR representations in a more focused way. This is crucial because most of the medical problems have very limited patient samples, which are not enough for unsupervised algorithms to learn meaningful representations form. We apply TaGiTeD on real world EHR data warehouse and demonstrate that TaGiTeD can learn representations that are both interpretable and predictive.

Leveraging Food and Drug Administration Adverse Event Reports for the Automated Monitoring of Electronic Health Records in a Pediatric Hospital.

The objective of this study was to determine whether the Food and Drug Administration’s Adverse Event Reporting System (FAERS) data set could serve as the basis of automated electronic health record (EHR) monitoring for the adverse drug reaction (ADR) subset of adverse drug events. We retrospectively collected EHR entries for 71 909 pediatric inpatient visits at Cincinnati Children’s Hospital Medical Center. Natural language processing (NLP) techniques were used to identify positive diseases/disorders and signs/symptoms (DDSSs) from the patients’ clinical narratives. We downloaded all FAERS reports submitted by medical providers and extracted the reported drug-DDSS pairs. For each patient, we aligned the drug-DDSS pairs extracted from their clinical notes with the corresponding drug-DDSS pairs from the FAERS data set to identify Drug-Reaction Pair Sentences (DRPSs). The DRPSs were processed by NLP techniques to identify ADR-related DRPSs. We used clinician annotated, real-world EHR data as reference standard to evaluate the proposed algorithm. During evaluation, the algorithm achieved promising performance and showed great potential in identifying ADRs accurately for pediatric patients.

Validating an ontology-based algorithm to identify patients with Type 2 Diabetes Mellitus in Electronic Health Records

BackgroundImproving healthcare for people with chronic conditions requires clinical information systems that support integrated care and information exchange, emphasizing a semantic approach to support multiple and disparate Electronic Health Records (EHRs). Using a literature review, the Australian National Guidelines for Type 2 Diabetes Mellitus (T2DM), SNOMED-CT-AU and input from health professionals, we developed a Diabetes Mellitus Ontology (DMO) to diagnose and manage patients with diabetes. This paper describes the manual validation of the DMO-based approach using real world EHR data from a general practice (n=908 active patients) participating in the electronic Practice Based Research Network (ePBRN). MethodThe DMO-based algorithm to query, using Semantic Protocol and RDF Query Language (SPARQL), the structured fields in the ePBRN data repository were iteratively tested and refined. The accuracy of the final DMO-based algorithm was validated with a manual audit of the general practice EHR. Contingency tables were prepared and Sensitivity and Specificity (accuracy) of the algorithm to diagnose T2DM measured, using the T2DM cases found by manual EHR audit as the gold standard. Accuracy was determined with three attributes – reason for visit (RFV), medication (Rx) and pathology (path) – singly and in combination. ResultsThe Sensitivity and Specificity of the algorithm were 100% and 99.88% with RFV; 96.55% and 98.97% with Rx; and 15.6% and 98.92% with Path. This suggests that Rx and Path data were not as complete or correct as the RFV for this general practice, which kept its RFV information complete and current for diabetes. However, the completeness is good enough for this purpose as confirmed by the very small relative deterioration of the accuracy (Sensitivity and Specificity of 97.67% and 99.18%) when calculated for the combination of RFV, Rx and Path. The manual EHR audit suggested that the accuracy of the algorithm was influenced by data quality such as incorrect data due to mistaken units of measurement and unavailable data due to non-documentation or documented in the wrong place or progress notes, problems with data extraction, encryption and data management errors. ConclusionThis DMO-based algorithm is sufficiently accurate to support a semantic approach, using the RFV, Rx and Path to define patients with T2DM from EHR data. However, the accuracy can be compromised by incomplete or incorrect data. The extent of compromise requires further study, using ontology-based and other approaches.