Electronic Health Record Data Quality Research Articles

Estimating the prevalence of diabetic retinopathy in electronic health records with massive missing labels

ObjectiveThe paper aims to address the problem of massive unlabeled patients in electronic health records (EHR) who potentially have undiagnosed diabetic retinopathy (DR). It is desired to estimate the actual DR prevalence in EHR with 96 % missing labels. Materials and methodsThe Cerner Health Facts data are used in the study, with 3749 labeled DR patients and 97,876 unlabeled diabetic patients. This extensive dataset spans the demographics of the United States over the past two decades. We implemented state-of-art positive-unlabeled learning methods, including ensemble-based support vector machine, ensemble-based random forest, and Bayesian finite mixture modeling. ResultsThe estimated DR prevalence in the population represented by Cerner EHR is approximately 25 % and the classification techniques generally achieve an AUC of around 87 %. As a by-product, a predictive inference on the risk of DR based on a patient's personalized medical information is derived. DiscussionMissing labels is a common issue for EHR data quality. Ignoring these missing labels can lead to biased results in the analyses of EHR data. The problem is especially severe in the context of DR. It is thus important to use machine learning or statistical tools to identify the unlabeled patients. The tool in this paper helps both data analysts and clinicians in their practices.

The benefit of chart curation of electronic health record (EHR) data: Comparing data completeness between curated and structured data.

11165 Background: The main objective of an EHR is to effectively standardize clinical workflow. However, EHR data is also utilized extensively as secondary data for clinical research and quality improvement. Healthcare data exists in different forms such as paper, digital images, and notes, only some of which are recorded as structured data in the EHR. It is therefore essential to curate detailed information from unstructured data so that the patient’s journey can be understood in depth for research and quality studies. Methods: Patients diagnosed with metastatic breast cancer (mBC) between 01-Jan-2020 and 31-Dec-2022 were identified from the Integra Connect PrecisionQ de-identified database of 3 million cancer patients across 500 sites of care across ~80% community oncology and ~20% academic practices. Manual curation was conducted for a sample of these patients and the fill rates of crucial elements were captured as part of this study. Results: A total of 13,763 mBC patients were identified during this study period. The availability of data for the different variables was assessed from structured data in 13,120 patients. Additional information was obtained by manual curation for 643 patients. Information on staging and grade were available for 62.8% in the structured data, while curation increased the availability of these details to 99.5% among those 643 patients. The fill rates for patients’ tumor size and nodal status, as defined by the T and N values, were found to be 99% in the curated data compared to 60% in the structured data. Similarly, the fill rates were more than 98% for estrogen, progesterone, and HER2 receptor by curation, while they were only approximately 65% in the structured data. HER2 low, HER2-ve, and HER2+ve status was identified in 8%, 45.2%, and 10.1% by structured data and in 57.1%, 84.1%, and 14.0%, respectively, by curated data. Furthermore, the fill rates via curation were 97.1% for HER2 immunohistochemistry tests compared to 49.0% for structured data, while the fill rates for in-situ hybridization (ISH) or fluorescent in-situ hybridization (FISH) tests were 61.3% for curated data compared to 28.4% for structured data. This discrepancy is caused by the fact that IHC and FISH/ISH tests are primarily found only in PDF format. Conclusions: This study highlights the need for curation in order to maximize the utilization of EHR data for secondary research purposes and quality studies. Natural language processing and augmented curation methods can further enhance the quality of EHR data for secondary research.

Development and validation of a rule-based algorithm to identify periodontal diagnosis using structured electronic health record data.

To develop and validate an automated electronic health record (EHR)-based algorithm to suggest a periodontal diagnosis based on the 2017 World Workshop on the Classification of Periodontal Diseases and Conditions. Using material published from the 2017 World Workshop, a tool was iteratively developed to suggest a periodontal diagnosis based on clinical data within the EHR. Pertinent clinical data included clinical attachment level (CAL), gingival margin to cemento-enamel junction distance, probing depth, furcation involvement (if present) and mobility. Chart reviews were conducted to confirm the algorithm's ability to accurately extract clinical data from the EHR, and then to test its ability to suggest an accurate diagnosis. Subsequently, refinements were made to address limitations of the data and specific clinical situations. Each refinement was evaluated through chart reviews by expert periodontists at the study sites. Three-hundred and twenty-three charts were manually reviewed, and a periodontal diagnosis (healthy, gingivitis or periodontitis including stage and grade) was made by expert periodontists for each case. After developing the initial version of the algorithm using the unmodified 2017 World Workshop criteria, accuracy was 71.8% for stage alone and 64.7% for stage and grade. Subsequently, 16 modifications to the algorithm were proposed and 14 were accepted. This refined version of the algorithm had 79.6% accuracy for stage alone and 68.8% for stage and grade together. Our findings suggest that a rule-based algorithm for suggesting a periodontal diagnosis using EHR data can be implemented with moderate accuracy in support of chairside clinical diagnostic decision making, especially for inexperienced clinicians. Grey-zone cases still exist, where clinical judgement will be required. Future applications of similar algorithms with improved performance will depend upon the quality (completeness/accuracy) of EHR data.

An EHR Data Quality Evaluation Approach Based on Medical Knowledge and Text Matching

Introduction: Recently, medical artificial intelligence based on Electronic Health Records (EHR) is a significant research field, and EHR data has been widely used in clinical decision support systems and medical diagnosis systems. However, because EHR are used to record the patient's disease information and are not primarily designed for research and discovery, the utility of EHR for research will be hindered by data quality problems. Therefore, it is a meaningful and challenging task to evaluate the data quality of EHR before they are used in medical artificial intelligence. Most of the current EHR data quality evaluation methods are based on some conventional evaluation indicators, and rarely consider the introduction of clinical evidence.Materials and methods: we propose an EHR data quality evaluation approach based on clinical evidence and a deep text matching model. First, based on the medical knowledge of the particular disease, we establish the list of standard clinical evidence descriptions including typical symptoms and special signs, etc. Then we find the relevant clinical evidence from the EHR based on the text matching model, and finally evaluate the quality of the EHR based on the quantity and quality of the relevant clinical evidence found.Results: The experimental results of more than 1,000 EHR for two types of diseases show that our approach can effectively distinguish high-quality EHR from low-quality EHR, and the high-quality EHR found generally contains sufficient and consistent information related to disease diagnosis.Conclusions: Experiments results on a real-world dataset demonstrate the effectiveness of our EHR data quality evaluation approach based on medical knowledge and text matching.

Electronic health record data quality variability across a multistate clinical research network.

Electronic health record (EHR) data have many quality problems that may affect the outcome of research results and decision support systems. Many methods have been used to evaluate EHR data quality. However, there has yet to be a consensus on the best practice. We used a rule-based approach to assess the variability of EHR data quality across multiple healthcare systems. To quantify data quality concerns across healthcare systems in a PCORnet Clinical Research Network, we used a previously tested rule-based framework tailored to the PCORnet Common Data Model to perform data quality assessment at 13 clinical sites across eight states. Results were compared with the current PCORnet data curation process to explore the differences between both methods. Additional analyses of testosterone therapy prescribing were used to explore clinical care variability and quality. The framework detected discrepancies across sites, revealing evident data quality variability between sites. The detailed requirements encoded the rules captured additional data errors with a specificity that aids in remediation of technical errors compared to the current PCORnet data curation process. Other rules designed to detect logical and clinical inconsistencies may also support clinical care variability and quality programs. Rule-based EHR data quality methods quantify significant discrepancies across all sites. Medication and laboratory sources are causes of data errors.

Abstract A020: Assessment of EHR data quality variabilities among different racial groups in the cohort of de novo stage IV breast cancer

Abstract Introduction: Despite the decline in overall mortality and incidence of cancer in the US population, disparities in cancer care still largely exist within certain groups. The proportion of racial and ethnic minorities recruited to participate in cancer research is persistently lower than the US population. The rapid adoption of electronic health records (EHRs) systems has enabled computational cohort identification and eligibility screening for facilitating cancer research. However, EHR data is known to suffer from data quality issues due to numerous factors, such as care access. Patients with limited care access due to low socioeconomic status would naturally be sparsely represented in EHR data relative to those of higher status. Prospective enrollees may be disqualified due to biased or incomplete EHR data at the screening stage. Methods: To systematically identify data quality variabilities in EHRs, we implemented an information score (i-score) to reflect the density, variability, and irregularity of patients’ data. The pattern can be measured by estimating the variability of the time gaps between observations. For a patient observed n times, the encountered observation times are represented as x(1)…x(n). The relative time interval g for each observed encounter can be defined as g1 = [x(i+1)-x(i)]/[x(n)-x(1)], for I = 1…n - 1. The average amount of information of each observation is then defined as I, between 0 and 1, calculated as: I = 2/n+(n-2)/n[1-sqrt((n-1)Var{gi;i=1…n-1})]. An equality-spaced observation would receive a high i-score. The lower the i-score, the higher the data irregularity. We conducted a case study in a cohort of de novo stage IV breast cancer (DNIV) assembled by a previously validated phenotyping algorithm (Wang AACR 2020). The cohort contains 1,918 DNIV cases between 2004 and 2018 at Mayo Clinic Rochester. We used the i-score to assess patient visits, breast cancer diagnosis codes, and narrative documentation, achieved by the extraction of stage, surgery, de novo, and metastasis information based on the phenotyping definitions. The summary statistics of the i-scores were compared among four racial groups: African American (3%), Asian (1%), Caucasian (87%), and Other (9%). ANOVA F test was used to compare the group differences. Results and Conclusion: The mean i-scores for visit, diagnosis and narrative documentation were 0.712, 0.200, and 0.056 for African American, 0.412, 0.301, and 0.073 for Asian, 0.438, 0.278, and 0.056 for Caucasian, and 0.519, 0.343, and 0.043 for the other category, respectively. The F test indicated significant differences between visit (p<0.001) and diagnosis (p=0.0267) data. No difference was found in narrative documentation (p=0. 8591). In conclusion, there was a substantial i-score variation discovered, especially within the African American group, suggesting a strong need to address EHR quality disparity for cancer research. Citation Format: Sunyang Fu, Liwei Wang, Folakemi T. Odedina, Hongfang Liu. Assessment of EHR data quality variabilities among different racial groups in the cohort of de novo stage IV breast cancer [abstract]. In: Proceedings of the 15th AACR Conference on the Science of Cancer Health Disparities in Racial/Ethnic Minorities and the Medically Underserved; 2022 Sep 16-19; Philadelphia, PA. Philadelphia (PA): AACR; Cancer Epidemiol Biomarkers Prev 2022;31(1 Suppl):Abstract nr A020.

Quantifying Electronic Health Record Data Quality in Telehealth and Office-Based Diabetes Care.

Data derived from the electronic health record (EHR) are commonly reused for quality improvement, clinical decision-making, and empirical research despite having data quality challenges. Research highlighting EHR data quality concerns has largely been examined and identified during traditional in-person visits. To understand variations in data quality among patients managing type 2 diabetes mellitus (T2DM) with and without a history of telehealth visits, we examined three EHR data quality dimensions: timeliness, completeness, and information density. We used EHR data (2016-2021) from a local enterprise data warehouse to quantify timeliness, completeness, and information density for diagnostic and laboratory test data. Means and chi-squared significance tests were computed to compare data quality dimensions between patients with and without a history of telehealth use. Mean timeliness or T2DM measurement age for the study sample was 77.8 days (95% confidence interval [CI], 39.6-116.4). Mean completeness for the sample was 0.891 (95% CI, 0.868-0.914). The mean information density score was 0.787 (95% CI, 0.747-0.827). EHR data for patients managing T2DM with a history of telehealth use were timelier (73.3 vs. 79.8 days), and measurements were more uniform across visits (0.795 vs. 0.784) based on information density scores, compared with patients with no history of telehealth use. Overall, EHR data for patients managing T2DM with a history of telehealth visits were generally timelier and measurements were more uniform across visits than for patients with no history of telehealth visits. Chronic disease care relies on comprehensive patient data collected via hybrid care delivery models and includes important domains for continued data quality assessments prior to secondary reuse purposes.

Quantifying the Electronic Health Record Burden in Head and Neck Cancer Care.

Background Although the main task of health care providers is to provide patient care, studies show that increasing amounts of time are spent on documentation. Objective To quantify the time and effort spent on the electronic health record (EHR) in head and neck cancer care. Methods Cross-sectional time–motion study. Primary outcomes were the percentages of time spent on the EHR and the three main tasks (chart review, input, placing orders), number of mouse events, and keystrokes per consultation. Secondary outcome measures were perceptions of health care providers regarding EHR documentation and satisfaction. Results In total, 44.0% of initial oncological consultation (IOC) duration and 30.7% of follow-up consultation (FUC) duration are spent on EHR tasks. During 80.0% of an IOC and 67.9% of a FUC, the patient and provider were actively communicating. Providers required 593 mouse events and 1,664 keystrokes per IOC and 140 mouse events and 597 keystrokes per FUC, indicating almost 13 mouse clicks and close to 40 keystrokes for every minute of consultation time. Less than a quarter of providers indicated that there is enough time for documentation. Conclusion This study quantifies the widespread concern of high documentation burden for health care providers in oncology, which has been related to burnout and a decrease of patient–clinician interaction. Despite excessive time and effort spent on the EHR, health care providers still felt this was insufficient for proper documentation. However, the need for accurate and complete documentation is high, as reuse of information becomes increasingly important. The challenge is to decrease the documentation burden while increasing the quality of EHR data.

Chasm Between Cancer Quality Measures and Electronic Health Record Data Quality.

The Medicare Access and CHIP Reauthorization Act of 2015 (MACRA) requires eligible clinicians to report clinical quality measures (CQMs) in the Merit-Based Incentive Payment System (MIPS) to maximize reimbursement. To determine whether structured data in electronic health records (EHRs) were adequate to report MIPS CQMs, EHR data aggregated by ASCO's CancerLinQ platform were analyzed. Using the CancerLinQ health technology platform, 19 Oncology MIPS (oMIPS) CQMs were evaluated to determine the presence of data elements (DEs) necessary to satisfy each CQM and the DE percent population with patient data (fill rates). At the time of this analysis, the CancerLinQ network comprised 63 active practices, representing eight different EHR vendors and containing records for more than 1.63 million unique patients with one or more malignant neoplasms (1.73 million cancer cases). Fill rates for the 63 oMIPS-associated DEs varied widely among the practices. The average site had at least one filled DE for 52% of the DEs. Only 35% of the DEs were populated for at least one patient record in 95% of the practices. However, the average DE fill rate of all practices was 23%. No data were found at any practice for 22% of the DEs. Since any oMIPS CQM with an unpopulated DE component resulted in an inability to compute the measure, only two (10.5%) of the 19 oMIPS CQMs were computable for more than 1% of the patients. Although EHR systems had relatively high DE fill rates for some DEs, underfilling and inconsistency of DEs in EHRs render automated oncology MIPS CQM calculations impractical.

Evaluating a Prototype Clinical Decision Support Tool for Chronic Pain Treatment in Primary Care.

The Chronic Pain Treatment Tracker (Tx Tracker) is a prototype decision support tool to aid primary care clinicians when caring for patients with chronic noncancer pain. This study evaluated clinicians' perceived utility of Tx Tracker in meeting information needs and identifying treatment options, and preferences for visual design. We conducted 12 semi-structured interviews with primary care clinicians from four health systems in Indiana. The interviews were conducted in two waves, with prototype and interview guide revisions after the first six interviews. The interviews included exploration of Tx Tracker using a think-aloud approach and a clinical scenario. Clinicians were presented with a patient scenario and asked to use Tx Tracker to make a treatment recommendation. Last, participants answered several evaluation questions. Detailed field notes were collected, coded, and thematically analyzed by four analysts. We identified several themes: the need for clinicians to be presented with a comprehensive patient history, the usefulness of Tx Tracker in patient discussions about treatment planning, potential usefulness of Tx Tracker for patients with high uncertainty or risk, potential usefulness of Tx Tracker in aggregating scattered information, variability in expectations about workflows, skepticism about underlying electronic health record data quality, interest in using Tx Tracker to annotate or update information, interest in using Tx Tracker to translate information to clinical action, desire for interface with visual cues for risks, warnings, or treatment options, and desire for interactive functionality. Tools like Tx Tracker, by aggregating key information about past, current, and potential future treatments, may help clinicians collaborate with their patients in choosing the best pain treatments. Still, the use and usefulness of Tx Tracker likely relies on continued improvement of its functionality, accurate and complete underlying data, and tailored integration with varying workflows, care team roles, and user preferences.

Assessing quality and agreement of structured data in automatic versus manual abstraction of the electronic health record for a clinical epidemiology study

Objective We evaluate data agreement between an electronic health record (EHR) sample abstracted by automated characterization with a standard abstracted by manual review. Study Design and Setting We obtain data for an epidemiology cohort study using standard manual abstraction of the EHR and automated identification of the same patients using a structured algorithm to query the EHR. Summary measures of agreement (e.g., Cohen’s kappa) are reported for 12 variables commonly used in epidemiological studies. Results Best agreement between abstraction methods is observed among demographic characteristics such as age, sex, and race, and for positive history of disease. Poor agreement is found in missing data and negative history, suggesting potential impact for researchers using automated EHR characterization. EHR data quality depends upon providers, who may be influenced by both institutional and federal government documentation guidelines. Conclusion Automated EHR abstraction discrepancies may decrease power and increase bias; therefore, caution is warranted when selecting variables from EHRs for epidemiological study using an automated characterization approach. Validation of automated methods must also continue to advance in sophistication with other technologies, such as machine learning and natural language processing, to extract non-structured data from the EHR, for application to EHR characterization for clinical epidemiology.

Quality of Hospital Electronic Health Record (EHR) Data Based on the International Consortium for Health Outcomes Measurement (ICHOM) in Heart Failure: Pilot Data Quality Assessment Study.

BackgroundThere is increasing recognition that health care providers need to focus attention, and be judged against, the impact they have on the health outcomes experienced by patients. The measurement of health outcomes as a routine part of clinical documentation is probably the only scalable way of collecting outcomes evidence, since secondary data collection is expensive and error-prone. However, there is uncertainty about whether routinely collected clinical data within electronic health record (EHR) systems includes the data most relevant to measuring and comparing outcomes and if those items are collected to a good enough data quality to be relied upon for outcomes assessment, since several studies have pointed out significant issues regarding EHR data availability and quality.ObjectiveIn this paper, we first describe a practical approach to data quality assessment of health outcomes, based on a literature review of existing frameworks for quality assessment of health data and multistakeholder consultation. Adopting this approach, we performed a pilot study on a subset of 21 International Consortium for Health Outcomes Measurement (ICHOM) outcomes data items from patients with congestive heart failure.MethodsAll available registries compatible with the diagnosis of heart failure within an EHR data repository of a general hospital (142,345 visits and 12,503 patients) were extracted and mapped to the ICHOM format. We focused our pilot assessment on 5 commonly used data quality dimensions: completeness, correctness, consistency, uniqueness, and temporal stability.ResultsWe found high scores (>95%) for the consistency, completeness, and uniqueness dimensions. Temporal stability analyses showed some changes over time in the reported use of medication to treat heart failure, as well as in the recording of past medical conditions. Finally, the investigation of data correctness suggested several issues concerning the characterization of missing data values. Many of these issues appear to be introduced while mapping the IMASIS-2 relational database contents to the ICHOM format, as the latter requires a level of detail that is not explicitly available in the coded data of an EHR.ConclusionsOverall, results of this pilot study revealed good data quality for the subset of heart failure outcomes collected at the Hospital del Mar. Nevertheless, some important data errors were identified that were caused by fundamentally different data collection practices in routine clinical care versus research, for which the ICHOM standard set was originally developed. To truly examine to what extent hospitals today are able to routinely collect the evidence of their success in achieving good health outcomes, future research would benefit from performing more extensive data quality assessments, including all data items from the ICHOM standards set and across multiple hospitals.

Clinical comparison between trial participants and potentially eligible patients using electronic health record data: A generalizability assessment method

ObjectiveTo present a generalizability assessment method that compares baseline clinical characteristics of trial participants (TP) to potentially eligible (PE) patients as presented in their electronic health record (EHR) data while controlling for clinical setting and recruitment period. MethodsFor each clinical trial, a clinical event was defined to identify patients of interest using available EHR data from one clinical setting during the trial’s recruitment timeframe. The trial’s eligibility criteria were then applied and patients were separated into two mutually exclusive groups: (1) TP, which were patients that participated in the trial per trial enrollment data; (2) PE, the remaining patients. The primary outcome was standardized differences in clinical characteristics between TP and PE per trial. A standardized difference was considered prominent if its absolute value was greater than or equal to 0.1. The secondary outcome was the difference in mean propensity scores (PS) between TP and PE per trial, in which the PS represented prediction for a patient to be in the trial. Three diverse trials were selected for illustration: one focused on hepatitis C virus (HCV) patients receiving a liver transplantation; one focused on leukemia patients and lymphoma patients; and one focused on appendicitis patients. ResultsFor the HCV trial, 43 TP and 83 PE were found, with 61 characteristics evaluated. Prominent differences were found among 69% of characteristics, with a mean PS difference of 0.13. For the leukemia/lymphoma trial, 23 TP and 23 PE were found, with 39 characteristics evaluated. Prominent differences were found among 82% of characteristics, with a mean PS difference of 0.76. For the appendicitis trial, 123 TP and 242 PE were found, with 52 characteristics evaluated. Prominent differences were found among 52% of characteristics, with a mean PS difference of 0.15. ConclusionsDifferences in clinical characteristics were observed between TP and PE among all three trials. In two of the three trials, not all of the differences necessarily compromised trial generalizability and subsets of PE could be considered similar to their corresponding TP. In the remaining trial, lack of generalizability appeared present, but may be a result of other factors such as small sample size or site recruitment strategy. These inconsistent findings suggest eligibility criteria alone are sometimes insufficient in defining a target group to generalize to. With caveats in limited scalability, EHR data quality, and lack of patient perspective on trial participation, this generalizability assessment method that incorporates control for temporality and clinical setting promise to better pinpoint clinical patterns and trial considerations.

Factors affecting clinicians' adherence to principles of diagnosis documentation: A concept mapping approach for improved decision-making.

The quality of data in electronic health records (EHRs) depends on adherence of clinicians to principles of diagnosis documentation. A concept mapping (CM) approach was used to extract factors related to quality of clinicians' documentation that govern EHR data quality. Influential factors extracted from brainstorming sessions were sorted by individual participants, followed by a quantitative analysis using multidimensional scaling and cluster analysis to categorise sorted factors. Finally, a questionnaire was used to elicit the importance-feasibility of the extracted factors. Results were visualised by cluster maps and Go-Zone plots. Factors were classified into seven clusters: "knowledge about International Classification of Diseases and clinical coding," "need for facilitators and guidelines," "explaining the importance of the issue and defining responsibilities," "cooperation of other personnel," "codify legal requirements," "workload" and "clinical obstacles," as ranked by importance. To enhance the quality of EHR data, a collaboration between physicians, nurses, managers and EHR developers is required. CM is an acceptable approach to meet this objective. Our findings highlight the significance of clinical coding knowledge, awareness about its importance and applicability and use of well-structured information systems. In combination, these three factors can have a strong positive impact on the quality of EHR data. A list of solutions is provided for policymakers, and two interventions suggested, based on the findings of this study, including the adoption of EHRs that incorporate documentation guidelines. We further propose updated clinical training programs and a monitoring and feedback mechanism to facilitate the EHR documentation process.

Suitability of Electronic Health Record Data for Computational Phenotyping of Diabetes Mellitus at Nairobi Hospital, Nairobi City County, Kenya

This research aims to determine the applicability of routine healthcare in clinical informatics research. One of the key areas of research in precision medicine is computational phenotyping from longitudinal Electronic Health Record (EHR) data. The objective of this research was to determine how the interplay of EHR software design, the use of a data dictionary, the process of data collection, and the training and motivation of the human resource involved in the collection and entry of data into the EHR affect the quality of EHR data thus the suitability of such data for utility in computational phenotyping of diabetes mellitus. This research employed a prospective/retrospective study design at the diabetes clinic in Nairobi Hospital. The first source of data was from interviews with 32 staff; nurses, doctors, and health record officers using a referenced peer-reviewed usability questionnaire. Thereafter, a sample of EHR data collected during routine care between January 2012 and December 2016 was also analyzed by looking into the quality of clusters identified in the data using a density-based clustering algorithm and Statistical Package for Social Sciences (SPSS) version 21. Regression analysis shows that software design and the utility of a data dictionary explained 50.7% and 32.3% respectively in the improvement of the suitability of EHR data for computational phenotyping of diabetes mellitus. Also, EHR software was rated useful (82%) in accomplishing users’ daily tasks. However, EHR data were found to be unsuitable for utility in computational phenotyping of diabetes. Despite the fact that 88% of EHR data were clustered as noise, the clustering algorithm identified a total of 23 clusters from the diabetes dataset. However, with improved quality of EHR data, sub-phenotyping tasks would be achievable. This research concludes that the poor quality of EHR data is a result of employees’ unmet intrinsic factors of motivation.

Cohort selection for construction of a clinical natural language processing corpus

In Electronic Health Record (EHR) systems, key patient information is often captured in the form of unstructured clinical notes. The information from these notes can be extracted using Clinical Natural Language Processing (NLP). Training corpus is a key factor in development of efficient clinical NLP models. Clinical NLP corpus construction is complex and multifaceted. There are several challenges in corpus construction, but one challenge often not researched well is cohort selection aspect. In this study, we present methods employed and challenges encountered in cohort selection for construction of a clinical NLP corpus. In specific we present our methods in selection of cancer pathology reports to construct a corpus for automatic deidentification. 2100 pathology reports were extracted from 1833 (518 male and 1313 female) cancer patients using Health Level-7 (HL7) message standard. In terms of the age group distribution, the age group 60–70 years was highest with 872 patients. Our findings suggest deciphering the segment information from HL7 messages that are collected from different hospitals is a challenging task. The quality of HL7 messages also varied significantly with inconsistent tags making it difficult to identify reports that meet criteria set a priori. One of key lessons learned is linking the HL7 reports data with additional EMR data such as admissions, would help in identifying high quality reports and resolve duplicates. Also, our findings suggest that, in general the EHR data quality is poor with varying clinical coding and metadata standards between different hospitals. It is vital to identify and address these challenges for development of a high-quality corpus.

Electronic health records in IS research: Quality issues, essential thresholds and remedial actions

The increase in adoption of Electronic Health Record (EHR) systems by healthcare organizations has led to the proliferation of the use of EHR as a secondary data source in both IS and supporting fields. It is imperative that EHR data is exploited appropriately, which would lead to high quality results and enhanced reproducibility. However, the quality of the EHR data being used can vary significantly and can have repercussions for research outcomes. In this paper, we first discuss four major data quality issues present in EHR data. These issues are: (a) non-standard coding schemes, (b) missing data, (c) inconsistencies and (d) aggregation and augmentation of EHR data. Then, we discuss quality thresholds that need to be met in order to avoid the negative impacts of quality issues. Lastly, we discuss some remedial actions that researchers can take to enhance the quality of EHR data to meet the quality thresholds. The discussed issues, thresholds and remedial actions can also apply to a much wider set of data sources when used as secondary data in research.

Electronic Health Record Data Quality Issues Are Not Remedied by Increasing Granularity of Diagnosis Codes

Electronic Health Record Data Quality Issues Are Not Remedied by Increasing Granularity of Diagnosis Codes

Evaluation of interventions to improve inpatient hospital documentation within electronic health records: A Systematic Review

IntroductionDespite increased use of electronic health records (EHRs), EHR documentation quality remains poor. Consequently, EHR data quality is also negatively affected. Many services, including disease surveillance and health services research, utilize EHR data. Accordingly, several studies have attempted to improve EHR documentation quality in the inpatient setting using various interventions.
 Objectives and ApproachThe purpose of this systematic review was to synthesize the literature, and assess the effectiveness of interventions seeking to improve inpatient EHR documentation quality. To identify relevant experimental, quasi-experimental and observational studies, a search strategy was developed based on elaborate inclusion/exclusion criteria using four main themes: EHR, documentation, interventions, and type of study. Four databases, Cochrane, Medline, EMBASE, and CINAHL, were searched. Study quality assessment and data extraction from selected studies were performed using a Downs and Black and Newcastle-Ottawa Scale hybrid tool, and a REDCap form, respectively. Data was then analyzed and synthesized in a narrative semi-quantitative manner.
 ResultsAn in-depth search of the identified databases, grey literature and reference lists, revealed a final 20 studies for inclusion in this systematic review. Due to high heterogeneity in study design, population, interventions, comparators, document types and outcomes, data could not be standardized for a quantitative comparison. However, statistically significant results in interventions and affected outcomes were further presented and discussed. A higher number of studies reported significantly improved EHR documentation when using the interventions: ‘Education’ and ‘Implementing a new EHR Reporting System’. When implementing two or more interventions, more outcome measures were affected. There was no association between study quality or study design and number of interventions used. Only one of the 20 studies found EHR documentation worsened with the interventions used.
 Conclusion/ImplicationsInterventions implemented to enhance EHR documentation are highly variable and require standardization. Emphasis should be placed on this novel area of research to improve communication between healthcare providers, enhance continuity of care, reduce the burden in health information management, and to facilitate data sharing between centers, provinces, and countries.

PRM44 - Evaluation of Electronic Health Record Data Quality to Assess the Clinical Indices of Anemia in Dialysis Patients

To assess the quality and completeness of data in eHRs from an LDO in the US, focusing on demographic and clinical variables. We used Kahn’s framework (Analytic Methods, 2012) to objectively assess data quality from all de-identified patients across 2,900 US dialysis centers in 2015 (~229,000). Specific focus was made on the following quality domains as they related to anemia associated with CKD and its treatment: 1) missing values and anomalies in data, 2) accurate relationships between data tables, 3) temporal relationships, and 4) logical events. Demographic, clinical, and treatment variables included age, sex, hemoglobin (Hb), iron parameters (transferrin saturation [TSAT], ferritin), and erythropoietin stimulating agent (ESA) use were descriptively assessed using means, ranges, counts, and proportions. Results were analyzed with and without lab results that were missing or entered as zero. Low missingness was seen for age (0.08%), sex (0.04%), and dialysis treatment (0%) data. TSAT was recorded for 93% of patients (range 0-100%; mean 30.6%) and ferritin recorded for 93% of patients (range 0-15,743 ng/mL; mean 733 ng/mL). Hb labs were recorded for 96% of the LDO cohort (range 0-46.8 g/dL; mean 10.9 g/dL, same result obtained without zero values). Among patients with available Hb concentrations, 78% had non-zero Hb values <10.0 g/dL (the KDIGO suggested threshold for treatment). Of these patients, 11% were below this threshold for a span of at least 90 days based on subsequent lab draw dates. Of patients with Hb <10 g/dL, 98% were receiving an ESA. Using Kahn’s framework, certain demographic features, anemia treatment, and iron indices were consistently recorded in eHR LDO data. This provides support for the use of these LDO eHR anemia-related variables for future research. Most patients with Hb labs were anemic (78%) but received ESA treatment as suggested by KDIGO guidelines.