Design and Refinement of a Data Quality Assessment Workflow for a Large Pediatric Research Network.

Ritu Khare,Daniel Eckrich,L Charles Bailey,Matthew W Miller,Evanette Burrows,Hanieh Razzaghi,Harris Weinstein,Levon H Utidjian,Richard Hoyt,David Soler,Joshua Tucker,Victoria Soucek

doi:10.5334/egems.294

Abstract

Background:Clinical data research networks (CDRNs) aggregate electronic health record data from multiple hospitals to enable large-scale research. A critical operation toward building a CDRN is conducting continual evaluations to optimize data quality. The key challenges include determining the assessment coverage on big datasets, handling data variability over time, and facilitating communication with data teams. This study presents the evolution of a systematic workflow for data quality assessment in CDRNs.Implementation:Using a specific CDRN as use case, the workflow was iteratively developed and packaged into a toolkit. The resultant toolkit comprises 685 data quality checks to identify any data quality issues, procedures to reconciliate with a history of known issues, and a contemporary GitHub-based reporting mechanism for organized tracking.Results:During the first two years of network development, the toolkit assisted in discovering over 800 data characteristics and resolving over 1400 programming errors. Longitudinal analysis indicated that the variability in time to resolution (15day mean, 24day IQR) is due to the underlying cause of the issue, perceived importance of the domain, and the complexity of assessment.Conclusions:In the absence of a formalized data quality framework, CDRNs continue to face challenges in data management and query fulfillment. The proposed data quality toolkit was empirically validated on a particular network, and is publicly available for other networks. While the toolkit is user-friendly and effective, the usage statistics indicated that the data quality process is very time-intensive and sufficient resources should be dedicated for investigating problems and optimizing data for research.

Highlights

Clinical data research networks (CDRNs) aggregate electronic health record data from multiple hospitals to enable large-scale research
We evaluate and report the evolution of the workflow from different perspectives, including the underlying checks, issues reported to the sites, and the usage of the workflow by the sites
The first step of the workflow is executed locally and the results are shared with the Data Coordinating Center (DCC) for subsequent steps of the workflow

Summary

Introduction

Clinical data research networks (CDRNs) aggregate electronic health record data from multiple hospitals to enable large-scale research. Clinical data research networks (CDRNs) combine electronic health record (EHR) data from multiple hospital systems to provide integrated access for conducting large-scale research studies. One of the most critical aspects in building a CDRN is conducting continual quality evaluation to ensure that the patient-level clinical datasets are “fit for research use” [3,4,5]. A well-designed data quality (DQ) assessment program helps data developers in identifying programming and logic errors when deriving secondary datasets from EHRs (e.g. an incorrect mapping of patient’s race information into controlled vocabularies). It assists data consumers and scientists in learning the peculiar characteristics of network data It assists data consumers and scientists in learning the peculiar characteristics of network data (e.g. “acute respiratory tract infections” and “attention deficit hyperactive disorder” are likely to be among the most frequent diagnoses in a pediatric data resource) as well as helps assess the readiness of network data for specific research studies [6]

Results

Discussion

Conclusion