Distributed Research Network Research Articles

Lessons from the Electronic Data Methods Forum

The Electronic Data Methods (EDM) Forum, with support from the Agency for Healthcare Research and Quality, exists to advance knowledge and practice on the use of electronic clinical data (ECD) for comparative effectiveness research, patient-centered outcomes research, and quality improvement (QI). The EDM Forum facilitates collaboration between the Prospective Outcome Systems using Patient-specific Electronic data to Compare Tests and therapies, Scalable Distributed Research Network, and Enhanced registry projects funded by Agency for Healthcare Research and Quality. This overview describes a second set of papers commissioned by the EDM Forum, published in this supplement to Medical Care. The papers that are included discuss challenges and innovations from the research and QI community using ECD. The papers in this supplement provide lessons learned based on experiences building transparent, scalable, reusable networks for research and QI. Through these papers, and a new open access e-journal, eGEMs, the EDM Forum is working to advance the science of health research and QI using ECD to improve patient outcomes.

PRM10 - Exploration of heterogeneity in distributed research network drug safety analyses

PRM10 - Exploration of heterogeneity in distributed research network drug safety analyses

Managing protected health information in distributed research network environments: automated review to facilitate collaboration

BackgroundMulti-site health sciences research is becoming more common, as it enables investigation of rare outcomes and diseases and new healthcare innovations. Multi-site research usually involves the transfer of large amounts of research data between collaborators, which increases the potential for accidental disclosures of protected health information (PHI). Standard protocols for preventing release of PHI are extremely vulnerable to human error, particularly when the shared data sets are large.MethodsTo address this problem, we developed an automated program (SAS macro) to identify possible PHI in research data before it is transferred between research sites. The macro reviews all data in a designated directory to identify suspicious variable names and data patterns. The macro looks for variables that may contain personal identifiers such as medical record numbers and social security numbers. In addition, the macro identifies dates and numbers that may identify people who belong to small groups, who may be identifiable even in the absences of traditional identifiers.ResultsEvaluation of the macro on 100 sample research data sets indicated a recall of 0.98 and precision of 0.81.ConclusionsWhen implemented consistently, the macro has the potential to streamline the PHI review process and significantly reduce accidental PHI disclosures.

Stages and Tools for Multinational Collaboration: The Perspective from the Coordinating Center of the International Consortium of Orthopaedic Registries (ICOR)

National and regional orthopaedic registries are an important resource for addressing the growing safety and effectiveness concerns associated with orthopaedic implants worldwide. Unfortunately, even the largest single-country registries or large regional registries are not large or diverse enough to address the needs of a variety of stakeholders such as patients, clinicians, regulatory agencies, payers, and manufacturers. The International Consortium of Orthopaedic Registries (ICOR) is a robust initiative whose aim is to overcome the limitations of individual registries and create an infrastructure for multinational, multiregional evaluations of orthopaedic devices and procedures through a network of international registries1. To enhance multinational/multiregional collaboration, ICOR will rely on several levels/methods available for creating research networks. One of the most recognized methods is the creation of large, centralized databases. While the centralized registry seems attractive because of its capacity to reduce the efforts needed to conduct research across multiple data holders, there are many security, proprietary, operational, legal, and patient-privacy concerns associated with this approach2,3. This approach often requires sharing identifiable information such as a national identifier (for example, a Social Security number), name, and mailing address. Even if data are de-identified, there is still a risk of re-identification based on the unique combination of multiple patient factors that are needed for the conduct of longitudinal studies. This limitation can hamper the coordination of data contained in multiple registries. In addition, this method is particularly limiting at early stages of collaborative research work, when confidence building and trustful partnerships are critical components of success. Many decentralized approaches for collaboration that protect each individual data holder and allow comprehensive device/technology evaluations with little or no loss of efficiency are available to ICOR researchers. These approaches can in general be described as distributed research network methods, such as those described for …

Distributed Health Data Networks

Comparative effectiveness research, medical product safety evaluation, and quality measurement will require the ability to use electronic health data held by multiple organizations. There is no consensus about whether to create regional or national combined (eg, "all payer") databases for these purposes, or distributed data networks that leave most Protected Health Information and proprietary data in the possession of the original data holders. Demonstrate functions of a distributed research network that supports research needs and also address data holders concerns about participation. Key design functions included strong local control of data uses and a centralized web-based querying interface. We implemented a pilot distributed research network and evaluated the design considerations, utility for research, and the acceptability to data holders of methods for menu-driven querying. We developed and tested a central, web-based interface with supporting network software. Specific functions assessed include query formation and distribution, query execution and review, and aggregation of results. This pilot successfully evaluated temporal trends in medication use and diagnoses at 5 separate sites, demonstrating some of the possibilities of using a distributed research network. The pilot demonstrated the potential utility of the design, which addressed the major concerns of both users and data holders. No serious obstacles were identified that would prevent development of a fully functional, scalable network. Distributed networks are capable of addressing nearly all anticipated uses of routinely collected electronic healthcare data. Distributed networks would obviate the need for centralized databases, thus avoiding numerous obstacles.

C-D3-03: Developing a Distributed Research Network to Conduct Population-based Studies and Safety Surveillance

Introduction: The overall objective of this project is to design a scalable distributed research network (DRN) that supports secure data analyses for research on the comparative effectiveness and safety of therapeutics. The plan for achieving this objective includes designing an information system architecture, implementation of a prototype of the specified system architecture, and completion of a proof of concept pilot project conducted in partnership with the Agency for Healthcare Research and Quality (AHRQ). Methods already implemented by the HMORN provide the foundation for the development of this DRN. These include established procedures for: use of decentralized data files that remain in possession of the covered entities; governance methods emphasizing local control and decision-making about participation in research projects; transparency in choosing projects, protocol development, and communication of results; and 4) best practices for preserving privacy, ensuring data integrity, and accurate data interpretation. DRN System Architecture: The DRN architecture is being modeled as a federation of separate, heterogeneous databases that will be accessible through a common exchange model, implemented as a common query system (CQS). Under the common exchange model, heterogeneous data and systems do not need to be physically integrated, but the model affords interoperability, such that users can access these data resources using a single interface using a common syntax and semantic representation of data elements and coding schemes. This is a distinct advantage for the DRN, since there will be different data resource providers, some of which place strict constraints on the use of data. The common exchange model obviates the need for users to learn the syntactic and semantic characteristics of the resources available across the entire DRN. Prototype Development: The DRN is being developed using a rapid prototype life-cycle. This user centered approach employs incrementally-developed system prototypes such that all participants become stakeholders in the development process and have access to increasingly complex, concrete representations of the final system.

PS1-24: Proof of Principle Demonstrations of a Distributed Research Network: Findings and Lessons Learned

Researchers, policy makers and others commonly use electronic data routinely collected during the delivery of, or payment for, medical care to study the “real world” effectiveness, comparative effectiveness, safety, and costs of medical interventions. However, even very large individual healthcare data resources are often not big enough to adequately conduct post-marketing evidence studies. To improve the ability to use multiple distributed data resources, like the HMORN’s Virtual Data Warehouse, the Developing Evidence to Inform Decisions about Effectiveness (DEcIDE) centers at the HMORN Center for Education and Research on Therapeutics (CERT) and the University of Pennsylvania are developing a design for a scalable distributed research network. Funded by AHRQ’s Effective Health Care Program, the project intends to develop the framework for a distributed research network that will help close the knowledge gap regarding the “real world” effectiveness, comparative effectiveness, and safety of medical technologies. The network will have these important attributes: distributed architecture, strong local control of data uses, and federated querying. Constructing the network is a challenge, even among sites with fully operational local virtual data warehouse (VDW) installations, because of differences in computing environments and information systems, the need for responsible stewardship of clinical records, concerns related to data privacy, software development obstacles, the use of proprietary data, and the need to develop effective governance mechanisms. To investigate some of these issues, the project includes implementation of a proof of concept prototype involving these HMORN sites: Kaiser Permanente Northern California, Kaiser Permanente Colorado, Harvard Pilgrim Health Care, Group Health Cooperative, and Geisinger Health System. The goals of the proof of concept work are to distribute a query to separately stored data resources, execute the query remotely, and present aggregated results to an authorized system user. The proof of concept will be designed with a single network portal that manages the network software, including security, authorization, authentication, messaging, and query aggregation. We will present our findings from the proof of concept, including information on specific architecture and design used, the success of the proof of concept, and lessons learned from implementation.

C-D3-04: Governing Access to a Distributed Research Network's Data Resources

To answer many public health questions, it is essential to use information from more than one electronic data system, and efficient ways are needed to securely access and use data from multiple organizations while respecting the regulatory, legal, proprietary, and privacy implications of this data use and access. One approach centers on the development of distributed research networks that allow data owners to maintain confidentiality and physical control over their data, while permitting authorized users to ask essential questions. Once such a network is fully operating and key elements are in place, sharable data resources can be made available to approved network users, under approved conditions. For instance, data from a large cohort of hypertensive patients with five years of utilization (a hypertension cohort) could be available on the network. The following questions will need to be addressed: Who can have access? Under what conditions should access be granted? What policies/procedures are required? To address the specific needs associated with governance of a network’s resource(s), the authors call for the establishment of user eligibility requirements, policies to deal with funders (i.e., access rules for study funders), clear standard operating procedures, and guidelines for accessing the network. Recommendations to meet to those needs include: establishing data oversight policies; defining responsibilities for data resource access; defining responsibilities for data owners at each site (i.e., responding to queries when requests come in); creating standard operating procedures for the data resource; creating collaboration guidelines for external partners; and monitoring overall resource use. For the purpose of this poster, we propose to illustrate responsibilities for data owners at each site.

C-D3-01: Developing and Implementation of Secure Linear Regression on Distributed Databases

Background/Aims: Improving access to multiple sources of electronic health information is crucial for population-based health care research. But it also raises serious issues relating to data confidentiality and security. We propose to develop methods and tools (e.g., SAS macros) to enable multivariate analyses to be conducted using multiple datasets that are stored separately without exposing any patient level information. Phase I of this project focuses on linear regression. Methods: A two-step process is carried out by SAS macros. The macros will be installed on both the central and site computers that hold the analytic datasets. After the central computer initiates the process, each site computer executes the SAS program locally and submits intermediate summarized statistical results to the central computer. The central computer then combines the intermediate results, and computes the parameter estimates. A second round of program execution delivers the variance/covariance estimates of the parameter estimates. Results and Conclusions: Simulated separated data sets were used to test the macros. The results were identical to the outputs from the SAS PROC REG procedure when the data sets were combined. The macro enables remote execution of multivariate linear regression analysis within the context of a multi- institutional distributed research network. The successful implementation of the secure statistical analysis within a distributed database model will overcome one of the largest hurdles in fully developing such a model, namely, the current need to combine data for multivariate analyses. Obviating this need will save the time and resources currently spent developing data-sharing agreements and will greatly minimize the patient confidentiality and proprietary concerns

D*D:Analytical Clinical Information Retrieval System based on Hospital Information System-Overview and Use Examples-

Objective: Standardized clinical data are invaluable for secondary use of medical information. We constructed a standardized database and a data warehouse called D*D, based on the Standardized Structured Medical Information Exchange(SS-MIX)scheme. D*D enables physicians and researchers to perform complex searches with combined conditions, e.g. time to event. It contains data from 1999 for approximately 400,000 individual patients. The objective of this study was to provide an overview of the features of this database system, especially from the perspective of drug safety research.Methods: Three models of research questions were identified from established drug-risk combinations:1)gatifloxacin and hypoglycemia;2)statins and rhabdomyolysis;and 3)oral 5-fluorouracil S-1 and hepatotoxicity. D*D was searched using predefined keywords and conditions.Results: 1)A total of 3,635 patients were treated for diabetes. Among 20 diabetic patients prescribed gatifloxacin, hypoglycemia was recorded in one patient(1/38 prescriptions). 2)Among 5,926 patients who had been prescribed any statin within 10 years in our hospital, 6 patients(0.1%)experienced rhabdomyolysis. The incidence was similar to that for fibrate (1/740, 0.1%). The most confounded diagnosis was stiff shoulder. 3)Among 244 patients prescribed S-1, 19 patients(7.8%) experienced hepatotoxicity higher than CTCAE grade3 within 2 months from the prescription.Conclusion: With limited data items and search keys in standardized data storage, definitions of exposures and outcomes require careful assessment during protocol development. Considering that the system can be implemented at more than half of the hospitals that have already installed ordering systems, D*D can be one of the Japanese models for distributed research network.

A distributed research network model for post‐marketing safety studies: the Meningococcal Vaccine Study

We describe a multi-center post-marketing safety study that uses distributed data methods to minimize the need for covered entities to share protected health information (PHI). Implementation has addressed several issues relevant to creation of a large scale post-marketing drug safety surveillance system envisioned by the FDA's Sentinel Initiative. This retrospective cohort study of Guillain-Barré syndrome (GBS) following meningococcal conjugate vaccination incorporates the data and analytic expertise of five research organizations closely affiliated with US health insurers. The study uses administrative claims data, plus review of full text medical records to adjudicate the status of individuals with a diagnosis code for GBS (ICD9 357.0). A distributed network approach is used to create the analysis files and to perform most aspects of the analysis, allowing nearly all of the data to remain behind institutional firewalls. Pooled analysis files transferred to a central site will contain one record per person for approximately 0.2% of the study population, and contain PHI limited to the month and year of GBS onset for cases or the index date for matched controls. The first planned data extraction identified over 9 million eligible adolescents in the target age range of 11-21 years. They contributed an average of 14 months of eligible time on study over 27 months of calendar time. MCV4 vaccination coverage levels exceeded 20% among 17-18-year olds and 16% among 11-13 and 14-16-year-old age groups by the second quarter of 2007. This study demonstrates the feasibility of using a distributed data network approach to perform large scale post-marketing safety analyses and is scalable to include additional organizations and data sources. We believe these results can inform the development of a large national surveillance system.