The American Heart Association's Recommendations for Expanding the Applications of Existing and Future Clinical Registries

Abstract

HomeCirculationVol. 123, No. 19The American Heart Association's Recommendations for Expanding the Applications of Existing and Future Clinical Registries Free AccessResearch ArticlePDF/EPUBAboutView PDFView EPUBSections ToolsAdd to favoritesDownload citationsTrack citationsPermissions ShareShare onFacebookTwitterLinked InMendeleyReddit Jump toFree AccessResearch ArticlePDF/EPUBThe American Heart Association's Recommendations for Expanding the Applications of Existing and Future Clinical RegistriesA Policy Statement From the American Heart Association Vincent J. Bufalino, MD, FAHA, Frederick A. Masoudi, MD, MSPH, FAHA,, Steven K. Stranne, MD, JD, Katie Horton, RN, MPH, JD, Nancy M. Albert, PhD, FAHA, Craig Beam, CRE, Robert O. Bonow, MD, FAHA, Roger L. (Vern) Davenport, Meighan Girgus, MBA, Gregg C. Fonarow, MD, FAHA, Harlan M. Krumholz, MD, SM, FAHA, Mark W. Legnini, DrPH, William R. Lewis, MD, Graham Nichol, MD, MPH, FAHA, Eric D. Peterson, MD, MPH, FAHA, Wayne Rosamond, PhD, FAHA, John S. Rumsfeld, MD, PhD, FAHA, Lee H. Schwamm, MD, FAHA, David M. Shahian, MD, FAHA, John A. Spertus, MD, MPH, FAHA, Pamela K. Woodard, MD, FAHA, Clyde W. Yancy, MD, FAHA and Vincent J. BufalinoVincent J. Bufalino Search for more papers by this author , Frederick A. MasoudiFrederick A. Masoudi Search for more papers by this author , Steven K. StranneSteven K. Stranne Search for more papers by this author , Katie HortonKatie Horton Search for more papers by this author , Nancy M. AlbertNancy M. Albert Search for more papers by this author , Craig BeamCraig Beam Search for more papers by this author , Robert O. BonowRobert O. Bonow Search for more papers by this author , Roger L. (Vern) DavenportRoger L. (Vern) Davenport Search for more papers by this author , Meighan GirgusMeighan Girgus Search for more papers by this author , Gregg C. FonarowGregg C. Fonarow Search for more papers by this author , Harlan M. KrumholzHarlan M. Krumholz Search for more papers by this author , Mark W. LegniniMark W. Legnini Search for more papers by this author , William R. LewisWilliam R. Lewis Search for more papers by this author , Graham NicholGraham Nichol Search for more papers by this author , Eric D. PetersonEric D. Peterson Search for more papers by this author , Wayne RosamondWayne Rosamond Search for more papers by this author , John S. RumsfeldJohn S. Rumsfeld Search for more papers by this author , Lee H. SchwammLee H. Schwamm Search for more papers by this author , David M. ShahianDavid M. Shahian Search for more papers by this author , John A. SpertusJohn A. Spertus Search for more papers by this author , Pamela K. WoodardPamela K. Woodard Search for more papers by this author , Clyde W. YancyClyde W. Yancy Search for more papers by this author and Search for more papers by this author and on behalf of the American Heart Association Advocacy Coordinating Committee Originally published11 Apr 2011https://doi.org/10.1161/CIR.0b013e3182181529Circulation. 2011;123:2167–2179Other version(s) of this articleYou are viewing the most recent version of this article. Previous versions: January 1, 2011: Previous Version 1 Clinical registries play an important role in measuring healthcare delivery and supporting quality improvement for individuals with cardiovascular disease and stroke. Well-designed clinical registry programs provide important mechanisms to monitor patterns of care, evaluate healthcare effectiveness and safety, and improve clinical outcomes. The use of clinical registries is likely to grow given the increasing focus on measuring and improving healthcare delivery and patient outcomes by stakeholders in both the private and public sectors.The American Heart Association (AHA) has a longstanding commitment to promoting the innovative and effective use of clinical registries. The importance of clinical registries was highlighted recently in an AHA Scientific Statement on “Essential Features of a Surveillance System to Support the Prevention and Management of Heart Disease and Stroke” in the United States.1 This policy statement expands on the previous scientific statement by providing recommendations to policy makers and the healthcare community for expansion of the applications of existing and future clinical registries.The term “clinical registry” is defined here as an observational database of a clinical condition, procedure, therapy, or population in which there are often no registry-mandated approaches to therapy and relatively few inclusion or exclusion criteria. The focus of clinical registries is to capture data that reflect “real-world” clinical practice in large patient populations. The data from clinical registries do not replace the need for traditional randomized controlled trials. Rather, registries and trials are complementary approaches, each with unique advantages and imperfections.2 Such clinical registries do not solely contain claims or administrative data yet may be linked to such data sources.There are at least 3 classifications of clinical registries based on the patient population, including procedure/therapy/encounter-based, disease-based, and population-based registries. Registries also can be classified from a functional perspective, such as whether the registry is used to conduct clinical research, to perform quality measurement, or to provide feedback to clinicians for quality improvement. Such uses are not mutually exclusive. Although clinical registries can be either prospective or retrospective in design, this policy statement focuses on prospective clinical registries.The potential uses of clinical registries have captured the attention of policy makers at the federal level. This has resulted in changes in the substance and scope of roles that clinical registries play within the US health system. Some examples include the use of registries to support safety and comparative effectiveness research of marketed drugs, devices, or treatment strategies; to gather data on new therapies granted provisional coverage by Medicare under Medicare's coverage with evidence development initiative; or to assess the reliability and validity of potential performance measures for use by public payers, private payers, and accreditation organizations. Clinical registries are developed and operated by many types of entities, including researchers, research consortiums, nonprofit organizations, government agencies (eg, the National Institutes of Health), and industry.Clinical registries also provide the opportunity to identify and evaluate healthcare disparities within a broad patient population in community practice outside of the structured research protocol setting. This promotes the ability to examine important issues involving patient access and outcomes in subpopulations, including racial and ethnic minorities, women, the elderly, individuals with multiple comorbidities, and individuals with congenital heart conditions.The potential value derived from investing resources in clinical registry development is demonstrated by national clinical cardiovascular disease and stroke registries that have matured over the past decade. Prominent examples include the AHA's Get With The Guidelines (GWTG) registries for coronary artery disease (GWTG-CAD), heart failure (GWTG-CHF), stroke (GWTG-Stroke), and outpatient care (GWTG-Outpatient); the Interagency Registry for Mechanically Assisted Circulatory Support; the Centers for Disease Control and Prevention's Paul Coverdell National Acute Stroke Registry; the AHA's National Registry of Cardiopulmonary Resuscitation; the Society of Thoracic Surgeons National Database, which includes the Society of Thoracic Surgeons Adult Cardiac Surgery Database; the American College of Cardiology Foundation National Cardiovascular Data Registry (NCDR) programs, including the ICD Registry for implantable cardiac defibrillators; the Carotid Artery Revascularization and Endarterectomy Registry; the CathPCI Registry for diagnostic cardiac catheterizations and percutaneous coronary interventions; the Practice Innovation and Clinical Excellence (PINNACLE) registry for quantifying and improving the quality of outpatient care; and the ACTION Registry–GWTG for acute myocardial infarction.Considerations in Establishing Clinical Registries and Interpreting Registry DataMany of the opportunities and challenges associated with clinical registry implementation arise from the fact that clinical registries typically serve multiple functions. Examples include public health surveillance, quality improvement, evaluation of temporal trends in care and outcomes, monitoring for drug or device safety and efficacy (including postmarketing surveillance), evaluation of access to medically necessary care (including underuse, overuse, and misuse), and the assessment of clinical effectiveness, cost-effectiveness, and value.Clinical registries must be designed with sufficient safeguards, rigor, and transparency to ensure that the primary functions of the registry are well served. Regardless of the purpose, data collection must be of high quality to avoid erroneous conclusions. Examples of threats to a registry's validity include selected and incomplete patient sampling, ambiguous data definitions, and high rates of missing data. Many of these considerations have been described extensively in a comprehensive guidance document published by the Agency for Healthcare Research and Quality (AHRQ) in 2007, which will be updated in the near future.3 Although it is not within the scope of the present policy statement to reiterate these practices, the AHRQ-published document provides important guidance about registry planning, registry design, data elements and sources, data quality procedures, and the uses of registries. Additional information is provided in the Appendix.In this policy statement, the AHA provides guidance to policy makers and the healthcare community on navigating the challenges inherent in establishing and maintaining existing and future clinical registries. Emphasis is placed on the opportunities and challenges associated with cardiovascular disease and stroke, including short- and long-term prevention, diagnosis, treatment, and rehabilitation of individuals with these diseases. The discussion and recommendations are divided into the following 5 categories: Ensuring high quality dataLinking clinical registries with supplemental dataIntegrating clinical registries with electronic health recordsSafeguarding privacy while reducing barriers to healthcare improvementSecuring adequate funding and developing business models to initiate and sustain clinical registriesEnsuring High-Quality DataBecause the usefulness of any clinical registry depends directly on the quality of the data it collects, ongoing quality-monitoring procedures that characterize data validity and reliability and improve and maintain data quality are critical. Limitations in the quality of the data collection markedly undermine the potential uses and applications of a registry.A particularly important aspect of data quality is the degree to which the population targeted by the registry is accurately represented. Regardless of the use of a registry, an understanding of the composition of the population is important. The quality of the registry depends on consistent adherence to the explicit inclusion, exclusion, and sampling rules of the registry. For example, a registry may establish inclusion and exclusion criteria, but individual sites may apply these rules inconsistently. Registries need to assess and disclose the level of adherence to the intended entry criteria. To the extent possible, comparisons of the included and excluded populations with data sources common to both populations can be useful in gauging representativeness.Gauging the representativeness of clinical registry data is also critical in ongoing efforts to identify and address disparities in our healthcare system. Clinical registries can and should play a central role in both the inclusion of adequate numbers of patients who traditionally have been underrepresented in the scientific literature (including racial and ethnic minorities, women, the elderly, and socioeconomically disadvantaged individuals) and the capture of meaningful data for these subpopulations of patients. If necessary, oversampling can be performed to ensure adequate numbers of traditionally underrepresented populations. To ensure data quality, clinical registries can also play a leadership role in the collection of data on sex, race, ethnicity, language, and other important indicators using standardized definitions for such data.Data completeness is another important component of registry quality. Incomplete medical record documentation of key elements (eg, medical history, laboratory data, differential diagnosis) is common in both hospital and ambulatory settings.4,5 Data completeness also may be a function of the extent to which collection is integrated with clinical care, the training and consistency of those who enter data, and the extent to which data elements are required. Case completeness issues may arise from errors made when records are reviewed manually or when automated coding efforts are undermined by changing data definitions within electronic medical records. In a particularly egregious example, an analysis of 78 practices participating in a disease registry demonstrated 100-fold differences in the rates for recording relevant data, including missing diagnostic codes in patients who received treatment and infrequently recorded data related to the primary purpose of the registry.6 Missing data are a particular challenge because one cannot assume that data are missing randomly. Efforts to handle missing data statistically are complex and cannot specifically address data not missing at random. The approach of removing cases from an analysis for missing data elements may result in bias or decreased generalizability of findings.7 For these reasons, efforts to minimize missing data are critical to minimizing bias.Even when skilled abstractors are instructed effectively on data collection, source documents may be absent, incomplete, or contradictory because of the large number of healthcare providers involved in documentation and inconsistencies in recording. There are numerous other sources of abstraction errors, including failure to meet the expectation of consecutive case inclusion,8 inconsistent coding, lack of common data elements or inappropriate application of definitions, ambiguous data definitions, poor layout of data collection forms, use of nonvalidated scales (ie, those that have not undergone psychometric testing for validity and reliability), haphazard adjudication of complex data, and insufficient systems for follow-up data collection.9Many of these threats to data quality, including variability in case completeness and data accuracy, may be overcome with careful registry planning and design, training of personnel, and mechanisms to assess and improve data quality. In 1 study, case completeness and data accuracy improved with interventions to ensure adequate training of staff, use of supplementary source reporting, conformity with published standards such as those developed by the AHA and American College of Cardiology Foundation, and achievement of national certification.10 Additional methods for ensuring data accuracy and consistency include the use of site visits, chart reviews, clarifications of all discrepancies, core laboratories, and critical events committees. Such steps help ensure uniformity in definitions and high-quality data, although these safeguards may be cost prohibitive, especially in the case of registries for large patient populations.When data definitions are not explicit or response options do not encompass the entire spectrum of potential choices, variability in data interpretation is more likely to occur.11–13 Paper data collection systems that require secondary electronic data entry are predisposed to data entry errors that may be avoided with a direct electronic data collection system that automatically performs checks for consistency, plausible value ranges, and missing values. Nonetheless, errors also occur in systems with direct data entry. Explicit, comprehensive, and interpretable data standards; primary electronic data entry; and automated data integrity assessment are all likely to enhance the quality of registry data.Over the past decade, some large clinical registries published reports that addressed data quality and variability. However, because there are no standard requirements for such reporting, there is possibly unrecognized bias within other registries that could undermine the reliability, credibility, and representativeness of registry reports.Some registries have reported good interrater reliability between registry and medical records data.10,13,14 However, even when overall agreement for a particular measure of quality may be at least moderate (ie, κ-scores ≥0.60), there may be some data fields with poor reliability.15 For example, in an acute stroke registry with good overall interrater reliability for quality of care metrics, several individual data fields demonstrated poor reliability, including stroke onset time, stroke team consultation, time to initial brain imaging, and discharge destination.14 Recommendations to improve data recording of symptom onset time may improve the quality of this traditionally poorly recorded variable.16 Furthermore, one would expect greater variability for data that reflect a subjective measure than for data on objective measures.Thus, although an individual report from clinical registry data may provide only an overall κ-value that summarizes all performance measures, publication of a detailed audit of registry data that specifies the major fields used in the analysis is necessary to substantiate clinical registry quality and to identify areas for improvement in data quality. Information about the accuracy of specific data fields is necessary to improve the quality of submitted data, as well as the registry's overall quality.Furthermore, because of their observational nature, the data generated by clinical registries must be interpreted with an understanding of the limitations that arise from treatment selection bias when alternative forms of therapy are compared (such as in the case of comparative effectiveness research). Advanced statistical methods often can help diminish the adverse effects of treatment selection bias, and sensitivity analyses may be useful. However, in many cases, there are no statistical techniques that completely eliminate treatment selection bias, and no statistical technique can account for confounders that either are not measured or are measured inadequately in the data collected for the registry.Publications are available that provide an overview of steps used in database audits,3,17 and other reports provide methods used in quality control and statistical analysis; however, there are no written standards that have been described and evaluated to determine the effectiveness of attempts to ensure registry quality. Furthermore, the minimum acceptable standard of clinical registry data quality (overall and individual data fields) has not been specified beyond a requirement for statistical agreement between data abstracted manually from patient medical records and registry data.Linking Clinical Registries With Supplemental DataThe value of linking population-based clinical registries with supplemental data is substantial, and such linking contributes to a wide range of important functions, including public health surveillance, clinical research, quality improvement, and monitoring of patient safety. The linking of clinical registry data to other data sources allows researchers to leverage existing data to create a linked clinical-longitudinal database that capitalizes on the strengths of both types of data sources. This is increasingly important at a time when there is otherwise little infrastructure to answer important safety, clinical efficacy, comparative effectiveness, and other clinical questions for large cardiovascular and stroke patient populations in real-world clinical settings. The linking of clinical registries with supplemental data also provides the potential ability to examine outcomes for smaller subpopulations, including assessment of disparities for racial and ethnic minorities and other groups that may otherwise be underrepresented.The linking of clinical registry data with claims data for longitudinal follow-up has many advantages over direct longitudinal clinical follow-up. Obtaining data on clinical events such as hospitalization and vital statistics from existing claims sources is far more efficient and may be more complete than direct data collection.18 Longitudinal data on hundreds of thousands of patients can be linked and analyzed in a highly efficient fashion.19–21 To the extent that the process can be based on anonymous identifiers, a data linkage system would likely be more complete than a system that requires hospitals and patients to opt-in for collection of longitudinal follow-up data.22,23 The resulting linked database would not be subject to participation-based selection biases that have posed challenges for other registries or data sets that require informed consent or that have lost patients to follow-up.22,23 Potential limitations of such administrative databases are significant and include the general absence of clinical data (eg, blood pressures) or patient-centered assessments (eg, symptoms, health status, psychosocial status, quality of life) and the systematic exclusion of those who are not insured by the plan that administers the data source, thus systematically excluding individuals without formal access to care.A particular challenge in data linkage is the availability of unique direct patient identifiers (eg, Social Security numbers).18 Without such direct identifiers, it may still be possible through matching algorithms to create a high-quality link between inpatient clinical registry data and claims data (so-called probabilistic matching).18,22,24 A number of studies have validated the use of indirect identifiers to link healthcare databases (eg, a combination of data elements such as date of birth, sex, and date of healthcare encounter). It is essential to ensure there is adequate privacy and security protection for personal health information when data sets are linked, because the reidentification of previously deidentified records has occurred.25Methods have been developed to identify records from clinical registries by use of Medicare inpatient claims data through indirect identifiers (eg, admission date, discharge date, patient age, date of birth).18,24,26 This method takes advantage of the hospital clustering observed in each database by demonstrating that different combinations of indirect identifiers within hospitals yield a large proportion of unique patient records.18 This high level of uniqueness allows linking without advance knowledge of the provider number of each registry hospital.18 Once such records have been linked to a data source that includes a patient identifier, additional identified data sets can then be linked as well.However, such linkages through probabilistic matching inevitably result in the loss of some of the sample because of the inability to match individuals in both data sources. In addition, the use of indirect identifiers to link databases may result in some proportion of erroneous links. Although it is unlikely that incorrect links would introduce a large amount of systematic bias such biases are difficult to assess.18,22 Further investigations of these biases by comparison of the results obtained by direct linkage with those in the same data sets using indirect linkages will be useful in further delineating the potential limitations of the indirect approach.Although the use of supplemental data for longitudinal follow-up or evaluation of other vital information has several advantages, it also has important limitations. For example, Medicare claims exist only for patients ≥65 years of age and patients with other qualified coverage in this fee-for-service sector.18–21 State-level all-payer claims files, Medicaid data, and major private insurer databases provide alternatives for linking patients ≥65 years of age. Specific data sources that could prove useful include the American Medical Association's Physician Masterfile, the Social Security Death Index, and the US Census. However, each of these data sources covers only select groups of patients. The assembly of a data set with complete follow-up in all patients would require the integration of multiple sources of supplemental data.Data linkages between clinical registries and longitudinal claims databases provide opportunities to assess outcomes such as mortality, readmission, and subsequent procedures. Likewise, outpatient, community, and prehospital clinical registry data linked to hospital data can facilitate evaluation of in-hospital diagnostic studies, treatments, procedures, and hospital-based quality measures. Such linked data sets may allow researchers to answer questions about long-term safety and efficacy of treatments and about the relative importance of prehospital, hospital, and outpatient processes for patient outcomes.A number of important issues must be considered in the interpretation of the findings from data sets composed of registry data linked with supplemental sources. The linked database will be limited by the biases contained in all of the component sources. There are potential selection biases inherent in clinical registries, unless consecutive patients are enrolled and the participating centers are representative of the nation as a whole. This may limit inferences that can be made about incidence of the disease state, cardiovascular events, or procedures and the patient population in general.Biases can also be introduced when dealing with missing data or missing links. Methods are available for imputing missing data, but more work is needed to develop and evaluate methods that address the special case of missing data in linked data sets from multiple sources. If linkage is available only in a certain subgroup of the patients enrolled in the clinical registry, this can introduce a population-selection bias not previously present in the clinical registry.Although the voluntary, nonrandom participation associated with clinical registries linked to supplemental data does not necessarily limit generalizability, the representativeness of the clinical registry linked to supplemental data must be established on a case-by-case basis.22 In evaluation of linked data sets, factors to consider include the completeness and accuracy of the linkages; the completeness, quality, and accuracy of the parent clinical registries; and the completeness, quality, and accuracy of the linkage data set. Rigorous standards need to be developed for the selection, evaluation, and interpretation of linkage data sets.3,27–31Integrating Clinical Registries With Electronic Health RecordsThe electronic health record (EHR) systematically collects health information about individual patients. Because it uses a digital format, an EHR can track patients across the continuum of care and collect data for national clinical registries to promote the study of practices and methods for improving health care for various populations.There are multiple potential benefits of integrating EHRs with clinical registries. First, clinical registries can help define data collection within EHRs that is meaningful, accurate, and actionable. More specifically, registries can guide the specification of which data are critical to capture in EHR formats (eg, categorical rather than narrative text) to directly support evaluation of care delivery and patient outcomes. The specification of data includes data elements and data definitions, as well as specification of data collection (eg, ranges) to enhance accurate and standardized data collection.Second, registries can serve a critical role with regard to the benchmarking of and feedback on care and patient outcomes based on data obtained from EHRs. A primary role of national clinical registry programs, such as GWTG and the NCDR, is to assess quality of care and provide benchmark reports to registry participants. Only with such benchmarking can providers, practices, and hospitals know where they may have gaps in care delivery compared with others, which permits tracking in relation to the nation over time. Finally, the transfer of data from EHRs to registries integrates data collection into clinical care, which reduces the time needed for data collection and potentially increases the reliability and completeness of data elements.Investigators and their institutions using EHRs that supply data to national clinical registries should adhere to the data specification, evaluation, benchmarking, recognition, and reporting standards developed by national clinical registries. There are multiple reasons why participation in national clinical registries is superior to having EHRs act independently. Given the number of EHR vendors and the customization of EHR interfaces within vendors, existing data structures are highly heterogeneous, which limits the validity of comparisons between EHRs. Furthermore, because of their commercial interests, EHR vendors typically strive to differentiate their products rather than seek compatibility with others. Collection of data according to the specifications of national clinical registries ensures comparable, valid data.There is also a risk that EHRs will not keep pace with current c