Ecological Studies and Cardiovascular Outcomes Research

Abstract

HomeCirculationVol. 118, No. 24Ecological Studies and Cardiovascular Outcomes Research Free AccessResearch ArticlePDF/EPUBAboutView PDFView EPUBSections ToolsAdd to favoritesDownload citationsTrack citationsPermissions ShareShare onFacebookTwitterLinked InMendeleyReddit Jump toFree AccessResearch ArticlePDF/EPUBEcological Studies and Cardiovascular Outcomes Research Jack V. Tu, MD, PhD and Dennis T. Ko, MD, MSc Jack V. TuJack V. Tu From the Institute for Clinical Evaluative Sciences, Toronto, Ontario, and Division of Cardiology, Schulich Heart Centre, Sunnybrook Health Sciences Centre, University of Toronto, Canada. Search for more papers by this author and Dennis T. KoDennis T. Ko From the Institute for Clinical Evaluative Sciences, Toronto, Ontario, and Division of Cardiology, Schulich Heart Centre, Sunnybrook Health Sciences Centre, University of Toronto, Canada. Search for more papers by this author Originally published9 Dec 2008https://doi.org/10.1161/CIRCULATIONAHA.107.729806Circulation. 2008;118:2588–2593Is it better to have a heart attack in the United States or Canada?Ecological studies are epidemiological evaluations in which the unit of analysis is populations, or groups of people, rather than individuals.1 They can answer important questions such as the question posed above that cannot easily be answered using any other form of study design. Ecological studies are conducted by researchers in a variety of disciplines, including outcomes researchers, to study potential causal associations between 1 or more exposure and outcome variables. They are frequently used where alternative study designs are not possible (eg, randomized control trials), such as when investigating the effect of geographical and temporal factors on disease incidence or the effect of a government policy change on health outcomes. In this overview, we provide an introduction to the use of ecological studies in cardiovascular outcomes research, drawing on published examples from our group and others. It is important for clinicians to be aware of both the potential strengths and weaknesses of this type of study design. Because ecological studies are ubiquitous in the medical literature, it is not possible to review all published cardiovascular ecological studies in a systematic manner.Individual-level variables are properties of each person whereas ecological variables are properties of groups, organizations, or places.1,2 All ecological studies are potentially prone to the so-called “ecological fallacy,” which is a term for the biases that may occur when the association that exists between variables at an aggregate level may not represent the true association that exists at an individual level.3 On the other hand, the ecological fallacy may be offset by avoidance of the “individualistic fallacy” whereby one assumes that individual patients are unaffected by the neighborhood in which they live or the setting in which they are treated.4 Complete ecological studies involve studies where all of the variables are ecological in nature, whereas partial ecological studies involve a mixture of individual-level and ecological variables.1 Some epidemiologists argue that ecological studies should be used purely as hypothesis-generating exercises that require further confirmation through other epidemiological study designs such as randomized clinical trials or cohort studies using only individual-level variables.2 An opposing view holds that although such cautions are appropriate, it may never be possible to conduct individual-level studies to answer certain questions and that if strong and biologically plausible effects are observed in an ecological study, the causal inferences may be sufficiently strong to warrant appropriate policy or clinical practice changes.2To assist readers in evaluating ecological outcomes research studies, we propose a series of questions that can be asked to evaluate their quality, as shown in Table 1. The criteria proposed in Table 1 are not necessarily exclusive to ecological studies but also may apply to other observational studies. Excellent ecological studies do not necessarily have to satisfy all the identified criteria, and readers should exercise their judgment when reviewing these types of studies, in terms of determining what the investigators could have practically done in terms of answering a given research question. In the present article, we discuss several examples of ecological outcomes research studies that have been previously published by our group and others and address both the insights that were gained from these studies and their limitations in light of the methodological issues raised in Table 1. Table 1. Questions That Should Be Considered in Evaluating the Quality of an Ecological StudyWould it be practical to conduct alternative ways of studying the same question (eg, randomized control trial) or was the ecological study the only alternative?Are the subjects in the ecological study representative of the group, place, or population of interest?Were the exposure and outcome variables measured and defined in the same or a similar way across the different populations or groups that are being studied?Have data been collected on important confounding variables that might also explain the exposure–outcome relationship and have they been statistically adjusted for? If data are not available on key factors, is it reasonable to assume that their prevalence is similar in the different groups or populations being compared?Is the identified ecological relationship between the exposure and outcomes biologically plausible and consistent which what is already known about a given topic at an individual-subject level?What is the strength of the quantitative and statistical associations between the exposure and the outcome? The stronger the associations, the greater the likelihood of a true causal relationship.Have the investigators interpreted their data with appropriate caveats? Did they acknowledge the possibility of an ecological fallacy? Were alternative explanations for the association between exposure and outcomes considered by the investigators?Have the study data been collected at multiple levels (eg, individual, physician, hospital, community, or country)? If yes, was multilevel modeling considered or used for analyzing the data?Example 1: Ecological Studies of Care in Different Healthcare SystemsA common type of ecological study design involves comparison of treatment patterns and outcomes of patients living in different geographical regions or countries; such studies attempt to make inferences about the quality of care provided by the respective healthcare systems. In the field of cardiology, multiple high-profile studies have been published comparing the quality of care and outcomes in patients hospitalized with an acute myocardial infarction (AMI) in the United States and Canada.5–9 These studies have generally used a study design in which individual-level data from disease cohorts have been aggregated so that the entire analysis is performed at the level of the ecological variable, the patient’s country of residence. Patient-level characteristics such as age and sex are no longer taken as attributes of individuals, but rather the mean age and sex ratios are calculated at the country level.These studies have certain strengths. Considerable debate exists about the advantages and disadvantages of a Canadian single-payer universal healthcare system as a potential alternative model for the United States, which has a more expensive, multipayer system in which millions are uninsured.10–12 Whereas the ideal way to study the potential impact of the adoption of a single-payer healthcare system would be to randomize citizens in different parts of the United States to different types of insurance coverage, such a study would likely be impossible to carry out. An ecological study of people living under different insurance systems is a practical alternative option for studying this important and complex policy issue. Comparing the treatments and outcomes for patients receiving care for common medical conditions has the potential to inform policy makers and the public in both countries about the strengths and weaknesses of each system. AMI is an ideal condition to study because it is a common cause of morbidity and mortality in both the United States and Canada. Both inexpensive and expensive evidence-based therapies are available for the treatment of patients, and the utilization of such therapies depends in part on the health system in which a patient is being treated.Studies comparing the care and outcomes between the United States and Canada have included patients in selected hospitals,13 clinical trials,5,6,8,14 clinical registries,9,15and administrative databases (Table 2).7 Although not all studies have shown the same results, some common findings have emerged from these studies. The most consistent and striking finding is that AMI patients in the United States are much more likely to receive cardiac catheterization and revascularization procedures after an AMI than are patients treated in Canada.5–9,13,14 This discrepancy appears to be related predominantly to the presence of fewer cardiac invasive facilities in Canada, rather than to differences in illness severity.9 In spite of marked differences in procedural access, most studies have shown that the short-term and long-term mortality rates after an AMI within the 2 countries are similar,5,7,9,13,14 leading to the conclusion that some of the cardiac procedures performed in the United States may be unnecessary. However, a few studies have also suggested that angina and quality of life may be improved with the more aggressive strategy used in the United States.5,6,13 There remains uncertainty about whether this finding is due to the procedures themselves or to other unmeasured confounding factors. Table 2. Ecological Studies Comparing the Care and Outcomes of AMI Patients in the United States and CanadaFirst AuthorYearTypes of DataData SourcesTotal No. of AMI PatientsIn-Hospital Angiography Rates in the US and Canada, %Clinical OutcomesTIMI indicates Thrombolysis in Myocardial Ischemia.*30-day angiography rates.Rouleau (5)1993Clinical trialSAVE study223168 vs 35No significant difference in myocardial infarction or death rates at median of 42 months follow-up. Less frequent activity-limiting angina in the USMark (6)1994Clinical trialGUSTO 1 economics and quality of life substudy300072 vs 25Functional status similar at 30 days but less frequent chest pain and dyspnea in the US at 1 yearPilote (13)1994Hospital databasesSingle hospital in California, United States and single hospital in Quebec, Canada51853 vs 34No significant difference in myocardial infarction or death rates. Better functional status in the USTu (7)1997AdministrativeHospitalized elderly AMI patients in the United States and Ontario, Canada233 70235 vs 7*Mortality rates lower in the US at 30 days but no significant difference at 1 yearAnderson (15)1997Clinical registryTIMI III47546 vs 67No significant difference in myocardial infarction or death rates at 1 yearFu (14)2000Clinical trialGUSTO 2b141081 vs 42No significant difference in myocardial infarction or death rates at 1 year.Kaul (8)2004Clinical trialGUSTO 126 00373 vs 31Lower mortality rates in the US at 5 years follow-upKo (9)2007Clinical registryNational Heart Care Project in the US and EFFECT study in Ontario, Canada44 52039 vs 17Lower mortality rates in Canada at 3 years of follow-upSeveral of these United States–Canadian studies have used cohorts of AMI patients enrolled in landmark clinical trials such as the Global Utilization of Streptokinase and t-PA for Occluded Coronary Arteries (GUSTO) trials6,13,14 and the Survival and Ventricular Enlargement (SAVE) study.4 The primary advantage of using these clinical trial cohorts is that detailed high-quality data are collected in a virtually identical manner on the characteristics, treatments, and outcomes of patients who meet strict trial inclusion/exclusion criteria, so that one may have a high degree of confidence that these studies are comparing generally similar patients and that the statistical adjustments for confounding variables are based on high-quality data. The primary limitation of these studies is that patients in clinical trials are generally healthier than those found in routine clinical practice, and so it is uncertain whether one can reach any definitive conclusions about the merits of a healthcare system based on a very highly selected subset of the patients treated within it.16–18 In general, it is recommended that ecological studies be conducted on a sample that is as representative as possible of all patients in a geographical region or group of interest.To overcome this limitation, some studies have used large population-based administrative database cohorts of patients hospitalized in the 2 countries to examine rates of cardiac procedures and the associated outcomes after an AMI.7 One study by our group found large differences in cardiac procedure rates but similar 1-year mortality rates in the 2 countries.7 This type of study has an advantage in that patient selection biases are minimal because the study uses information on all patients treated within a given country or region for an AMI. On the other hand, these types of studies do not provide much insight into patient differences or treatment differences (other than differences in cardiac procedures) that could vary between countries, and thus if any outcome differences are found, investigators can only speculate on the potential causal factors. Another key limitation of this type of administrative data–based ecological study is that very limited data are available on confounding factors (eg, type and severity of infarct or cardiac patient risk profile) that might also influence patient outcomes. In these situations, readers need to make a judgment as to whether the distribution of unmeasured confounding factors might reasonably be expected to be similar between the different groups being compared.A third alternative for an ecological study of system contrasts is the use of patients enrolled in a clinical registry.9,15 The best clinical registries are based on unselected “real-world” patient populations and have detailed information on patient characteristics, treatment patterns, and relevant clinical outcomes such as mortality. Our group recently published a study comparing AMI patients in the Enhanced Feedback for Effective Cardiac Treatment (EFFECT) project from Ontario, Canada, with Medicare AMI patients hospitalized in the United States.9 In both registries, the patients were a random sample of the larger population of AMI patients in the respective countries.9 Both of these registries included detailed clinical information. However, national privacy laws prohibited the pooling of the data in a single location, which would have been the best method of analysis. To overcome this limitation, we used the validated acute coronary syndrome risk score from the Global Registry of Acute Coronary Events (GRACE) registry to calculate the predicted cardiac risk of AMI patients and then calculated a risk-standardized mortality rate for patients in both countries.9,19 Somewhat surprisingly, this study showed that long-term mortality was significantly better in elderly Canadian AMI patients. However, the lack of comparable outpatient follow-up data precluded us from determining the reason for these differences.As demonstrated in these previous investigations, ecological studies of different healthcare systems can offer policy makers and the public important insights into the patterns of care and outcomes of patients with similar conditions in different countries. Even though it is tempting to extrapolate from these studies to make broader “political” conclusions about the merits or lack thereof of different healthcare systems, the primary reason for these studies should be to uncover potential opportunities for improving the quality of care and patient outcomes in different healthcare systems. It should be noted that no perfect ecological study is possible, and the strength of the conclusions that can be reached from a given ecological study are stronger if additional studies report similar results. Because so many determinants exist of patient outcomes such as mortality, it is important for researchers to gather as much data as possible on patient characteristics and treatments when conducting these studies and to adjust for them in a statistically rigorous manner. One should always keep in mind, however, that unmeasured confounding variables may play a role and that investigators are often not able to collect data on all important confounding factors in ecological studies. This limitation is less of a concern when one might reasonably assume that the frequency of these unmeasured factors is similar in the different groups being compared.Example 2: Ecological Studies of Geographical Differences in Cardiovascular Mortality RatesOutcomes researchers have observed geographical differences in both the incidence of and the death rates from different diseases using large population-based databases. They have then worked backwards to collect a series of potential explanatory variables in an effort to determine the causal reasons for such geographical variations. One recent example of a large national ecological study designed to explore the reasons for geographical differences in cardiovascular mortality rates was published as part of the “Canadian Cardiovascular Atlas” project of the Canadian Cardiovascular Outcomes Research Team (CCORT).20 In part of this project, researchers from our team initially analyzed data on cardiovascular mortality rates in 10 Canadian provinces and 139 health regions across Canada during the 1995 to 1997 time period.20 In general, cardiovascular mortality rates followed an “East to West” gradient in which we observed the highest mortality rates in Eastern Canada, in the province of Newfoundland, and the lowest mortality rates in Western Canada, in the province of British Columbia. Mortality rates also varied widely across health regions, with several regions identified as “hot spots” where the death rates relative to the national average were particularly high (ie, >30% above the national average).20To try and understand why these mortality differences existed, we then analyzed data collected from across Canada on over 130 000 Canadians participating in the Canadian Community Health Survey (CCHS).20 The CCHS is a pan-Canadian telephone survey of the general Canadian population with an 85% response rate and a sample size that enables the creation of 14 ecological variables at the health region level, including 5 traditional cardiovascular risk factors (eg, smoking rate, hypertension rate, diabetes mellitus rate, obesity rate, and physical activity rate), social determinants of health variables (eg, low income rates and unemployment rates), and several community-level characteristics (eg, proportion of immigrants). We conducted an analysis in which we regressed each of these ecological variables simultaneously against the death rates in each health region across Canada.20 We found that 2 variables alone (the smoking rate and the obesity rate in a heath region) were the strongest determinants of geographical variation in cardiovascular death rates in Canada and that the traditional cardiac risk factors explained 42% of the total variation.20 Other ecological variables such as the unemployment rate in a health region were also associated with death rates, but their incremental explanatory power was limited. These findings were broadly disseminated in Canada via a national CCORT Canadian Cardiovascular Atlas, available free of charge on the Web (www.ccort.ca/atlas.asp).This type of ecological study illustrates both the potential power of ecological studies and their limitations. We identified a series of individual-level variables that have been found in observational cohort studies to be associated with an increased risk for cardiovascular disease and then created a series of ecological variables from a national health survey to study the causes of cardiovascular disease death rate variations at a health region level. Understanding the key causes of the regional variations was particularly important to communities in Canada with the highest death rates that wanted to know what should be done to improve their situation. Our study provides some evidence, albeit cross-sectional and ecological, that strategies to prevent cardiovascular disease at a community level and reduce regional disparities in outcomes may need to go beyond individual-level strategies and to consider community characteristics such as smoking bylaws, cigarette prices, and physical activity policies in schools, because the smoking and obesity rates in a community were the strongest determinants of its cardiovascular mortality rates. Interest is increasing in understanding how the “built environment” (eg, access to parks and recreation, sidewalks, and neighborhood grocery stores) influences the development of cardiovascular risk factors, and ecological studies will likely play a central role in this type of research.21On the other hand, this study has its limitations, and it may be that factors other than smoking and obesity rates are the most important determinants of variations in cardiovascular death rates across communities. For example, it is possible that other unmeasured confounding factors (eg, environmental factors such as pollution levels or genetic differences) explain some of the observed regional variations. Many of the variables in our study are correlated with each other (eg, socioeconomic status and poorer cardiovascular risk profiles), and we cannot be certain which came first in the causal pathway on the basis of an ecological regression analysis alone. Because so many potential determinants of cardiovascular mortality rates exist, including patient-, physician-, hospital-, and community-level variables, it is difficult to identify with certainty all of the causes of the regional variation in cardiovascular mortality, and additional follow-up studies should be considered to confirm hypotheses generated by this type of study. For example, a follow-up study to confirm the findings in our original Atlas study might be to randomize different communities to different types of community-level cardiovascular prevention strategies focused on smoking, obesity, or both.Example 3. Ecological Studies of Socioeconomic Status and Cardiovascular DiseaseIt is increasingly recognized that the broader determinants of health such as an individual’s socioeconomic status and educational level may have a profound influence on his or her health status and longevity.4,22–25 However, gathering data on a patient’s socioeconomic status may be a sensitive issue and may vary dramatically depending on the life course of an individual. A retired individual might have a low annual income yet have a high socioeconomic status based on income accumulated over his or her entire lifetime. To overcome these challenges, many researchers have conducted ecological studies of socioeconomic status and cardiovascular outcomes by making the assumption that the neighborhood in which one lives is an accurate reflection of one’s socioeconomic status.4,22 A common way in which the socioeconomic status variable is created is by determining a patient’s zip code or postal code (the equivalent used in Canada) and then linking that code to information from governmental census data on the median household income within a given small geographical area.This type of ecological study is one in which the analysis can be conducted at the level of individual, but an ecological variable (socioeconomic status) is attached to each individual. It makes the assumption that all those living within a given small geographical area have a similar socioeconomic status, an assumption that may or may not be true. Validation studies in which income data have been collected at the individual-patient level have shown variable correlations when compared with the patient’s socioeconomic status as estimated using an ecological variable based on their geographical location of residence.26,27 When the sample sizes are large, potential misclassification of the individual’s socioeconomic status on the basis of the socioeconomic status of the neighborhood is likely offset by the avoidance of the assumption that that individual patients are unaffected by the neighborhood in which they live.4One well-known ecological study was conducted by Alter and colleagues from our group examining the relationship between a patient’s socioeconomic status, along with his or her access to cardiac procedures (ie, cardiac catheterization, angioplasty, and bypass surgery), and mortality after an AMI in Ontario, Canada.4 This study used large, linked, population-based administrative databases of AMI patients in Ontario and attempted to infer patients’ socioeconomic status by linking information on their postal code of residence to Canadian census data information to determine the median income of households in that given small geographical area. Patients were categorized into 5 income quintiles from the lowest to highest. In spite of the universal healthcare system in Canada, which attempts to minimize inequity, we found that patients residing in higher-income neighborhoods have the highest rates of cardiac procedure use and the lowest 1-year mortality rates.4Although this study provided a strong case for an “income–outcome gradient,” it was not possible to determine the causal factors for such a relationship because this study was based on linked administrative databases with limited clinical data. Follow-up studies by Alter and colleagues in Ontario with detailed individual-level clinical and socioeconomic status data showed that much of the AMI mortality gradient was mediated by differences in patients’ baseline cardiovascular risk profiles rather than discrepancies in processes of care.23 This finding has great importance for policy makers because it suggests that universal healthcare by itself cannot eliminate health disparities. Rather, aggressive management of known cardiac risk factors and promotion of healthy behavior such as smoking cessation and healthy eating may be most important among the socially disadvantaged. It also highlights the synergy between study types, whereby findings observed in an ecological study can stimulate an individual-level study that determines the primary mechanisms for the findings in the ecological study.Similar ecological studies may involve data at different levels of aggregation (eg, individual, physician, hospital, and community). The problem with analyzing aggregated ecological variables such as physician characteristics (eg, specialty) or hospital characteristics (eg, teaching versus nonteaching) at an individual-patient level in 1 regression model is that the SEs of the outcomes variables may be correlated (ie, patients treated at a given hospital are more likely to be similar to each other), and this correlation can lead to erroneous statistical conclusions.28,29 To overcome this limitation, multilevel or hierarchical regression models are increasingly being used in which each level of data is analyzed using a series of separate nested regression equations. Multilevel models are complicated and tend to be more conservative in terms of the conclusions they render about aggregate-level ecological variables.30 Comparisons of the conclusions from traditional regression models and multilevel regression models have shown that the conclusions about higher-level variables can vary depending on the method of statistical analysis chosen.30 In general, we favor the use of multilevel modeling for ecological studies where appropriate, as it is a statistically superior method of data analysis. However, it is also more complicated to implement, and it can sometimes be more difficult to interpret the significance of the results.SummaryEcological studies have played an increasingly important role in the field of cardiovascular outcomes research and are increasingly attracting the attention of policy makers, health system managers, clinicians, and the public. We have illustrated in this review some examples of how ecological research methods can be used to identify and answer important policy and research questions that might not be identified or answered using alternative study designs. The primary “strength” of many ecological studies is that they are often the only way to answer a given scientific question. However, as many of these examples show, ecological studies can generate many “hypotheses,” but their findings are often not conclusive and need careful interpretation and further investigations. Investigators should always be cognizant of the possibility of an ecological fallacy whereby potentially misleading causal inferences might be generated. Ecological studies of groups and studies of individuals complement one another, and both have an important role to play in increasing our understanding of how best to improve the outcomes of cardiovascular patients.Sources of FundingDr Tu is supported by a Canada Research Chair in Health Services Research and by a Career Investigator Award from the Heart and Stroke Foundation of Ontario, Toronto, Ontario. Dr Ko is supported by a Clinician Scientist Award from the Heart and Stroke Foundation of Ontario. This work was supported in part by a Canadian Institutes of Health Research Team Grant in Cardiovascular Outcomes Research.DisclosuresThe results and conclusions of this study are those of the authors and should not be attribut