Selection Bias In Observational Studies Research Articles

Use of instrumental variables in the presence of heterogeneity and self‐selection: an application to treatments of breast cancer patients

Instrumental variable (IV) methods are widely used in the health economics literature to adjust for hidden selection biases in observational studies when estimating treatment effects. Less attention has been paid in the applied literature to the proper use of IVs if treatment effects are heterogeneous across subjects and individuals select treatments based on expected idiosyncratic gains or losses from treatments. In this paper we compare conventional IV analysis with alternative approaches that use IVs to estimate treatment effects in models with response heterogeneity and self-selection. Instead of interpreting IV estimates as the effect of treatment at an unknown margin of patients, we identify the marginal patients and we apply the method of local IVs to estimate the average treatment effect and the effect on the treated on 5-year direct costs of breast-conserving surgery and radiation therapy compared with mastectomy in breast cancer patients. We use a sample from the Outcomes and Preferences in Older Women, Nationwide Survey which is designed to be representative of all female Medicare beneficiaries (aged 67 or older) with newly diagnosed breast cancer between 1992 and 1994. Our results reveal some of the advantages and limitations of conventional and alternative IV methods in estimating mean treatment effect parameters.

Analysis of Observational Studies in the Presence of Treatment Selection Bias

Comparisons of outcomes between patients treated and untreated in observational studies may be biased due to differences in patient prognosis between groups, often because of unobserved treatment selection biases. To compare 4 analytic methods for removing the effects of selection bias in observational studies: multivariable model risk adjustment, propensity score risk adjustment, propensity-based matching, and instrumental variable analysis. A national cohort of 122,124 patients who were elderly (aged 65-84 years), receiving Medicare, and hospitalized with acute myocardial infarction (AMI) in 1994-1995, and who were eligible for cardiac catheterization. Baseline chart reviews were taken from the Cooperative Cardiovascular Project and linked to Medicare health administrative data to provide a rich set of prognostic variables. Patients were followed up for 7 years through December 31, 2001, to assess the association between long-term survival and cardiac catheterization within 30 days of hospital admission. Risk-adjusted relative mortality rate using each of the analytic methods. Patients who received cardiac catheterization (n = 73 238) were younger and had lower AMI severity than those who did not. After adjustment for prognostic factors by using standard statistical risk-adjustment methods, cardiac catheterization was associated with a 50% relative decrease in mortality (for multivariable model risk adjustment: adjusted relative risk [RR], 0.51; 95% confidence interval [CI], 0.50-0.52; for propensity score risk adjustment: adjusted RR, 0.54; 95% CI, 0.53-0.55; and for propensity-based matching: adjusted RR, 0.54; 95% CI, 0.52-0.56). Using regional catheterization rate as an instrument, instrumental variable analysis showed a 16% relative decrease in mortality (adjusted RR, 0.84; 95% CI, 0.79-0.90). The survival benefits of routine invasive care from randomized clinical trials are between 8% and 21%. Estimates of the observational association of cardiac catheterization with long-term AMI mortality are highly sensitive to analytic method. All standard risk-adjustment methods have the same limitations regarding removal of unmeasured treatment selection biases. Compared with standard modeling, instrumental variable analysis may produce less biased estimates of treatment effects, but is more suited to answering policy questions than specific clinical questions.

Cardiovascular health and the menopause: the gynecologist as the patients' interface

The large decrease in estrogen following menopause appears to explain the dramatic increase in cardiovascular disease (CVD) in postmenopausal women. Gynecologists are well placed to play a primary role in the diagnosis, prevention and management of CVD in these patients; this role may include advice on lifestyle changes, and, if appropriate, prescribing preventative treatments such as hormone replacement therapy (HRT) and lipid-lowering drugs. The use of estrogen replacement therapy (ERT) to prevent CVD is supported by a number of observational studies. However, recently, large, randomized trials gave unexpected, conflicting data on the cardiovascular benefits of HRT, leading to confusion, and influencing both patient and clinical perceptions regarding the role of HRT postmenopause. These different outcomes may be due to differences in the HRT regimens, mean age and mean time from menopause at enrollment, duration of therapy, and patient selection bias in observational studies. A ‘unified hypothesis’ consistent with findings from all studies has now been developed: HRT initiated at the time of the menopause prevents CVD, whereas HRT initiated years after the menopause seems to increase CHD events. This knowledge is essential for gynecologists making clinical decisions regarding HRT use.

Prospective study of screening for stomach cancer in Japan.

Although screening for stomach cancer is a widespread community service in Japan, the benefits of the screening program remain unclear. Our study investigated prospectively the relation between participation in stomach-cancer screening during the past 12 months and subsequent deaths. Data was derived from the Japan Collaborative Cohort Study, in which 480 stomach-cancer deaths were identified during an 8-year follow-up period. Cox proportional hazard regression was used to estimate the relative risk of death from stomach cancer and that from other causes while adjusting for potential confounding factors. In men, screening participation was associated significantly with a reduced risk of death from stomach cancer (relative risk [RR] = 0.54; 95% confidence interval [CI] = 0.41-0.70). The extent of the risk reduction was greater than potential health selection (for deaths other than stomach, RR = 0.71). The adjustment for potential confounding variables, however, attenuated the difference in risk of death (stomach cancer, RR = 0.65; other causes, RR = 0.71). In women, the magnitude of the association between screening participation and death from stomach cancer (RR = 0.74; 95% CI = 0.52-1.07) was equal to that for deaths from non-stomach cancers (RR = 0.74). Subgroup analysis showed that women with a parental history of stomach cancer had a reduced risk of death from stomach cancer associated with screening (RR = 0.32; 95% CI = 0.12-0.87). The present results underline the potential for selection bias in observational studies, and thus it remains an open question whether screening for stomach cancer is effective.

WW15: IDENTIFYING THE IMPACT OF A TREATMENT ON OUTCOMES AND COSTS USING OBSERVATIONAL DATA: OVERCOMING SELECTION BIAS

OBJECTIVES: Selection bias in observational studies is described. Instrumental variable estimation is presented in detail as a method of identification of treatment effects. Practical issues related to the proper implementation of this technique are stressed. Propensity score methods are also discussed. PARTICIPANTS WHO WOULD BENEFIT: Researchers who use, evaluate, or conduct observational studies. While estimates of treatment outcomes and costs from randomized clinical trials are still considered the gold standard, the use of observational data for these estimates is common. Observational data are less costly to obtain, readily available, and represent actual practice. However, there is always a danger of selection bias when these data are used to identify the costs or outcomes of one particular treatment versus another. This is true even when quality risk adjustment methods and propensity score methods are employed. Instrumental variable estimation and propensity score methods have both been proposed to overcome the selection bias problem in observational studies. We describe selection bias and how instrumental variables analysis and propensity score methods overcome the problem. We then explain how one would apply these techniques correctly to common research problems emphasizing the importance of choosing when propensity score methods are sufficient or when instrumental variables analysis should be used.

Sources of selection bias in evaluating social programs: an interpretation of conventional measures and evidence on the effectiveness of matching as a program evaluation method.

This paper decomposes the conventional measure of selection bias in observational studies into three components. The first two components are due to differences in the distributions of characteristics between participant and nonparticipant (comparison) group members: the first arises from differences in the supports, and the second from differences in densities over the region of common support. The third component arises from selection bias precisely defined. Using data from a recent social experiment, we find that the component due to selection bias, precisely defined, is smaller than the first two components. However, selection bias still represents a substantial fraction of the experimental impact estimate. The empirical performance of matching methods of program evaluation is also examined. We find that matching based on the propensity score eliminates some but not all of the measured selection bias, with the remaining bias still a substantial fraction of the estimated impact. We find that the support of the distribution of propensity scores for the comparison group is typically only a small portion of the support for the participant group. For values outside the common support, it is impossible to reliably estimate the effect of program participation using matching methods. If the impact of participation depends on the propensity score, as we find in our data, the failure of the common support condition severely limits matching compared with random assignment as an evaluation estimator.

Survivor treatment selection bias in observational studies: examples from the AIDS literature.

Unlike patients in a randomized, clinical trial, patients in an observational study choose if and when to begin treatment. Patients who live longer have more opportunities to select treatment; those who die earlier may be untreated by default. These facts are the essence of an often overlooked bias, termed "survivor treatment selection bias," which can erroneously lead to the conclusion that an ineffective treatment prolongs survival. Unfortunately, misanalysis of survivor treatment selection bias has been prevalent in the recent literature on the acquired immunodeficiency syndrome. Approaches to mitigating this bias involve complex statistical models. At a minimum, initiation of therapy should be treated as a time-dependent covariate in a proportional hazards model. Investigators and readers should be on the alert for survivor treatment selection bias and should be cautious when interpreting the results of observational treatment studies.