The Impact of Selection Bias in Transplant Research Intended to Inform Patient Selection: An Example and Potential Solutions.

Abstract

A major stated purpose of transplant outcomes research is to guide the selection of patients for inclusion on the deceased donor waiting list. Because of this listing process, transplant research is uniquely susceptible to selection bias. In this article, we define one type of selection bias in transplantation research intended to inform listing practices, give an example of how bias can lead to incorrect inferences, and discuss potential approaches to mitigating the impact of this bias on the field. WHAT IS SELECTION BIAS? Researchers consider 3 key populations when designing a study: the source population, the study population, and the target population.1 The source population includes those individuals who will be sampled for inclusion in the study. The study population is either a subset or complete set of the source population that contributes data to the study. The target population includes the broader population the study intends to represent. Although the target population is theoretical, the source population is a tangible group of individuals who represent the experience of the target population. Selection bias occurs when the effect of the exposure on the outcome observed in the study population is different from the effect in the source population.1 This is an issue of internal validity—the study’s results are not valid for the population that the study sampled from. In contrast, when the study results are not valid for the target population, there are issues of external validity. This is often referred to as generalizability—the results of the study cannot be generalized to the broader population of interest. In research intended to inform listing practices, the target population is future patients undergoing evaluation for listing (ie, patients referred to transplant centers). The corresponding source population is patients who previously underwent this process. However, when the exposure of interest is a posttransplant outcome, the study population is necessarily limited to patients with available data about the outcome—patients who received a transplant. Transplanted patients are selected in part based on their likelihood to do well after transplant and tend to be healthier than those who are not listed.2 Because of this discrepancy between the study and the source population, the effect of an exposure on an outcome among the population of patients who did receive a transplant may not be the same as the effect among patients who could have received a transplant. In other words, the study may not be internally valid because of selection bias. In the life cycle of research, this bias becomes circular—selection bias impacts research findings, biased findings are used to guide listing, and listing practices contribute to bias in future studies. SOBRIETY AND TRANSPLANT OUTCOMES EXAMPLE Alcohol-associated liver disease (ALD) is a leading indication for liver transplantation in the United States.3 When assessing ALD patients as potential transplant candidates, clinicians typically consider the likelihood of excessive alcohol use after transplant, which increases the risk of recurrent alcoholic cirrhosis, graft failure, and mortality.4 The majority of transplant centers in the United States have historically recommended or required 6 mo of sobriety for ALD patients before listing for liver transplant.5 In this example, Transplant Center A recommends 6 mo of sobriety for ALD patients before listing. Patients with <6 mo of sobriety may be considered for listing if they undergo additional assessments with a counselor to evaluate their risk of relapse after transplantation. Patients felt to be at particularly high risk for relapse are required to document attendance at inpatient or outpatient rehabilitation before listing. Center A has observed equivalent outcomes between ALD and non-ALD patients with respect to alcohol use, graft failure, and mortality under this protocol. In recent years, studies have found similar rates of alcohol use and graft failure among patients with and without 6 mo of pretransplant sobriety, challenging current guidelines.6-8 Based on this evidence and their prior success, Center A has decided to drop their sobriety period recommendation and, as a consequence, their additional selection criteria. After a period of time, they begin observing increased rates of relapse and graft dysfunction among their ALD recipients. This is contrary to research used to guide their listing practices and their own experience. One explanation for this apparent paradox is that the relationship between pretransplant sobriety and posttransplant outcomes is different among the more stringently evaluated group of previous transplant recipients than in the population now listed by Center A. Center A decides to conduct a study to evaluate the association between 6 mo of pretransplant sobriety and posttransplant graft failure to inform their listing practices. The target population of this study is future transplant candidates at Center A. The source population is previous transplant candidates at Center A. The study population includes patients from the source population who were listed at Center A, received a transplant, and contributed data on posttransplant graft failure. Figure 1 provides hypothetical data for multiple selection scenarios from the source population.FIGURE 1.: A, The true association between a pretransplant exposure and a posttransplant outcome. B–D, Examples of multiple transplant selection scenarios and their impacts on the observed association.Figure 1A shows what would have theoretically happened if every patient evaluated for liver transplant at Center A received a transplant—in other words, what we would have observed if we had complete data on the source population. In this example, patients with <6 mo of sobriety at transplant would have had 2.25 times the odds of graft failure after transplant (true odds ratio [OR] = 2.25). In contrast, Figure 1B and C shows situations in which selection from this source population into the study population (ie, selection for and receipt of transplant) differs by either pretransplant sobriety or risk of graft failure. In Figure 1B, patients with <6 mo of sobriety were half as likely to be selected for transplant and therefore included in the study population as patients with ≥6 mo, and in Figure 1C, patients who are at high risk for graft failure were half as likely to be selected for transplant. In both situations, the OR in the study population is the same as the OR in the source population (OR = 2.25)—there is no selection bias. However, in the last situation (Figure 1D), patients with <6 mo of sobriety and at high risk for graft failure are less likely than any other group to be selected for transplant. This differential selection by both the exposure and the outcome results in an observed OR of 1.13, a notably lower estimate than the true OR of 2.25, indicating selection bias. The stringent listing criteria previously applied by Center A to patients with <6 mo of sobriety correspond to the scenario shown in Figure 1D. After removing their additional listing criteria, they began observing results under the scenario shown in Figure 1C, in which patients were still selected with regard to their outcome risk but not differentially selected with respect to sobriety status. This resulted in an observed shift from no association between sobriety and graft failure to an increased risk of graft failure among patients with <6 mo of sobriety. This shift demonstrates the impact that listing criteria has on the selection of the study population from the source population and how this selection can bias research findings. WHAT CAN WE DO ABOUT SELECTION BIAS IN TRANSPLANT RESEARCH? In practice, transplant center selection committees will continue to take pretransplant factors into account when making listing decisions. The purpose of this article is to encourage researchers and clinicians to consider how selection practices impact the evidence base guiding these decisions. Researchers should define their target, source, and study populations and carefully consider whether selection into their study population may have been differential by both exposure and outcome. Particularly for single-center studies, information on listing guidelines at that center should be reported in the methods section, and consideration of how selection bias may impact findings should be included in the discussion. This information should be used when evaluating evidence to inform clinical practice, including listing guidelines. For researchers with access to prelisting data, there are additional options for addressing this type of selection bias through quantitative bias analysis, which can provide an estimate of the direction, magnitude, and uncertainty arising from bias.9 Ideally, transplant research should include data on referred patients to allow for comparisons of the listed or transplanted population (study population) to the referred (source population) by both the exposure of interest and other relevant covariates (ie, age, disease severity). National collection of transplant referral data could further facilitate the goal of improving evidence-based practice in transplantation by expanding available data on the source population and providing information needed for quantitative bias analysis.10 A stated purpose of many transplant studies is to guide listing practices. In this article, we demonstrate how one type of selection bias may impact observational research with a target population of potential transplant candidates. Researchers and clinicians should carefully consider how listing practices may impact research findings. Broader reporting and collection of data about listing practices may facilitate improved rigor in transplant research.