Selection Bias in Observational Studies of Palliative Care: Lessons Learned

Abstract

ContextPalliative care (PC) programs are typically evaluated using observational data, raising concerns about selection bias. ObjectivesTo quantify selection bias because of observed and unobserved characteristics in a PC demonstration program. MethodsProgram administrative data and 100% Medicare claims data in two states and a 20% sample in eight states (2013–2017). The sample included 2983 Medicare fee-for-service beneficiaries aged 65+ participating in the PC program and three matched cohorts: regional; two states; and eight states. Confounding because of observed factors was measured by comparing patient baseline characteristics. Confounding because of unobserved factors was measured by comparing days of follow-up and six-month and one-year mortality rates. ResultsAfter matching, evidence for observed confounding included differences in observable baseline characteristics, including race, morbidity, and utilization. Evidence for unobserved confounding included significantly longer mean follow-up in the regional, two-state, and eight-state comparison cohorts, with 207 (P < 0.001), 192 (P < 0.001), and 187 (P < 0.001) days, respectively, compared with the 162 days for the PC cohort. The PC cohort had higher six-month and one-year mortality rates of 53.5% and 64.5% compared with 43.5% and 48.0% in the regional comparison, 53.4% and 57.4% in the two-state comparison, and 55.0% and 59.0% in the eight-state comparison. ConclusionThis case study demonstrates that selection of comparison groups impacts the magnitude of measured and unmeasured confounding, which may change effect estimates. The substantial impact of confounding on effect estimates in this study raises concerns about the evaluation of novel serious illness care models in the absence of randomization. We present key lessons learned for improving future evaluations of PC using observational study designs.