Assessing the impact of unmeasured confounding in external control arms via tipping point analyses.

Abstract

Listen

Translate article

e23065 Background: Estimates of the comparative efficacy of new therapies from single-arm settings can be obtained in advance of RCTs through use of external control arms (ECAs).[1] ECAs are collections of patients with the index disease who were treated outside of the single-arm trial, whose measured baseline attributes are matched to the single-arm trial patients and whose outcomes are compared to trial patients’ to estimate comparative efficacy. Without randomization, observed treatment-outcome associations may be confounded by unmeasured patient attributes. Applying established statistical methods, we estimate the magnitude and prevalence of unmeasured confounding required to move an apparently favorable hazard ratio (HR) to the “tipping point” where the new drug is no longer associated with a favorable survival statistically or clinically. Methods: Studying previously reported patients with multiple myeloma treated with an experimental therapy in a clinical trial (N = 290) and patients treated with standard of care therapy from a rigorously matched ECA (N = 290), we applied the method of Lin et al. to adjust the observed treatment effect (HR and 95% CIs) for overall survival to reflect the impact of a (set of) unmeasured confounder(s).[1,2] Lin’s formula incorporates both the unmeasured confounder’s theoretical association with mortality and its prevalence according to treatment group (i.e., single-arm trial vs. ECA). Results: The observed treatment effect for the single-arm trial treated vs. ECA patients was HR 0.76 (95% CI 0.63-0.91). We estimated the impact of an unmeasured confounder (where HR for overall survival of those patients with and without the confounder is set to 1.5) by its prevalence in each group. When the prevalence of the unmeasured confounder is balanced across groups there is no change in the observed treatment effect. When the presence of the confounder is 70% for ECA patients and absent in the trial patients, the clinical tipping point occurs with loss of the favorable HR (i.e., HR 1.02, 95% CI: 0.85-1.23). Conclusions: While novel analytic methods like ECAs have the potential to accelerate drug development, the lack of randomization raises concern for potential unmeasured confounding. Applying Lin’s method, we illustrate that the impact of unmeasured confounding on HR estimates from a single-arm trial vs.an ECA is a function of both the association with mortality and asymmetries in prevalence. Consistency in the efficacy conclusion for all clinically tenable assumptions indicates a qualitatively reliable conclusion. Friends of Cancer Research whitepaper (2019): available online at https://www.focr.org/sites/default/files/Panel-1_External_Control_Arms2019AM.pdf. Lin D, Psaty B, Kronmal R. Assessing the sensitivity of regression results to unmeasured confounders in observational studies. Biometrics.1998; 54:948–963.