Outcome imbalance in clinical trials: Mitigating loss of statistical power

Abstract

AbstractBackgroundUnequal rates of decline between placebo and treatment arms reduce a trial’s power. We use machine learning to differentiate decliners from non‐decliners to mitigate the impact of outcome imbalance.MethodWe trained a model to classify decliners (i.e. individuals with higher CDR‐SB at 24 months of follow‐up) and non‐decliners on 1329 individuals with MCI or Alzheimer’s dementia from ADNI (adni.loni.usc.edu) and NACC (naccdata.org). Input features included baseline age, sex, APOE4 status, gray matter volumes of brain regions from MRI, MMSE, and CDR‐SB. We 1) simulated 100,000 trials by randomizing individuals to placebo and treatment (n = 250 per arm) and measured the proportions of decliners between the arms to assess the likelihood of imbalance, 2) measured the power to detect a 25% treatment effect across simulated trials with varying levels of imbalance (0‐5% more decliners in treatment, 1000 simulations per imbalance level), and 3) studied whether covariate adjustment and enrichment with predicted decliners alleviate imbalance‐related power losses.Result22.4% of our simulated trials had inter‐arm imbalances of 5% or more, which translated into reduced power (‐15%) and effect sizes (‐0.11) compared to balanced trials (mean ± sem 89.2 ± 0.6% power, 0.39 ± 0.006 Cohen’s d for balanced trials; 73.8 ± 0.9% power, 0.28 ± 0.004 Cohen’s d for 5%‐imbalanced trials). Covariate adjustment on prognostic factors (e.g. APOE4, diagnosis, baseline CDR‐SB) increased power, but imbalance still reduced power by 9% (balanced: 97.2 ± 0.3%; 5%‐imbalanced: 87.8 ± 0.7%). Subgroups analyses of the predicted decliners (excluding predicted non‐decliners) increased power, despite smaller sample sizes (balanced: 97.4 ± 0.3%; 5%‐imbalanced: 92.2 ± 0.5%). Using enriched samples with 250 predicted decliners per arm obtained the greatest power and constrained the power loss to only 3% (balanced: 99.1 ± 0.1%; 5%‐imbalanced: 96.2 ± 0.4%) (Figure).ConclusionTo improve a trial’s likelihood of success, covariate adjustment and enrichment with likely decliners should be used to mitigate power loss related to outcome imbalance.