Evaluating Ebola vaccine trials: insights from simulation

Abstract

Piszczek and Parlow1 outlined expected benefits of a stepped-wedge cluster trial (SWCT) design, with specific reference to the Sierra Leone Trial to Introduce a Vaccine against Ebola (STRIVE). STRIVE, however, is not an SWCT, but a phased-rollout trial in which randomization to immediate or delayed vaccination arms occurs at the individual level (RCT) within trial clusters.2 While the SWCT design is advantageous in certain circumstances, many of the benefits described by Piszczek and Parlow would not apply to evaluation of Ebola vaccine candidates in Sierra Leone. In a recently published study, we used simulations to compare statistical validity and power for an SWCT and a STRIVE-like RCT in the same trial population.3 Piszczek and Parlow contend that an SWCT can achieve greater statistical power than an RCT via multiple before-and-after and between-group comparisons; however, we found that the declining and heterogeneous epidemic incidence across Sierra Leone undermine such cluster-level comparisons and, consequently, the power of an SWCT. Specifically, we estimated that the SWCT design would be 3–10 times less likely than an individually randomized, phased roll-out RCT to definitively identify an efficacious vaccine. For example, an SWCT starting in April 2015 was expected to have a less than 10% chance of detecting the effect of a 90% efficacious vaccine. As highlighted by Piszczek and Parlow (and the article to which they respond4), the primary advantage of an SWCT is that it avoids the ethical problem of withholding a potentially lifesaving intervention from trial participants. Phased roll-out RCTs can address this, in part, by vaccinating all control participants at the end of the trial, as in STRIVE, although this introduces a delay in vaccination of some participants in the interest of experimental design. When risk is highly variable in space and time, as with Ebola in Sierra Leone, however, a phased roll-out RCT has an additional ethical advantage the SWCT lacks: it allows prioritized vaccination of clusters experiencing high infection risk. Such prioritization would confer the highest likelihood of benefit to those at highest risk, thereby reducing the total risk to trial participants relative to a non-risk-prioritized design. An SWCT, in contrast, requires random-ordered roll-out by definition5 and therefore cannot allow such prioritization. An observational impact assessment of risk-prioritized vaccine roll-out without a control arm would produce biased efficacy estimates, since vaccination order would be confounded with other factors associated with infection risk. The relative merits of trial designs are context-specific, and the benefits conventionally associated with certain designs may be achievable by alternative designs, when carefully tailored to local situations. We believe that proposed designs should be rigorously analyzed and compared (e.g., via simulation) as a matter of course in trial planning to ensure that trials are valid, efficiently powered, and ethically justified within the setting in which the trial will be conducted.