Type I error rates of multi-arm multi-stage clinical trials: strong control and impact of intermediate outcomes.

Daniel J Bratton,Babak Choodari-Oskooei,Patrick P J Phillips,Mahesh K B Parmar

doi:10.1186/s13063-016-1382-5

Abstract

BackgroundThe multi-arm multi-stage (MAMS) design described by Royston et al. [Stat Med. 2003;22(14):2239–56 and Trials. 2011;12:81] can accelerate treatment evaluation by comparing multiple treatments with a control in a single trial and stopping recruitment to arms not showing sufficient promise during the course of the study. To increase efficiency further, interim assessments can be based on an intermediate outcome (I) that is observed earlier than the definitive outcome (D) of the study. Two measures of type I error rate are often of interest in a MAMS trial. Pairwise type I error rate (PWER) is the probability of recommending an ineffective treatment at the end of the study regardless of other experimental arms in the trial. Familywise type I error rate (FWER) is the probability of recommending at least one ineffective treatment and is often of greater interest in a study with more than one experimental arm.MethodsWe demonstrate how to calculate the PWER and FWER when the I and D outcomes in a MAMS design differ. We explore how each measure varies with respect to the underlying treatment effect on I and show how to control the type I error rate under any scenario. We conclude by applying the methods to estimate the maximum type I error rate of an ongoing MAMS study and show how the design might have looked had it controlled the FWER under any scenario.ResultsThe PWER and FWER converge to their maximum values as the effectiveness of the experimental arms on I increases. We show that both measures can be controlled under any scenario by setting the pairwise significance level in the final stage of the study to the target level. In an example, controlling the FWER is shown to increase considerably the size of the trial although it remains substantially more efficient than evaluating each new treatment in separate trials.ConclusionsThe proposed methods allow the PWER and FWER to be controlled in various MAMS designs, potentially increasing the uptake of the MAMS design in practice. The methods are also applicable in cases where the I and D outcomes are identical.

Highlights

The multi-arm multi-stage (MAMS) design described by Royston et al [Stat Med. 2003;22(14):2239–56 and Trials. 2011;12:81] can accelerate treatment evaluation by comparing multiple treatments with a control in a single trial and stopping recruitment to arms not showing sufficient promise during the course of the study
The multi-arm multi-stage (MAMS) clinical trial design described by Royston et al [1, 2] for time-to-event outcomes and by Bratton et al [3] for binary outcomes is a relatively simple and effective framework for accelerating the evaluation of new treatments
*Correspondence: b.choodari-oskooei@ucl.ac.uk Medical Research Council (MRC) Clinical Trials Unit at UCL, 125 Kingsway, WC2B 6NH London, UK. In this particular family of MAMS designs, multiple experimental arms are compared to a common control at a series of interim analyses on an appropriate intermediate outcome (I) that is on the causal pathway to the definitive primary outcome of the study (D)

Summary

Introduction

The multi-arm multi-stage (MAMS) design described by Royston et al [Stat Med. 2003;22(14):2239–56 and Trials. 2011;12:81] can accelerate treatment evaluation by comparing multiple treatments with a control in a single trial and stopping recruitment to arms not showing sufficient promise during the course of the study. The multi-arm multi-stage (MAMS) clinical trial design described by Royston et al [1, 2] for time-to-event outcomes and by Bratton et al [3] for binary outcomes is a relatively simple and effective framework for accelerating the evaluation of new treatments. The design has already been successfully implemented in cancer [4] and is starting to be used in other areas such as tuberculosis [5] In this particular family of MAMS designs, multiple experimental arms are compared to a common control at a series of interim analyses on an appropriate intermediate outcome (I) that is on the causal pathway to the definitive primary outcome of the study (D). Recruitment is stopped to experimental arms that fail

Methods

Results

Discussion

Conclusion