Effects of Allocation Method and Time Trends on Identification of the Best Arm in Multi-Arm Trials

Abstract

Many trial designs, such as dose-finding trials, shared-control designs, or adaptive platform trials, investigate multiple therapies simultaneously. Often these trials seek to identify the best arm and compare it to a control. Adaptive trials are commonly considered in this space, focusing on methods that drop arms or adjust allocation in response to accumulating information. These methods continue to be compared in the literature, most recently with an emphasis on the effect of time trends during the experiment. Here we compare several methods, considering their performance with and without time trends present. The four procedures are: (a) fixed allocation, (b) arm dropping based on p-values (two variants), (c) arm dropping based on the posterior probability each arm is best (two variants), and (d) response-adaptive randomization. These procedures are compared in terms of their ability to identify the best arm, statistical power, accuracy of estimation, and potential benefit to participants inside the trial. We find arm dropping based on the probability each arm is best and RAR among the best options from the methods considered. Arm dropping based on p-values performs moderately worse, and fixed allocation is much worse on all metrics within this context.