Outcome-adaptive Randomization Research Articles

Rationale and Design of the Randomized Bayesian Multicenter COME-TAVI Trial in Patients With a New Onset Left Bundle Branch Block

Patients with new-onset left bundle branch block (LBBB) after transcatheter aortic valve implantation (TAVI) are at risk of developing delayed high-degree atrioventricular block. Management of new-onset LBBB post-TAVI remains controversial. In the Comparison of a Clinical Monitoring Strategy Versus Electrophysiology-Guided Algorithmic Approach in Patients With a New LBBB After TAVI (COME-TAVI) trial, consenting patients with new-onset LBBB that persists on day 2 after TAVI, meeting exclusion/inclusion criteria, are randomized to an electrophysiological study (EPS)-guided approach or 30-day electrocardiographic monitoring. In the EPS-guided approach, patients with a His to ventricle (HV) interval ≥ 65 ms undergo permanent pacemaker implantation. Patients randomized to noninvasive monitoring receive a wearable continuous electrocardiographic recording and transmitting device for 30 days. Follow-up will be performed at 3, 6, and 12 months. The primary endpoint is a composite outcome designed to capture net clinical benefit. The endpoint incorporates major consequences of both strategies in patients with new-onset LBBB after TAVI, as follows: (i) sudden cardiac death; (ii) syncope; (iii) atrioventricular conduction disorder requiring a pacemaker (for a class I or IIa indication); and (iv) complications related to the pacemaker or EPS. The trial incorporates a Bayesian design with a noninformative prior, outcome-adaptive randomization (initially 1:1), and 2 prespecified interim analyses once 25% and 50% of the anticipated number of primary endpoints are reached. The trial is event-driven, with an anticipated upper limit of 452 patients required to reach 77 primary outcome events over 12 months of follow-up. In summary, the aim of this Bayesian multicentre randomized trial is to compare 2 management strategies in patients with new-onset LBBB post-TAVI—an EPS-guided approach vs noninvasive 30-day monitoring. Trial registration number: NCT03303612.

Over-accrual in Bayesian adaptive trials with continuous futility stopping.

We explore frequentist operating characteristics of a Bayesian adaptive design that allows continuous early stopping for futility. In particular, we focus on the power versus sample size relationship when more patients are accrued than originally planned. We consider the case of a phase II single-arm study and a Bayesian phase II outcome-adaptive randomization design. For the former, analytical calculations are possible; for the latter, simulations are conducted. Results for both cases show a decrease in power with an increasing sample size. It appears that this effect is due to the increasing cumulative probability of incorrectly stopping for futility. The increase in cumulative probability of incorrectly stopping for futility is related to the continuous nature of the early stopping, which increases the number of interim analyses with accrual. The issue can be addressed by, for instance, delaying the start of testing for futility, reducing the number of futility tests to be performed or by setting stricter criteria for concluding futility.

Adaptive Enrichment Designs in Clinical Trials.

Adaptive enrichment designs for clinical trials may include rules that use interim data to identify treatment-sensitive patient subgroups, select or compare treatments, or change entry criteria. A common setting is a trial to compare a new biologically targeted agent to standard therapy. An enrichment design's structure depends on its goals, how it accounts for patient heterogeneity and treatment effects, and practical constraints. This article first covers basic concepts, including treatment-biomarker interaction, precision medicine, selection bias, and sequentially adaptive decision making, and briefly describes some different types of enrichment. Numerical illustrations are provided for qualitatively different cases involving treatment-biomarker interactions. Reviews are given of adaptive signature designs; a Bayesian design that uses a random partition to identify treatment-sensitive biomarker subgroups and assign treatments; and designs that enrich superior treatment sample sizes overall or within subgroups, make subgroup-specific decisions, or include outcome-adaptive randomization.

Bayesian biomarker-driven outcome-adaptive randomization with an imperfect biomarker assay.

We investigate the impact of biomarker assay's accuracy on the operating characteristics of a Bayesian biomarker-driven outcome-adaptive randomization design. In a simulation study, we assume a trial with two treatments, two biomarker-based strata, and a binary clinical outcome (response). Pbt denotes the probability of response for treatment t (t = 0 or 1) in biomarker stratum (b = 0 or 1). Four different scenarios in terms of true underlying response probabilities are considered: a null (P00 = P01 = 0.25, P10 = P11= 0.25) and consistent (P00 = P10 = 0.25, P01 = 0.5) treatment effect scenario, as well as a quantitative (P00 = P01 = P10 = 0.25, P11 = 0.5) and a qualitative (P00 = P11 = 0.5, P01 = P10 = 0.25) stratum-treatment interaction. For each scenario, we compare the case of a perfect with the case of an imperfect biomarker assay with sensitivity and specificity of 0.8 and 0.7, respectively. In addition, biomarker-positive prevalence values P(B = 1) = 0.2 and 0.5 are investigated. Results show that the use of an imperfect assay affects the operational characteristics of the Bayesian biomarker-based outcome-adaptive randomization design. In particular, the misclassification causes a substantial reduction in power accompanied by a considerable increase in the type-I error probability. The magnitude of these effects depends on the sensitivity and specificity of the assay, as well as on the distribution of the biomarker in the patient population. With an imperfect biomarker assay, the decision to apply a biomarker-based outcome-adaptive randomization design may require careful reflection.

The ProBio trial: molecular biomarkers for advancing personalized treatment decision in patients with metastatic castration-resistant prostate cancer

BackgroundMultiple therapies exist for patients with metastatic castration-resistant prostate cancer (mCRPC). However, their improvement on progression-free survival (PFS) remains modest, potentially explained by tumor molecular heterogeneity. Several prognostic molecular biomarkers have been identified for mCRPC that may have predictive potential to guide treatment selection and prolong PFS. We designed a platform trial to test this hypothesis.MethodsThe Prostate-Biomarker (ProBio) study is a multi-center, outcome-adaptive, multi-arm, biomarker-driven platform trial for tailoring treatment decisions for men with mCRPC. Treatment decisions in the experimental arms are based on biomarker signatures defined as mutations in certain genes/pathways suggested in the scientific literature to be important for treatment response in mCRPC. The biomarker signatures are determined by targeted sequencing of circulating tumor and germline DNA using a panel specifically designed for mCRPC.DiscussionPatients are stratified based on the sequencing results and randomized to either current clinical practice (control), where the treating physician decides treatment, or to molecularly driven treatment selection based on the biomarker profile. Outcome-adaptive randomization is implemented to early identify promising treatments for a biomarker signature. Biomarker signature-treatment combinations graduate from the platform when they demonstrate 85% probability of improving PFS compared to the control arm. Graduated combinations are further evaluated in a seamless confirmatory trial with fixed randomization. The platform design allows for new drugs and biomarkers to be introduced in the study.ConclusionsThe ProBio design allows promising treatment-biomarker combinations to quickly graduate from the platform and be confirmed for rapid implementation in clinical care.Trial registrationClinicalTrials.gov Identifier NCT03903835. Date of registration: April 4, 2019. Status: Recruiting.

Bias in retrospective analyses of biomarker effect using data from an outcome-adaptive randomized trial.

Biomarker-stratified outcome-adaptive randomization trials, in which randomization probabilities depend on both biomarker value and outcomes of previously treated patients, are receiving increased attention in oncology research. Data from these trials can also form the basis of investigation of additional biomarkers that may not have been incorporated into the original trial design. In this article, we investigate the validity of a standard analytical method that utilizes data from a biomarker-stratified outcome-adaptive randomization trial to assess the effect of a newly identified biomarker on patient outcomes. In the context of an ancillary biomarker study for a two-arm phase II trial with a response endpoint, we conduct analytic and simulation studies to investigate bias in estimated biomarker effects under outcome-adaptive randomization. Conditions under which bias arises and magnitude of the bias are examined in several settings. We then propose unbiased estimators of biomarker effects with appropriate variance estimators. We demonstrate that use of biomarker-stratified outcome-adaptive randomization perturbs the patient population and treatment assignments. Consequently, application of standard analysis methods to data from an outcome-adaptive randomization trial either to estimate prognostic effect of a new biomarker in uniformly treated patients or to estimate effect of treatment in relation to the new biomarker can lead to substantially biased estimates. The proposed adjusted estimators are asymptotically unbiased, and the proposed variance estimators correctly reflect the sample variability in the estimators. This article demonstrates existence of bias when standard, naïve statistical methods are utilized to assess biomarker effects using data from a biomarker-stratified outcome-adaptive randomization trial, and hence that results from naïve analyses must be interpreted with great caution. These findings highlight that, in an era where data and specimens are increasingly being shared for biomarker studies, care must be taken to document and understand implications of the study design under which specimens or data have been obtained.

Outcome-adaptive randomization in clinical trials: issues of participant welfare and autonomy

Outcome-adaptive randomization (OAR) has been proposed as a corrective to certain ethical difficulties inherent in the traditional randomized clinical trial (RCT) using fixed-ratio randomization. In particular, it has been suggested that OAR redresses the balance between individual and collective ethics in favour of the former. In this paper, I examine issues of welfare and autonomy arising in relation to OAR. A central issue in discussions of welfare in OAR is equipoise, and the moral status of OAR is crucially influenced by the way in which this concept is construed. If OAR is based on a model of equipoise that demands strict indifference between competing interventions throughout the trial, such equipoise is disturbed by accruing data favouring one treatment over another; OAR seeks to redress this by weighting randomization to the seemingly superior treatment. However, this is a partial response, as patients continue to be allocated to the inferior therapy. Moreover, it rests upon considerations of aggregate harms and benefits, and does not therefore uphold individual ethics. Issues of fairness also arise, as early and late enrollees are randomized on a different basis. Fixed-ratio randomization represents a fuller and more consistent response to a loss of equipoise, as so construed. With regard to consent, the complexity of OAR poses challenges to adequate disclosure and comprehension. Additionally, OAR does not offer a remedy to the therapeutic misconception—participants’ tendency to attribute treatment allocation in an RCT to individual clinical judgments, rather than to scientific considerations—and, if anything, accentuates rather than alleviates this misconception. In relation to these issues, OAR fails to offer ethical advantages over fixed-ratio randomization. More broadly, the ethical basis of OAR can be seen to lie more in collective than in individual ethics, and overall it fares worse in this territory than fixed-ratio randomization.

Adding experimental arms to platform clinical trials: randomization procedures and interim analyses.

Multi-arm clinical trials use a single control arm to evaluate multiple experimental treatments. In most cases this feature makes multi-arm studies considerably more efficient than two-arm studies. A bottleneck for implementation of a multi-arm trial is the requirement that all experimental treatments have to be available at the enrollment of the first patient. New drugs are rarely at the same stage of development. These limitations motivate our study of statistical methods for adding new experimental arms after a clinical trial has started enrolling patients. We consider both balanced and outcome-adaptive randomization methods for experimental designs that allow investigators to add new arms, discuss their application in a tuberculosis trial, and evaluate the proposed designs using a set of realistic simulation scenarios. Our comparisons include two-arm studies, multi-arm studies, and the proposed class of designs in which new experimental arms are added to the trial at different time points.

Adaptive Clinical Trials: Advantages and Disadvantages of Various Adaptive Design Elements.

There is a wide range of adaptive elements of clinical trial design (some old and some new), with differing advantages and disadvantages. Classical interim monitoring, which adapts the designbased on early evidence of superiority or futility of a treatment arm, has long been known to be extremely useful. A more recent application of interim monitoring is in the use of phase II/III designs, which can be very effective (especially in the setting of multiple experimental treatments and a reliable intermediate end point) but do have the cost of having to commit earlier to the phase III question than if separate phase II and phase III trials were performed. Outcome-adaptive randomization is an older technique that has recently regained attention; it increases trial complexity and duration without offering substantial benefits to the patients in the trial. The use of adaptive trials with biomarkers is new and has great potential for efficiently identifying patients who will be helped most by specific treatments. Master protocols in which trial arms and treatment questions are added to an ongoing trial can be especially efficient in the biomarker setting, where patients are screened for entry into different subtrials based on evolving knowledge about targeted therapies. A discussion of three recent adaptive clinical trials (BATTLE-2, I-SPY 2, and FOCUS4) highlights the issues.

Statistical controversies in clinical research: scientific and ethical problems with adaptive randomization in comparative clinical trials.

In recent years, various outcome adaptive randomization (AR) methods have been used to conduct comparative clinical trials. Rather than randomizing patients equally between treatments, outcome AR uses the accumulating data to unbalance the randomization probabilities in favor of the treatment arm that currently is superior empirically. This is motivated by the idea that, on average, more patients in the trial will be given the treatment that is truly superior, so AR is ethically more desirable than equal randomization. AR remains controversial, however, and some of its properties are not well understood by the clinical trials community. Computer simulation was used to evaluate properties of a 200-patient clinical trial conducted using one of four Bayesian AR methods and compare them to an equally randomized group sequential design. Outcome AR has several undesirable properties. These include a high probability of a sample size imbalance in the wrong direction, which might be surprising to nonstatisticians, wherein many more patients are assigned to the inferior treatment arm, the opposite of the intended effect. Compared with an equally randomized design, outcome AR produces less reliable final inferences, including a greatly overestimated actual treatment effect difference and smaller power to detect a treatment difference. This estimation bias becomes much larger if the prognosis of the accrued patients either improves or worsens systematically during the trial. AR produces inferential problems that decrease potential benefit to future patients, and may decrease benefit to patients enrolled in the trial. These problems should be weighed against its putative ethical benefit. For randomized comparative trials to obtain confirmatory comparisons, designs with fixed randomization probabilities and group sequential decision rules appear to be preferable to AR, scientifically, and ethically.

Commentary on Hey and Kimmelman.

Hey and Kimmelman disagree with the purported ethical advantage of outcome-adaptive randomization in the setting of two-arm trials and/or early phase studies. They also conclude that, compared to equal randomization, adaptive randomization might have a compelling ethical basis in multi-arm trials. They raise important considerations; however, many criticisms of adaptive randomization are broad and unjustified. It deserves a more thorough evaluation. Hey and Kimmelman use “ethics” to cover many things. It helps to discuss the usefulness of a trial design on the basis of three categories: statistical properties; practical considerations; and real-life performance.

Subgroup-Based Adaptive (SUBA) Designs for Multi-arm Biomarker Trials

Targeted therapies based on biomarker profiling are becoming a mainstream direction of cancer research and treatment. Depending on the expression of specific prognostic biomarkers, targeted therapies assign different cancer drugs to subgroups of patients even if they are diagnosed with the same type of cancer by traditional means, such as tumor location. For example, Herceptin is only indicated for the subgroup of patients with HER2+ breast cancer, but not other types of breast cancer. However, subgroups like HER2+ breast cancer with effective targeted therapies are rare and most cancer drugs are still being applied to large patient populations that include many patients who might not respond or benefit. Also, the response to targeted agents in humans is usually unpredictable. To address these issues, we propose SUBA, subgroup-based adaptive designs that simultaneously search for prognostic subgroups and allocate patients adaptively to the best subgroup-specific treatments throughout the course of the trial. The main features of SUBA include the continuous reclassification of patient subgroups based on a random partition model and the adaptive allocation of patients to the best treatment arm based on posterior predictive probabilities. We compare the SUBA design with three alternative designs including equal randomization, outcome-adaptive randomization and a design based on a probit regression. In simulation studies we find that SUBA compares favorably against the alternatives.

Outcome-adaptive randomization for a delayed outcome with a short-term predictor: imputation-based designs.

Delay in the outcome variable is challenging for outcome-adaptive randomization, as it creates a lag between the number of subjects accrued and the information known at the time of the analysis. Motivated by a real-life pediatric ulcerative colitis trial, we consider a case where a short-term predictor is available for the delayed outcome. When a short-term predictor is not considered, studies have shown that the asymptotic properties of many outcome-adaptive randomization designs are little affected unless the lag is unreasonably large relative to the accrual process. These theoretical results assumed independent identical delays, however, whereas delays in the presence of a short-term predictor may only be conditionally homogeneous. We consider delayed outcomes as missing and propose mitigating the delay effect by imputing them. We apply this approach to the doubly adaptive biased coin design (DBCD) for motivating pediatric ulcerative colitis trial. We provide theoretical results that if the delays, although non-homogeneous, are reasonably short relative to the accrual process similarly as in the iid delay case, the lag is also asymptotically ignorable in the sense that a standard DBCD that utilizes only observed outcomes attains target allocation ratios in the limit. Empirical studies, however, indicate that imputation-based DBCDs performed more reliably in finite samples with smaller root mean square errors. The empirical studies assumed a common clinical setting where a delayed outcome is positively correlated with a short-term predictor similarly between treatment arm groups. We varied the strength of the correlation and considered fast and slow accrual settings.

Ethical Considerations for Outcome‐adaptive Trial Designs: A Clinical Researcher's Perspective

In a typical comparative clinical trial the randomization scheme is fixed at the beginning of the study, and maintained throughout the course of the trial. A number of researchers have championed a randomized trial design referred to as 'outcome-adaptive randomization.' In this type of trial, the likelihood of a patient being enrolled to a particular arm of the study increases or decreases as preliminary information becomes available suggesting that treatment may be superior or inferior. While the design merits of outcome-adaptive trials have been debated, little attention has been paid to significant ethical concerns that arise in the conduct of such studies. These include loss of equipoise, lack of processes for adequate informed consent, and inequalities inherent in the research design which could lead to perceptions of injustice that may have negative implications for patients and the research enterprise. This article examines the ethical difficulties inherent in outcome-adaptive trials.

Abstract IA06: Adaptive designs and adaptive learning for biomarker-based clinical trials for targeted therapies.

Abstract Rapid advancements in genomics have changed the landscape of target identification and drug development for targeted therapies. Targeted agents work well when drugs are given to patients with the actionable targets. For example, TKI works well for lung cancer patients with EGFR mutation and crizotinib is highly effective for patients with EML4-ALK translocation. Targeted agents, however, do not work for everyone and may not work at all. Hence, the goals for developing target agents include (1) testing the treatment efficacy, (2) identifying prognostic and predictive markers, and (3) providing better treatments to patients enrolled in the trial. Several novel designs have been introduced to achieve these objectives. For example, Bayesian adaptive designs can assign more patients to the more effective treatments based on the available data at the time via the outcome-adaptive randomization. More frequent interim monitoring can drop the ineffective treatments early for futility. New treatments can be added, and effective treatments can graduate for further evaluation. The concepts of the basket design and the N-of-1 design have also been proposed to study the targeted agents based on patients' genomic profiles across different disease sites. Information collected from such trials can be put together to form a continuously updated knowledge database for adaptive learning. Statistical methods such as the hierarchical model and Individual level meta-analysis can be applied to model data collected in the knowledge database to estimate the treatment effect, the marker effect, and their interactions. A platform of a clinical trial infrastructure can be established to facilitate the conduct of such trials. In the talk, pros and cons of various designs will be discussed. Examples and lessons learned from the recently completed BATTLE trial and the on-going BATTLE-2 trial in non-small cell lung cancer will be given. Citation Format: J. Jack Lee. Adaptive designs and adaptive learning for biomarker-based clinical trials for targeted therapies. [abstract]. In: Proceedings of the AACR-IASLC Joint Conference on Molecular Origins of Lung Cancer; 2014 Jan 6-9; San Diego, CA. Philadelphia (PA): AACR; Clin Cancer Res 2014;20(2Suppl):Abstract nr IA06.

Efficient Adaptive Randomization and Stopping Rules in Multi-arm Clinical Trials for Testing a New Treatment

Motivated by applications to confirmatory clinical trials for testing a new treatment against a placebo or active control when the new treatment has k possible treatment strategies (arms)—for example, k possible doses for a new drug—we develop an asymptotic theory for efficient outcome-adaptive randomization schemes and optimal stopping rules. Our approach consists of developing asymptotic lower bounds for the expected sample sizes from the k treatment arms and the control arm and using generalized sequential likelihood ratio procedures to achieve these bounds. Implementation details of our design and analysis and comparative simulation studies are also provided.

Worth adapting? Revisiting the usefulness of outcome-adaptive randomization.

Outcome-adaptive randomization allocates more patients to the better treatments as the information accumulates in the trial. Is it worth it to apply outcome-adaptive randomization in clinical trials? Different views permeate the medical and statistical communities. We provide additional insights to the question by conducting extensive simulation studies. Trials are designed to maintain the type I error rate, achieve a specified power, and provide better treatment to patients. Generally speaking, equal randomization requires a smaller sample size and yields a smaller number of nonresponders than adaptive randomization by controlling type I and type II errors. Conversely, adaptive randomization produces a higher overall response rate than equal randomization with or without expanding the trial to the same maximum sample size. When there are substantial treatment differences, adaptive randomization can yield a higher overall response rate as well as a lower average sample size and a smaller number of nonresponders. Similar results are found for the survival endpoint. The differences between adaptive randomization and equal randomization quickly diminish with early stopping of a trial due to efficacy or futility. In summary, equal randomization maintains balanced allocation throughout the trial and reaches the specified statistical power with a smaller number of patients in the trial. If the trial's results are positive, equal randomization may lead to early approval of the treatment. Adaptive randomization focuses on treating patients best in the trial. Adaptive randomization may be preferred when the difference in efficacy between treatments is large or when the number of patients available is limited.

Looking Ahead: Clinical Trial Design in Adult Congenital Heart Disease

Continued growth in the field of adult congenital heart disease has given rise to a critical mass of patients, investigators and resources dedicated to advancing knowledge through research. Impressive gains over the past decade have largely been driven by successful observational studies. Despite the privileged role of clinical trials in the hierarchy of evidence-based medicine, few such studies have thus far been conducted in adult congenital heart disease. Obstacles and challenges particular to clinical trials in adult congenital heart disease are addressed in this review. In looking ahead, examples of creative methodological solutions to maximizing the efficiency of clinical trials for rare diseases are discussed, including Bayesian analyses, outcome-adaptive randomization and internal pilot studies.

On the Usefulness of Outcome-Adaptive Randomization

On the Usefulness of Outcome-Adaptive Randomization

Outcome-Adaptive Randomization: Is It Useful?

Outcome-adaptive randomization is one of the possible elements of an adaptive trial design in which the ratio of patients randomly assigned to the experimental treatment arm versus the control treatment arm changes from 1:1 over time to randomly assigning a higher proportion of patients to the arm that is doing better. Outcome-adaptive randomization has intuitive appeal in that, on average, a higher proportion of patients will be treated on the better treatment arm (if there is one). In both the randomized phase II and phase III settings with a short-term binary outcome, we compare outcome-adaptive randomization with designs that use 1:1 and 2:1 fixed-ratio randomizations (in the latter, twice as many patients are randomly assigned to the experimental treatment arm). The comparisons are done in terms of required sample sizes, the numbers and proportions of patients having an inferior outcome, and we restrict attention to the situation in which one treatment arm is a control treatment (rather than the less common situation of two experimental treatments without a control treatment). With no differential patient accrual rates because of the trial design, we find no benefits to outcome-adaptive randomization over 1:1 randomization, and we recommend the latter. If it is thought that the patient accrual rates will be substantially higher because of the possibility of a higher proportion of patients being randomly assigned to the experimental treatment (because the trial will be more attractive to patients and clinicians), we recommend using a fixed 2:1 randomization instead of an outcome-adaptive randomization.