Bayesian Adaptive Designs Research Articles

Implementation of statistical features of a Bayesian two-armed responsive adaptive randomization trial with post hoc analysis of time trend drift

ABSTRACT Bayesian adaptive designs with response adaptive randomization (RAR) have the potential to benefit more participants in a clinical trial. While there are many papers that describe RAR designs and results, there is a scarcity of works reporting the details of RAR implementation from a statistical point exclusively. In this paper, we introduce the statistical methodology and implementation of the trial Changing the Default (CTD). CTD is a single-center prospective RAR comparative effectiveness trial to compare opt-in to opt-out tobacco treatment approaches for hospitalized patients. The design assumed an uninformative prior, conservative initial allocation ratio, and a higher threshold for stopping for success to protect results from statistical bias. A particular emerging concern of RAR designs is the possibility that time trends will occur during the implementation of a trial. If there is a time trend and the analytic plan does not prespecify an appropriate model, this could lead to a biased trial. Adjustment for time trend was not pre-specified in CTD, but post hoc time-adjusted analysis showed no presence of influential drift. This trial was an example of a successful two-armed confirmatory trial with a Bayesian adaptive design using response adaptive randomization.

Two-stage randomized clinical trials with a right-censored endpoint: Comparison of frequentist and Bayesian adaptive designs.

Adaptive randomized clinical trials are of major interest when dealing with a time-to-event outcome in a prolonged observation window. No consensus exists either to define stopping boundaries or to combine values or test statistics in the terminal analysis in the case of a frequentist design and sample size adaptation. In a one-sided setting, we compared three frequentist approaches using stopping boundaries relying on -spending functions and a Bayesian monitoring setting with boundaries based on the posterior distribution of the log-hazard ratio. All designs comprised a single interim analysis with an efficacy stopping rule and the possibility of sample size adaptation at this interim step. Three frequentist approaches were defined based on the terminal analysis: combination of stagewise statistics (Wassmer) or of values (Desseaux), or on patientwise splitting (Jörgens), and we compared the results with those of the Bayesian monitoring approach (Freedman). These different approaches were evaluated in a simulation study and then illustrated on a real dataset from a randomized clinical trial conducted in elderly patients with chronic lymphocytic leukemia. All approaches controlled for the type I error rate, except for the Bayesian monitoring approach, and yielded satisfactory power. It appears that the frequentist approaches are the best in underpowered trials. The power of all the approaches was affected by the violation of the proportional hazards (PH) assumption. For adaptive designs with a survival endpoint and a one-sided alternative hypothesis, the Wassmer and Jörgens approaches after sample size adaptation should be preferred, unless violation of PH is suspected.

Developments in the Design, Conduct, and Reporting of Child Health Trials.

To identify priority areas to improve the design, conduct, and reporting of pediatric clinical trials, the international expert network, Standards for Research (StaR) in Child Health, was assembled and published the first 6 Standards in Pediatrics in 2012. After a recent review summarizing the 247 publications by StaR Child Health authors that highlight research practices that add value and reduce research "waste," the current review assesses the progress in key child health trial methods areas: consent and recruitment, containing risk of bias, roles of data monitoring committees, appropriate sample size calculations, outcome selection and measurement, and age groups for pediatric trials. Although meaningful change has occurred within the child health research ecosystem, measurable progress is still disappointingly slow. In this context, we identify and review emerging trends that will advance the agenda of increased clinical usefulness of pediatric trials, including patient and public engagement, Bayesian statistical approaches, adaptive designs, and platform trials. We explore how implementation science approaches could be applied to effect measurable improvements in the design, conducted, and reporting of child health research.

Measuring the robustness of predictive probability for early stopping in two-group comparisons

Physical experiments are often expensive and time-consuming. Test engineers must certify the compatibility of aircraft and their weapon systems before they can be deployed in the field, but the testing required is time consuming, expensive, and resource limited. Adopting Bayesian adaptive designs is a promising way to borrow from the successes seen in the clinical trials domain. The use of predictive probability (PP) to stop testing early and make faster decisions is particularly appealing given the aforementioned constraints. Given the high-consequence nature of the tests performed in the national security space, a strong understanding of new methods is required before being deployed. Although PP has been thoroughly studied for binary data, there is less work with continuous data, where many reliability studies are interested in certifying the specification limits of components. A simulation study evaluating the robustness of this approach indicates early stopping based on PP is reasonably robust to minor assumption violations, especially when only a few interim analyses are conducted. The simulation study also compares PP to conditional power, showing its relative strengths and weaknesses. A post-hoc analysis exploring whether release requirements of a weapon system from an aircraft are within specification with desired reliability resulted in stopping the experiment early and saving 33% of the experimental runs.

Covariate adjustment in Bayesian adaptive randomized controlled trials.

In conventional randomized controlled trials, adjustment for baseline values of covariates known to be at least moderately associated with the outcome increases the power of the trial. Recent work has shown a particular benefit for more flexible frequentist designs, such as information adaptive and adaptive multi-arm designs. However, covariate adjustment has not been characterized within the more flexible Bayesian adaptive designs, despite their growing popularity. We focus on a subclass of these which allow for early stopping at an interim analysis given evidence of treatment superiority. We consider both collapsible and non-collapsible estimands and show how to obtain posterior samples of marginal estimands from adjusted analyses. We describe several estimands for three common outcome types. We perform a simulation study to assess the impact of covariate adjustment using a variety of adjustment models in several different scenarios. This is followed by a real-world application of the compared approaches to a COVID-19 trial with a binary endpoint. For all scenarios, it is shown that covariate adjustment increases power and the probability of stopping the trials early, and decreases the expected sample sizes as compared to unadjusted analyses.

Optimizing dose-schedule regimens with bayesian adaptive designs: opportunities and challenges

Due to the small sample sizes in early-phase clinical trials, the toxicity and efficacy profiles of the dose-schedule regimens determined for subsequent trials may not be well established. The recent development of novel anti-tumor treatments and combination therapies further complicates the problem. Therefore, there is an increasing recognition of the essential place of optimizing dose-schedule regimens, and new strategies are now urgently needed. Bayesian adaptive designs provide a potentially effective way to evaluate several doses and schedules simultaneously in a single clinical trial with higher efficiency, but real-world implementation examples of such adaptive designs are still few. In this paper, we cover the critical factors associated with dose-schedule optimization and review the related innovative Bayesian adaptive designs. The assumptions, characteristics, limitations, and application scenarios of those designs are introduced. The review also summarizes some unresolved issues and future research opportunities for dose-schedule optimization.

Bayesian Statistics for Medical Devices: Progress Since 2010.

The use of Bayesian statistics to support regulatory evaluation of medical devices began in the late 1990s. We review the literature, focusing on recent developments of Bayesian methods, including hierarchical modeling of studies and subgroups, borrowing strength from prior data, effective sample size, Bayesian adaptive designs, pediatric extrapolation, benefit-risk decision analysis, use of real-world evidence, and diagnostic device evaluation. We illustrate how these developments were utilized in recent medical device evaluations. In Supplementary Material, we provide a list of medical devices for which Bayesian statistics were used to support approval by the US Food and Drug Administration (FDA), including those since 2010, the year the FDA published their guidance on Bayesian statistics for medical devices. We conclude with a discussion of current and future challenges and opportunities for Bayesian statistics, including artificial intelligence/machine learning (AI/ML) Bayesian modeling, uncertainty quantification, Bayesian approaches using propensity scores, and computational challenges for high dimensional data and models.

Bayesian Adaptive Randomization with Compound Utility Functions

Bayesian adaptive designs formalize the use of previous knowledge at the planning stage of an experiment, permitting recursive updating of the prior information. They often make use of utility functions, while also allowing for randomization. We review frequentist and Bayesian adaptive design methods and show that some of the frequentist adaptive design methodology can also be employed in a Bayesian context. We use compound utility functions for the Bayesian designs, that are a trade-off between an optimal design information criterion, that represents the acquisition of scientific knowledge, and some ethical or utilitarian gain. We focus on binary response models on two groups with independent Beta prior distributions on the success probabilities. The treatment allocation is shown to converge to the allocation that produces the maximum utility. Special cases are the Bayesian Randomized (simply) Adaptive Compound (BRAC) design, an extension of the frequentist Sequential Maximum Likelihood (SML) design and the Bayesian Randomized (doubly) Adaptive Compound Efficient (BRACE) design, a generalization of the Efficient Randomized Adaptive DEsign (ERADE). Numerical simulation studies compare BRAC with BRACE when D-optimality is the information criterion chosen. In analogy with the frequentist theory, the BRACE-D design appears more efficient than the BRAC-D design.

57: INTENSIVIST PHYSICIANS’ KNOWLEDGE AND ATTITUDES ABOUT BAYESIAN ADAPTIVE CLINICAL TRIALS

Introduction: Randomized clinical trials are critical to evidence-based medicine but have drawbacks including time and cost. Bayesian adaptive designs attempt to address some of these limitations and are increasingly common in critical care. However, the complexity of such trials raises concerns that clinicians may not adequately understand their methods or act on their results. There has been no systematic investigation of clinician understanding or acceptance of Bayesian trials. We sought to determine intensivist physicians’ understanding and acceptance of Bayesian trials compared to more traditional frequentist trials. Methods: We surveyed US intensivists as part of a longitudinal survey of how physicians develop and update treatment preferences in response to new evidence. We randomized participants to review an abstract reporting a hypothetical new drug trial that used a) Bayesian adaptive methods (“Bayesian group”) or b) traditional frequentist methods (“frequentist group”). The same simulated trial data were used for each abstract. Participants were asked eight Likert style questions to evaluate acceptance, self-perceived understanding, and measured understanding of the trial. Groups were compared using proportional odds logistic regression with an odds ratio >1 indicating lower Bayesian group acceptance or understanding. Results: The response rate was 273 of 592 surveys sent (46.1%). For the two questions related to general acceptance there were no significant differences between groups (OR=1.19 p=0.464 and OR=1.46 p=0.099). However, for the two questions related to self-perceived understanding the frequentist group scored significantly higher (OR=2.6 p< 0.001 and OR=3.21 p< 0.001). For the four questions related to measured understanding, the frequentist group scored significantly higher on three of four (OR=2.06 p=0.001, OR=1.49 p=0.077, OR=2.94 p< 0.001, and OR=1.60 p=0.041). Conclusions: We found no difference in Bayesian adaptive and frequentist trial acceptance among intensivists, but they had lower understanding of Bayesian trials. While efforts are needed to promote understanding of complex Bayesian trial designs, researchers and sponsors should confidently utilize these methods as clinicians are similarly accepting of results compared to traditional trials.

A comparison of clinical development pathways to advance tuberculosis regimen development.

Current tuberculosis (TB) regimen development pathways are slow and in urgent need of innovation. We investigated novel phase IIc and seamless phase II/III trials utilizing multi-arm multi-stage and Bayesian response adaptive randomization trial designs to select promising combination regimens in a platform adaptive trial. Clinical trial simulation tools were built using predictive and validated parametric survival models of time to culture conversion (intermediate endpoint) and time to TB-related unfavorable outcome (final endpoint). This integrative clinical trial simulation tool was used to explore and optimize design parameters for aforementioned trial designs. Both multi-arm multi-stage and Bayesian response adaptive randomization designs were able to reliably graduate desirable regimens in ≥ 95% of trial simulations and reliably stop suboptimal regimens in ≥ 90% of trial simulations. Overall, adaptive phase IIc designs reduced patient enrollment by 17% and 25% with multi-arm multi-stage and Bayesian response adaptive randomization designs respectively compared to the conventional sequential approach, while seamless designs reduced study duration by 2.6 and 3.5years respectively (typically ≥ 8.5years for standard sequential approach). In this study, we demonstrate that adaptive trial designs are suitable for TB regimen development, and we provide plausible design parameters for a platform adaptive trial. Ultimately trial design and specification of design parameters will depend on clinical trial objectives. To support decision-making for clinical trial designs in contemporary TB regimen development, we provide a flexible clinical trial simulation tool that can be used to explore and optimize design features and parameters.

A novel Bayesian adaptive design incorporating both primary and secondary endpoints for randomized IIB chemoprevention study of women at increased risk for breast cancer

BackgroundOur randomized controlled clinical trial will explore the potential of bazedoxifene plus conjugated estrogen to modulate breast tissue-based risk biomarkers as a surrogate for breast cancer risk reduction. This paper investigates the statistical design features of the trial and the rationale for the final choice of its design. Group sequential designs are a popular design approach to allow a trial to stop early for success or futility, potentially saving time and money over a fixed trial design. While Bayesian adaptive designs enjoy the same properties as group sequential designs, they have the added benefit of using prior information as well as inferential interpretation conditional on the data. Whether a frequentist or Bayesian trial, most adaptive designs have interim analyses that allow for early stopping, typically utilizing only the primary endpoint. A drawback to this approach is that the study may not have enough data for adequate comparisons of a single, key secondary endpoint. This can happen, for example, if the secondary endpoint has a smaller effect than the primary endpoint.MethodsIn this paper, we investigate a trial design called two-endpoint adaptive, which stops early only if a criterion is met for primary and secondary endpoints. The approach focuses the final analysis on the primary endpoint but ensures adequate data for the secondary analysis. Our study has two arms with a primary (change in mammographic fibroglandular volume) and secondary endpoint (change in mammary tissue Ki-67).ResultsWe present operating characteristics including power, trial duration, and type I error rate and discuss the value and risks of modeling Bayesian group sequential designs with primary and secondary endpoints, comparing against alternative designs. The results indicate that the two-endpoint adaptive design has better operating characteristics than competing designs if one is concerned about having adequate information for a key secondary endpoint.DiscussionOur approach balances trial speed and the need for information on the single, key secondary endpoint.

A Bayesian adaptive design for clinical trials of rare efficacy outcomes with multiple definitions.

Bayesian adaptive designs for clinical trials have gained popularity in the recent years due to the flexibility and efficiency that they offer. We consider the scenario where the outcome of interest comprises events with relatively low risk of occurrence and different case definitions resulting in varying control group risk assumptions. This is a scenario that occurs frequently for infectious diseases in global health research. We propose a Bayesian adaptive design that incorporates different case definitions of the outcome of interest that vary in stringency. A set of stopping rules are proposed where superiority and futility may be concluded with respect to different outcome definitions and therefore maintain a realistic probability of stopping in trials with low event rates. Through a simulation study, a variety of stopping rules and design configurations are compared. The simulation results are provided in an interactive web application that allows the user to explore and compare the design operating characteristics for a variety of assumptions and design parameters with respect to different outcome definitions. The results for select simulation scenarios are provided in the article. Bayesian adaptive designs offer the potential for maximizing the information learned from the data collected through clinical trials. The proposed design enables monitoring and utilizing multiple composite outcomes based on rare events to optimize the trial design operating characteristics.

Designing phase I oncology dose escalation using dose-exposure-toxicity models as a complementary approach to model-based dose-toxicity models.

One of the objectives of oncology phase I dose‐escalation studies has been to determine the maximum tolerated dose (MTD). Although MTD is no longer set as the dose for further development in contemporary oncology drug development, MTD determination is still important for informing the therapeutic index. Bayesian adaptive model‐based designs are becoming mainstream in oncology first‐in‐human trials. Herein, we illustrate via simulations the use of systemic exposure in Bayesian adaptive dose–toxicity models to estimate MTD. We extend traditional dose–toxicity models to incorporate pharmacokinetic exposure, which provides information on exposure–toxicity relationships. We pursue dose escalation until the maximum tolerated exposure (corresponding to the MTD) is reached. By leveraging pharmacokinetics, dose escalation considers exposure and interindividual variability on a continuous rather than discrete domain, offering additional information for dose‐escalation decisions. To demonstrate this, we generated 1000 simulations (starting dose of 1/25th the reference dose and six dose levels) for several different scenarios. Both rule‐based and model‐based designs were compared using metrics of potential safety, accuracy, and reliability. The mean results over simulations and different toxicity scenarios showed that model‐based designs were better than rule‐based methods and that exposure–toxicity model‐based methods have the potential to valuably complement dose–toxicity model‐based methods. Exposure–toxicity model‐based methods had decreased underdose risk accompanied by a relatively smaller increase in overdose risk, resulting in improved net reliability. MTD estimation accuracy was compromised when exposure variability was large, emphasizing the importance of appropriate control of pharmacokinetic variability in phase I dose‐escalation studies.

Bayesian adaptive clinical trial designs for respiratory medicine.

The use of Bayesian adaptive designs for clinical trials has increased in recent years, particularly during the COVID‐19 pandemic. Bayesian adaptive designs offer a flexible and efficient framework for conducting clinical trials and may provide results that are more useful and natural to interpret for clinicians, compared to traditional approaches. In this review, we provide an introduction to Bayesian adaptive designs and discuss its use in recent clinical trials conducted in respiratory medicine. We illustrate this approach by constructing a Bayesian adaptive design for a multi‐arm trial that compares two non‐invasive ventilation treatments to standard oxygen therapy for patients with acute cardiogenic pulmonary oedema. We highlight the benefits and some of the challenges involved in designing and implementing Bayesian adaptive trials.

Abstract A034: Bayesian adaptive design for phase I drug combination trials

Abstract Drug combination therapy is the most commonly used approach to overcome acquired resistance to cancer therapy. Designing phase I drug combination trials faces several challenges beyond the conventional single-agent dose escalation studies, e.g., two-dimensional dose space and partial order in toxicity. In this presentation, we will overview novel Bayesian adaptive designs for drug combination trials. In particularly, we will focus on model-assisted designs that are simply to implement, while yielding superior operating characteristics. Trial example and software will be provided to illustrate the application of the novel designs. Citation Format: Ying Yuan, Suyu Liu. Bayesian adaptive design for phase I drug combination trials [abstract]. In: Proceedings of the AACR Special Conference on the Evolutionary Dynamics in Carcinogenesis and Response to Therapy; 2022 Mar 14-17. Philadelphia (PA): AACR; Cancer Res 2022;82(10 Suppl):Abstract nr A034.

Comparison of Bayesian vs Frequentist Adaptive Trial Design in the Stroke Hyperglycemia Insulin Network Effort Trial

Bayesian adaptive trial design has the potential to create more efficient clinical trials. However, a barrier to the uptake of bayesian adaptive designs for confirmatory trials is limited experience with how they may perform compared with a frequentist design. To compare the performance of a bayesian and a frequentist adaptive clinical trial design. This prospective cohort study compared 2 trial designs for a completed multicenter acute stroke trial conducted within a National Institutes of Health neurologic emergencies clinical trials network, with individual patient-level data, including the timing and order of enrollments and outcome ascertainment, from 1151 patients with acute stroke and hyperglycemia randomized to receive intensive or standard insulin therapy. The implemented frequentist design had group sequential boundaries for efficacy and futility interim analyses at 90 days after randomization for 500, 700, 900, and 1100 patients. The bayesian alternative used predictive probability of trial success to govern early termination for efficacy and futility with a first interim analysis at 500 randomized patients and subsequent interims after every 100 randomizations. The main outcome was the sample size at end of study, which was defined as the sample size at which each of the studies stopped accrual of patients. Data were collected from 1151 patients. As conducted, the frequentist design passed the futility boundary after 936 participants were randomized. Using the same sequence and timing of randomization and outcome data, the bayesian alternative crossed the futility boundary approximately 3 months earlier after 800 participants were randomized. Both trial designs stopped for futility before reaching the planned maximum sample size. In both cases, the clinical community and patients would benefit from learning the answer to the trial's primary question earlier. The common feature across the 2 designs was frequent interim analyses to stop early for efficacy or for futility. Differences between how these analyses were implemented between the 2 trials resulted in the differences in early stopping.

Improving clinical trials using Bayesian adaptive designs: a breast cancer example

BackgroundTo perform virtual re-executions of a breast cancer clinical trial with a time-to-event outcome to demonstrate what would have happened if the trial had used various Bayesian adaptive designs instead.MethodsWe aimed to retrospectively “re-execute” a randomised controlled trial that compared two chemotherapy regimens for women with metastatic breast cancer (ANZ 9311) using Bayesian adaptive designs. We used computer simulations to estimate the power and sample sizes of a large number of different candidate designs and shortlisted designs with the either highest power or the lowest average sample size. Using the real-world data, we explored what would have happened had ANZ 9311 been conducted using these shortlisted designs.ResultsWe shortlisted ten adaptive designs that had higher power, lower average sample size, and a lower false positive rate, compared to the original trial design. Adaptive designs that prioritised small sample size reduced the average sample size by up to 37% when there was no clinical effect and by up to 17% at the target clinical effect. Adaptive designs that prioritised high power increased power by up to 5.9 percentage points without a corresponding increase in type I error. The performance of the adaptive designs when applied to the real-world ANZ 9311 data was consistent with the simulations.ConclusionThe shortlisted Bayesian adaptive designs improved power or lowered the average sample size substantially. When designing new oncology trials, researchers should consider whether a Bayesian adaptive design may be beneficial.

Bayesian Optimal Phase II Design for Randomized Clinical Trials

Randomized clinical trials are the gold standard to evaluate the efficacy of an experimental treatment. We propose a flexible Bayesian optimal phase II (BOP2) design for two-arm randomized trials. The proposed two-arm BOP2 design is flexible and can handle single, multiple primary and coprimary endpoints for superiority and noninferiority trials under a unified framework. It also allows users to specify the number and timing of interim analyses to meet clinical needs. While enjoying the flexibility of Bayesian adaptive designs, the two-arm BOP2 design explicitly controls the Type I error rate and is optimal for maximizing power, thereby ensuring desirable frequentist operating characteristics. Another feature of the two-arm BOP2 design is that its decision rule can be tabulated and included in the trial protocol prior to trial commence. To conduct the trial, no complicated Bayesian calculation is needed; clinicians can simply look up the table and make go/no-go decisions. Simulation studies show that the two-arm BOP2 design has desirable operating characteristics. Easy-to-use online application is freely available at www.trialdesign.org to facilitate the use of the two-arm BOP2 design in clinical trials.

Estimating design operating characteristics in Bayesian adaptive clinical trials.

Bayesian adaptive designs have gained popularity in all phases of clinical trials with numerous new developments in the past few decades. During the COVID‐19 pandemic, the need to establish evidence for the effectiveness of vaccines, therapeutic treatments, and policies that could resolve or control the crisis emphasized the advantages offered by efficient and flexible clinical trial designs. In many COVID‐19 clinical trials, because of the high level of uncertainty, Bayesian adaptive designs were considered advantageous. Designing Bayesian adaptive trials, however, requires extensive simulation studies that are generally considered challenging, particularly in time‐sensitive settings such as a pandemic. In this article, we propose a set of methods for efficient estimation and uncertainty quantification for design operating characteristics of Bayesian adaptive trials. Specifically, we model the sampling distribution of Bayesian probability statements that are commonly used as the basis of decision making. To showcase the implementation and performance of the proposed approach, we use a clinical trial design with an ordinal disease‐progression scale endpoint that was popular among COVID‐19 trials. However, the proposed methodology may be applied generally in the clinical trial context where design operating characteristics cannot be obtained analytically.

Novel bayesian adaptive early phase designs to accelerate the development of CAR T-cell therapy

Chimeric antigen receptor (CAR) T-cell therapy has revolutionized cancer treatment, particularly for hematopoietic malignancies. CAR T-cell therapy is a living drug with fundamentally different characteristics from those of other therapies. For example, CAR T-cell therapy efficacy may not increase with dose, and dose-limiting toxicity is rarely observed in the therapeutic dose range. Consequently, the conventional trial design paradigm is not suitable for the development of CAR T-cell therapy. Here, we review and introduce the phase I-II trial design paradigm to optimize the dose of CAR T-cell therapy on the basis of both toxicity and efficacy. We describe several novel Bayesian model-assisted designs, including BOIN12 and U-BOIN, which are simple to implement and have excellent operating characteristics for identifying the optimal biological dose for CAR T-cell therapy. Examples and software are provided to facilitate the use of these novel designs to accelerate the development of CAR T-cell therapy.