Bayesian Sequential Design Research Articles

Bayesian Sequential Design of Computer Experiments for Quantile Set Inversion

We consider an unknown multivariate function representing a system—such as a complex numerical simulator—taking both deterministic and uncertain inputs. Our objective is to estimate the set of deterministic inputs leading to outputs whose probability (with respect to the distribution of the uncertain inputs) of belonging to a given set is less than a given threshold. This problem, which we call Quantile Set Inversion (QSI), occurs for instance in the context of robust (reliability-based) optimization problems, when looking for the set of solutions that satisfy the constraints with sufficiently large probability. To solve the QSI problem we propose a Bayesian strategy, based on Gaussian process modeling and the Stepwise Uncertainty Reduction (SUR) principle, to sequentially choose the points at which the function should be evaluated to efficiently approximate the set of interest. We illustrate the performance and interest of the proposed SUR strategy through several numerical experiments.

A Pilot Study of the CD38 Antagonist Daratumumab in Patients with Metastatic Renal Cell Carcinoma or Muscle-Invasive Bladder Cancer.

We performed a pilot study of daratumumab (an mAb directed against CD38) in muscle-invasive bladder cancer (MIBC) and treatment-refractory metastatic renal cell carcinoma (mRCC). Patients with MIBC underwent baseline transurethral resection of the bladder tumor followed by four weekly doses of daratumumab prior to cystectomy. Patients with mRCC underwent baseline and sequential biopsies after eight weekly doses. The primary endpoint was safety. The secondary endpoints were pathologic complete response rate for the MIBC cohort and objective response rate and progression-free survival for the mRCC cohort. Exploratory analyses included immune monitoring and overall survival. A Bayesian sequential monitoring design for toxicity was used for excessive toxicity. In both the MIBC (n = 8) and mRCC (n = 8) cohorts, no toxicity events were encountered. In the MIBC cohort, one patient experienced pathologic complete response rate. In the mRCC cohort, no objective responses were reported, and the median progression-free survival was 1.5 months (95% confidence interval, 1.1-1.8 months). Immune monitoring found significant reductions in NK cells in circulation in both cohorts after treatment. In the tissue analysis, IHC found evidence of diminished CD38 presence in mRCC with treatment, whereas the baseline levels in MIBC were low. Treatment with daratumumab was safe. No signal of efficacy was detected in mRCC, and conclusions on the activity in MIBC were limited. Evidence of daratumumab targeting CD38 was detected in circulating immune cells and within the tumor microenvironment of mRCC and MIBC. In this prospective clinical trial of daratumumab, treatment in patients with MIBC and mRCC was safe. Limited efficacy was observed. Treatment with daratumumab resulted in CD38-expressing immune cell subsets to be targeted both in circulation and within the tumor microenvironment.

Bayesian sequential designs in studies with multilevel data.

In many studies in the social and behavioral sciences, the data have a multilevel structure, with subjects nested within clusters. In the design phase of such a study, the number of clusters to achieve a desired power level has to be calculated. This requires a priori estimates of the effect size and intraclass correlation coefficient. If these estimates are incorrect, the study may be under- or overpowered. This may be overcome by using a group-sequential design, where interim tests are done at various points in time of the study. Based on interim test results, a decision is made to either include additional clusters or to reject the null hypothesis and conclude the study. This contribution introduces Bayesian sequential designs as an alternative to group-sequential designs. This approach compares various hypotheses based on the support in the data for each of them. If neither hypothesis receives a sufficient degree of support, additional clusters are included in the study and the Bayes factor is recalculated. This procedure continues until one of the hypotheses receives sufficient support. This paper explains how the Bayes factor is used as a measure of support for a hypothesis and how a Bayesian sequential design is conducted. A simulation study in the setting of a two-group comparison was conducted to study the effects of the minimum and maximum number of clusters per group and the desired degree of support. It is concluded that Bayesian sequential designs are a flexible alternative to the group sequential design.

Bayesian Sequential Design for Identifying and Ranking Effective Patient Subgroups in Precision Medicine in the Case of Counting Outcome Data with Inflated Zeros.

Precision medicine is revolutionizing health care, particularly by addressing patient variability due to different biological profiles. As traditional treatments may not always be appropriate for certain patient subsets, the rise of biomarker-stratified clinical trials has driven the need for innovative methods. We introduced a Bayesian sequential scheme to evaluate therapeutic interventions in an intensive care unit setting, focusing on complex endpoints characterized by an excess of zeros and right truncation. By using a zero-inflated truncated Poisson model, we efficiently addressed this data complexity. The posterior distribution of rankings and the surface under the cumulative ranking curve (SUCRA) approach provided a comprehensive ranking of the subgroups studied. Different subsets of subgroups were evaluated depending on the availability of biomarker data. Interim analyses, accounting for early stopping for efficacy, were an integral aspect of our design. The simulation study demonstrated a high proportion of correct identification of the subgroup which is the most predictive of the treatment effect, as well as satisfactory false positive and true positive rates. As the role of personalized medicine grows, especially in the intensive care setting, it is critical to have designs that can manage complicated endpoints and that can control for decision error. Our method seems promising in this challenging context.

Bayesian three-stage a-optimal design for generalized linear models

Bayesian sequential designs are increasingly receiving attention in recent years, especially in clinical trials and biomedical research. Bayesian sequential design process can utilize the available prior information of the unknown parameters so that a better design can be achieved. In this paper, a hybrid computational method, which consists of the combination of a rough global optima search and a more precise local optima search, is proposed to efficiently search for the Bayesian A-optimal designs for multi-variable generalized linear models. Specifically, Poisson regression models and logistic regression models are investigated. Designs are examined for a range of prior distributions and the equivalence theorem is used to verify the design optimality. Design efficiency for various models were examined. Furthermore, the idea of the Bayesian sequential design is introduced and the Bayesian three-stage A-optimal design approach is introduced for generalized linear models. With the incorporation of the first stage data information into the second stage, the second stage data information into the third stage, the three-stage design procedure improved the design efficiency and produce more accurate and robust designs. The Bayesian three-stage A-optimal designs for Poisson and logistic regression models are evaluated based on simulation studies. The Bayesian three-stage A-optimal design is superior to the two-stage A-optimal design approach in terms of design optimality and efficiency criteria.

Bayesian sample size planning for developmental studies

AbstractRunning developmental experiments, particularly with infants, is often time‐consuming and intensive, and the recruitment of participants is hard and expensive. Thus, an important goal for developmental researchers is to optimize sampling plans such that neither too many nor too few participants are tested given the hypothesis of interest. One approach that enables such optimization is the use of Bayesian sequential designs. The use of such sequential designs allows data collection to be terminated as soon as the evidence is deemed sufficiently strong, without compromising the interpretability of the test outcome. In this tutorial, we illustrate how to plan a Bayesian sequential testing design prior to data collection by the method of Bayes factor design analysis—the Bayesian equivalent of power analysis—and discuss the relevance of this for developmental psychologists. The tutorial provides a step‐by‐step guide to perform such analyses, and the methods are illustrated using commonly used statistics in a typical infant‐looking time paradigm such that researchers can easily adapt these methods for their studies.Highlights Bayesian Sequential Testing can be used to optimize sample sizes and save on data collection. Bayes Factor Design Analysis can be used to analyze a sequential testing study prior to data collection. Step‐by‐step guide for performing Bayes Sequential Testing and Bayes Factor Design Analysis.

Uncertainty Quantification and Experimental Design for Large-Scale Linear Inverse Problems under Gaussian Process Priors

We consider the use of Gaussian process (GP) priors for solving inverse problems in a Bayesian framework. As is well known, the computational complexity of GPs scales cubically in the number of datapoints. Here we show that in the context of inverse problems involving integral operators, one faces additional difficulties that hinder inversion on large grids. Furthermore, in that context, covariance matrices can become too large to be stored. By leveraging recent results about sequential disintegrations of Gaussian measures, we are able to introduce an implicit representation of posterior covariance matrices that reduces the memory footprint by only storing low rank intermediate matrices, while allowing individual elements to be accessed on-the-fly without needing to build full posterior covariance matrices. Moreover, it allows for fast sequential inclusion of new observations. These features are crucial when considering sequential experimental design tasks. We demonstrate our approach by computing sequential data collection plans for excursion set recovery for a gravimetric inverse problem, where the goal is to provide fine resolution estimates of high density regions inside the Stromboli volcano, Italy. Sequential data collection plans are computed by extending the weighted integrated variance reduction (wIVR) criterion to inverse problems. Our results show that this criterion is able to significantly reduce the uncertainty on the excursion volume, reaching close to minimal levels of residual uncertainty. Overall, our techniques allow the advantages of probabilistic models to be brought to bear on large-scale inverse problems arising in the natural sciences. Particularly, applying the latest developments in Bayesian sequential experimental design on realistic large-scale problems opens new venues of research at a crossroads between mathematical modelling of natural phenomena, statistical data science, and active learning.

On Bayesian Sequential Clinical Trial Designs

Clinical trials usually involve sequential patient entry. When designing a clinical trial, it is often desirable to include a provision for interim analyses of accumulating data with the potential for stopping the trial early. We review Bayesian sequential clinical trial designs based on posterior probabilities, posterior predictive probabilities, and decision-theoretic frameworks. A pertinent question is whether Bayesian sequential designs need to be adjusted for the planning of interim analyses. We answer this question from three perspectives: a frequentist-oriented perspective, a calibrated Bayesian perspective, and a subjective Bayesian perspective. We also provide new insights into the likelihood principle, which is commonly tied to statistical inference and decision making in sequential clinical trials. Some theoretical results are derived, and numerical studies are conducted to illustrate and assess these designs.

Can Global Strategy Outperform Myopic Strategy in Bayesian Sequential Design?

Can Global Strategy Outperform Myopic Strategy in Bayesian Sequential Design?

Avoiding Under- and Overrecruitment in Behavioral Intervention Trials Using Bayesian Sequential Designs: Tutorial.

Reducing research waste and protecting research participants from unnecessary harm should be top priorities for researchers studying interventions. However, the traditional use of fixed sample sizes exposes trials to risks of under- and overrecruitment by requiring that effect sizes be determined a priori. One mitigating approach is to adopt a Bayesian sequential design, which enables evaluation of the available evidence continuously over the trial period to decide when to stop recruitment. Target criteria are defined, which encode researchers' intentions for what is considered findings of interest, and the trial is stopped once the scientific question is sufficiently addressed. In this tutorial, we revisit a trial of a digital alcohol intervention that used a fixed sample size of 2129 participants. We show that had a Bayesian sequential design been used, the trial could have ended after collecting data from approximately 300 participants. This would have meant exposing far fewer individuals to trial procedures, including being allocated to the waiting list control condition, and the evidence from the trial could have been made public sooner.

A Comparative Tutorial of Bayesian Sequential Design and Reinforcement Learning

Reinforcement learning (RL) is a computational approach to reward-driven learning in sequential decision problems. It implements the discovery of optimal actions by learning from an agent interacting with an environment rather than from supervised data. We contrast and compare RL with traditional sequential design, focusing on simulation-based Bayesian sequential design (BSD). Recently, there has been an increasing interest in RL techniques for healthcare applications. We introduce two related applications as motivating examples. In both applications, the sequential nature of the decisions is restricted to sequential stopping. Rather than a comprehensive survey, the focus of the discussion is on solutions using standard tools for these two relatively simple sequential stopping problems. Both problems are inspired by adaptive clinical trial design. We use examples to explain the terminology and mathematical background that underlie each framework and map one to the other. The implementations and results illustrate the many similarities between RL and BSD. The results motivate the discussion of the potential strengths and limitations of each approach.

A Bayesian data modelling framework for chemical processes using adaptive sequential design with Gaussian process regression

AbstractAccurate simulators are relied upon in the process industry for plant design and operation. Typical simulators, based on mechanistic models, require considerable resources: skilled engineers, computational time, and proprietary data. This article explores the complexities of developing a statistical modelling framework for chemical processes, focusing on inherent non‐linearity in phenomena and the difficulty of obtaining data. A Bayesian approach to modelling is forwarded in this article, utilising Bayesian sequential design to maximise information gain for each experiment. Gaussian process regression is used to provide a highly flexible model class to capture non‐linearities in the process data. A non‐linear process simulator, modelled in Aspen Plus is used as a surrogate for a real chemical process, to test the capabilities of the framework.

Bayesian sequential design for sensitivity experiments with hybrid responses

In experimental design, a common problem seen in practice is when the result includes one binary response and multiple continuous responses. However, this problem receives scant attention. Most studies pertaining to this problem usually consider the situation in which the continuous responses are independent of the stimulus level condition on the binary response. However, in many practical applications, real data show that this conditional independent assumption is not always appropriate. This article considers a new model for the dependent situation and a corresponding sequential design is proposed under the decision-theoretic framework. To deal with the problem of complex computation involved in searching for optimal designs, fast algorithms are presented using two strategies to approximate the optimal criterion, denoted as SI-optimal design and Bayesian D-optimal design, respectively. Simulation studies based on data from a Chinese chemical material factory show that the proposed methods perform well in estimating the interesting quantiles.

Positive emotion enhances conflict processing in preschoolers.

The rapid detection and resolution of conflict between opposing action tendencies is crucial for our ability to engage in goal-directed behavior. Research in adults suggests that emotions can serve as a "relevance detector" that alarms attentional and sensory systems, thereby leading to more efficient conflict processing. In contrast, previous research in children has almost exclusively stressed the impeding influence of emotion on the attentional system, as suggested by the protracted development of performance in "hot" executive function tasks. Do preschool children show a facilitative effect of emotion on conflict processing? We addressed this question applying a modified version of a color flanker task that either involved or did not involve positive emotional stimuli in preschool children (N=43, with preregistered Bayesian sequential design, aged 2.8-7.0 years). Our results show a robust conflict effect with higher error rates in incongruent compared to congruent trials. Crucially, conflict resolution was faster in emotional compared to neutral conditions. Furthermore, while efficient conflict processing increases with age, we find evidence against an age-related change in the influence of positive emotion on conflict processing. Taken together, these findings provide indication that positive emotion can trigger efficient control processes already from early on in life. In contrast to the predominant view in developmental psychology, this indicates that, depending on the role that emotion has in conflict processing, emotion may show a facilitative or impeding effect already in the preschool period.

A Sequential Predictive Power Design for a COVID Vaccine Trial

Medical investigations for therapeutics and vaccines for combating a pandemic such as COVID-19, call for flexible and adaptive trial designs that are capable of producing robust results amidst uncertainties. Here, we present a Bayesian sequential design to study the efficacy of Bacillus Calmette–Guérin (BCG) in providing protection against COVID-19 infections via its known “trained-immunity” mechanism. The main design consideration is to provide a framework to rapidly establish a proof-of-concept on the vaccine efficacy of BCG under a constantly evolving incidence rate and in the absence of prior efficacy data. The trial design is based on taking several interim looks and calculating the predictive power with the current cohort at each interim look. Decisions to stop the trial for futility or stopping enrollment for efficacy are made based on the current cohort predictive power computation. At any interim, if any of the above decisions cannot be taken then the study continues to enroll till the next interim look. Via extensive numerical studies, we show that the proposed design can achieve the desired frequentist operating characteristics, currently required by regulatory bodies while offering greater flexibility in terms of sample size and the ability to make robust interim decisions.

Autonomous Development of a Machine-Learning Model for the Plastic Response of Two-Phase Composites from Micromechanical Finite Element Models

We demonstrate a novel strategy for the autonomous development of a machine-learning model for predicting the equivalent stress–equivalent plastic strain response of a two-phase composite calibrated to micromechanical finite element models. A unique feature of the model is that it takes a user-defined three-dimensional, two-phase microstructure along with user-defined hardening laws for each constituent phase, and outputs the equivalent stress–plastic strain response of the microstructure modeled using J2-based isotropic plasticity theory for each constituent phase. Previously, this task was addressed using linear regression approaches on a large training dataset. In this work, it is demonstrated that the use of Gaussian process regression together with a Bayesian sequential design of experiments can lead to autonomous protocols for optimal generation of the training dataset and the development of the model. It is shown that this strategy dramatically reduces the time and effort expended in generating the training set.

A pilot study evaluating GSK1070806 inhibition of interleukin-18 in renal transplant delayed graft function

IntroductionDelayed graft function (DGF) following renal transplantation is a manifestation of acute kidney injury (AKI) leading to poor long-term outcome. Current treatments have limited effectiveness in preventing DGF. Interleukin-18 (IL18), a biomarker of AKI, induces interferon-γ expression and immune activation. GSK1070806, an anti-IL18 monoclonal antibody, neutralizes activated (mature) IL18 released from damaged cells following inflammasome activation. This phase IIa, single-arm trial assessed the effect of a single dose of GSK1070806 on DGF occurrence post donation after circulatory death (DCD) kidney transplantation.MethodsThe 3 mg/kg intravenous dose was selected based on prior studies and physiologically based pharmacokinetic (PBPK) modeling, indicating the high likelihood of a rapid and high level of IL18 target engagement when administered prior to kidney allograft reperfusion. Utilization of a Bayesian sequential design with a background standard-of-care DGF rate of 50% based on literature, and confirmed via extensive registry data analyses, enabled a statistical efficacy assessment with a minimal sample size. The primary endpoint was DGF frequency, defined as dialysis requirement ≤7 days post transplantation (except for hyperkalemia). Secondary endpoints included safety, pharmacokinetics and pharmacodynamic biomarkers.ResultsGSK1070806 administration was associated with IL18-GSK1070806 complex detection and increased total serum IL18 levels due to IL18 half-life prolongation induced by GSK1070806 binding. Interferon-γ−induced chemokine levels declined or remained unchanged in most patients. Although the study was concluded prior to the Bayesian-defined stopping point, 4/7 enrolled patients (57%) had DGF, exceeding the 50% standard-of-care rate, and an additional two patients, although not reaching the protocol-defined DGF definition, demonstrated poor graft function. Six of seven patients experienced serious adverse events (SAEs), including two treatment-related SAEs.ConclusionOverall, using a Bayesian design and extensive PBPK dose modeling with only a small sample size, it was deemed unlikely that GSK1070806 would be efficacious in preventing DGF in the enrolled DCD transplant population.Trial registrationNCT02723786.

Hard to Resist?

Abstract. Because more and more young people are constantly presented with the opportunity to access information and connect to others via their smartphones, they report to be in a state of permanent alertness. In the current study, we define such a state as smartphone vigilance, an awareness that one can always get connected to others in combination with a permanent readiness to respond to incoming smartphone notifications. We hypothesized that constantly resisting the urge to interact with their phones draws on response inhibition, and hence interferes with students’ ability to inhibit prepotent responses in a concurrent task. To test this, we conducted a preregistered experiment, employing a Bayesian sequential sampling design, where we manipulated smartphone visibility and smartphone notifications during a stop-signal task that measures the ability to inhibit prepotent responses. The task was constructed such that we could disentangle response inhibition from action selection. Results show that the mere visibility of a smartphone is sufficient to experience vigilance and distraction, and that this is enhanced when students receive notifications. Curiously enough, these strong experiences were unrelated to stop-signal task performance. These findings raise new questions about when and how smartphones can impact performance.

Hard to resist? The effect of smartphone visibility and notifications on response inhibition

Because more and more young people are constantly presented with the opportunity to access information and connect to others via their smartphones, they report to be in a state of permanent alertness. In the current study, we define such a state as smartphone vigilance, an awareness that one can always get connected to others in combination with a permanent readiness to respond to incoming smartphone notifications. We hypothesized that constantly resisting the urge to interact with their phones draws on response inhibition, and hence interferes with students' ability to inhibit prepotent responses in a concurrent task. To test this, we conducted a preregistered experiment, employing a Bayesian sequential sampling design, where we manipulated smartphone visibility and smartphone notifications during a stop-signal task that measures the ability to inhibit prepotent responses. The task was constructed such that we could disentangle response inhibition from action selection. Results show that the mere visibility of a smartphone is sufficient to experience vigilance and distraction, and that this is enhanced when students receive notifications. Curiously enough, these strong experiences were unrelated to stop-signal task performance. These findings raise new questions about when and how smartphones can impact performance.

A Bayesian Sequential Design for Clinical Trials With Time-to-Event Outcomes

There is increasing interest in Bayesian group sequential design because of its potential to improve efficiency in clinical trials, to shorten drug development time, and to enhance statistical inference precision without undermining the clinical trial’s integrity or validity. We propose a Bayesian sequential design for clinical trials with time-to-event outcomes and use alpha spending functions to control the overall Type I error rate. Bayes factor is adapted for decision-making at interim analyses. Algorithms are presented to make decision rules and to calculate power of the proposed tests. Sensitivity analysis is implemented to evaluate the impact of different choices of prior parameters on choosing critical values. The power of tests, the expected event size of the proposed design, and the quality of estimators are studied through simulations, and compared with the frequentist group sequential design. Simulations show that at fixed total number of events, the proposed design can achieve greater power and require smaller expected event size when appropriate priors are chosen, compared with the frequentist group sequential design. The feasibility of the proposed design is further illustrated on a real dataset. Supplementary materials for this article are available online.