Active Inference, Bayesian Optimal Design, and Expected Utility

Optimal Bayesian experimental design for subsurface flow problems

We perform a comparison study on Bayesian sequential optimal experimental design algorithms applied to linear regression in two unknowns. We transform the Bayesian sequential optimal experimental design problem into a reinforcement learning problem to determine the power of deep reinforcement learning algorithms against baselines including batch design, greedy design, dynamic programming, and approximate dynamic programming. Using KL-divergence to measure information gain in the unknown parameters, we construct objectives for each algorithm to maximize information gain. This work showcases novel comparisons between the aforementioned algorithms and provides a new application of reinforcement learning to Bayesian sequential optimal experimental design for inverse problems in linear regression with multiple parameters.

Bayesian optimal experimental design is a sub-field of statistics focused on developing methods to make efficient use of experimental resources. Any potential design is evaluated in terms of a utility function, such as the (theoretically well-justified) expected information gain (EIG); unfortunately however, under most circumstances the EIG is intractable to evaluate. In this work we build off of successful variational approaches, which optimize a parameterized variational model with respect to bounds on the EIG. Past work focused on learning a new variational model from scratch for each new design considered. Here we present a novel neural architecture that allows experimenters to optimize a single variational model that can estimate the EIG for potentially infinitely many designs. To further improve computational efficiency, we also propose to train the variational model on a significantly cheaper-to-evaluate lower bound, and show empirically that the resulting model provides an excellent guide for more accurate, but expensive to evaluate bounds on the EIG. We demonstrate the effectiveness of our technique on generalized linear models, a class of statistical models that is widely used in the analysis of controlled experiments. Experiments show that our method is able to greatly improve accuracy over existing approximation strategies, and achieve these results with far better sample efficiency.

SummaryAn optimal experimental set‐up maximizes the value of data for statistical inferences. The efficiency of strategies for finding optimal experimental set‐ups is particularly important for experiments that are time‐consuming or expensive to perform. In the situation when the experiments are modeled by partial differential equations (PDEs), multilevel methods have been proven to reduce the computational complexity of their single‐level counterparts when estimating expected values. For a setting where PDEs can model experiments, we propose two multilevel methods for estimating a popular criterion known as the expected information gain (EIG) in Bayesian optimal experimental design. We propose a multilevel double loop Monte Carlo, which is a multilevel strategy with double loop Monte Carlo, and a multilevel double loop stochastic collocation, which performs a high‐dimensional integration on sparse grids. For both methods, the Laplace approximation is used for importance sampling that significantly reduces the computational work of estimating inner expectations. The values of the method parameters are determined by minimizing the computational work, subject to satisfying the desired error tolerance. The efficiencies of the methods are demonstrated by estimating EIG for inference of the fiber orientation in composite laminate materials from an electrical impedance tomography experiment.

The Bayesian optimal experimental design (OED) problem seeks to identify data, sensor configurations, or experiments which can optimally reduce uncertainty. The goal of OED is to find an experiment that maximizes the expected information gain (EIG) about quantities of interest given prior knowledge about expected data. Therefore, within the context of seismic monitoring, we can use Bayesian OED to configure sensor networks by choosing sensor locations, types, and fidelity in order to improve our ability to identify and locate seismic sources. In this work, we develop the framework necessary to use Bayesian OED to optimize the ability to locate seismic events from arrival time data of detected seismic phases. In order to do utilize Bayesian OED we must develop four elements:1. A likelihood function that describes the uncertainty of detection and travel times; 2. A Bayesian solver that takes a prior and likelihood to identify the posterior; 3. An algorithm to compute EIG; and, 4. An optimizer that finds a sensor network which maximizes EIG. Once we have developed this framework, we can explore many relevant questions to monitoring such as: how and what multiphenomenology data can be used to optimally reduce uncertainty, how to trade off sensor fidelity and earth model uncertainty, and how sensor types, number, and locations influence uncertainty

We develop a fast and scalable computational framework to solve Bayesian optimal experimental design problems governed by partial differential equations (PDEs) with application to optimal sensor placement by maximizing expected information gain (EIG). Such problems are particularly challenging due to the curse of dimensionality for high-dimensional parameters and the expensive solution of large-scale PDEs. To address these challenges, we exploit two fundamental properties: (1) the low-rank structure of the Jacobian of the parameter-to-observable map, to extract the intrinsically low-dimensional data-informed subspace, and (2) a series of approximations of the EIG that reduce the number of PDE solves while retaining high correlation with the true EIG. Based on these properties, we propose an efficient offline-online decomposition for the optimization problem. The offline stage dominates the cost and entails precomputing all components that require PDE solves. The online stage optimizes sensor placement and does not require any PDE solves. For the online stage, we propose a new greedy algorithm that first places an initial set of sensors using leverage scores and then swaps the selected sensors with other candidates until certain convergence criteria are met, which we call a swapping greedy algorithm. We demonstrate the efficiency and scalability of the proposed method by both linear and nonlinear inverse problems. In particular, we show that the number of required PDE solves is small, independent of the parameter dimension, and only weakly dependent on the data dimension for both problems.

We address the solution of large-scale Bayesian optimal experimental design (OED) problems governed by partial differential equations (PDEs) with infinite-dimensional parameter fields. The OED problem seeks to find sensor locations that maximize the expected information gain (EIG) in the solution of the underlying Bayesian inverse problem. Computation of the EIG is usually prohibitive for PDE-based OED problems. To make the evaluation of the EIG tractable, we approximate the (PDE-based) parameter-to-observable map with a derivative-informed projected neural network (DIPNet) surrogate, which exploits the geometry, smoothness, and intrinsic low-dimensionality of the map using a small and dimension-independent number of PDE solves. The surrogate is then deployed within a greedy algorithm-based solution of the OED problem such that no further PDE solves are required. We analyze the EIG approximation error in terms of the generalization error of the DIPNet and show they are of the same order. Finally, the efficiency and accuracy of the method are demonstrated via numerical experiments on OED problems governed by inverse scattering and inverse reactive transport with up to 16,641 uncertain parameters and 100 experimental design variables, where we observe up to three orders of magnitude speedup relative to a reference double loop Monte Carlo method.

Preface . Acknowledgements. 1 Introduction to designs . 1.1 Introduction. 1.2 Stages of the research process. 1.3 Research design. 1.4 Types of research designs. 1.5 Requirements for a 'good' design. 1.6 Ethical aspects of design choice. 1.7 Exact versus approximate designs. 1.8 Examples. 1.9 Summary. 2 Designs for simple linear regression . 2.1 Design problem for a linear model. 2.2 Designs for radiation-dosage example. 2.3 Relative efficiency and sample size. 2.4 Simultaneous inference. 2.5 Optimality criteria. 2.6 Relative efficiency. 2.7 Matrix formulation of designs for linear regression. 2.8 Summary. 3 Designs for multiple linear regression analysis . 3.1 Design problem for multiple linear regression. 3.2 Designs for vocabulary-growth study. 3.3 Relative efficiency and sample size. 3.4 Simultaneous inference. 3.5 Optimality criteria for a subset of parameters. 3.6 Relative efficiency. 3.7 Designs for polynomial regression model. 3.8 The Poggendorff and Ponzo illusion study. 3.9 Uncertainty about best fitting regression models. 3.10 Matrix notation of designs for multiple regression models. 3.11 Summary. 4 Designs for analysis of variance models . 4.1 A typical design problem for an analysis of variance model. 4.2 Estimation of parameters and efficiency. 4.3 Simultaneous inference and optimality criteria. 4.4 Designs for groups under stress study. 4.5 Specific hypotheses and contrasts. 4.6 Designs for the composite faces study. 4.7 Balanced designs versus unbalanced designs. 4.8 Matrix notation for Groups under Stress study. 4.9 Summary. 5 Designs for logistic regression models . 5.1 Design problem for logistic regression. 5.2 The design. 5.3 The logistic regression model. 5.4 Approaches to deal with local optimality. 5.5 Designs for calibration of item parameters in item response theory models. 5.6 Matrix formulation of designs for logistic regression. 5.7 Summary. 6 Designs for multilevel models . 6.1 Design problem for multilevel models. 6.2 The multilevel regression model. 6.3 Cluster versus subject randomization. 6.4 Cost function. 6.5 Example: Nursing home study. 6.6 Optimal design and power. 6.7 Design effect in multilevel surveys. 6.8 Matrix formulation of the multilevel model . 6.9 Summary. 7 Longitudinal designs for repeated measurement models . 7.1 Design problem for repeated measurements. 7.2 The design. 7.3 Analysis techniques for repeated measures. 7.4 The linear mixed effects model for repeated measurement data. 7.5 Variance-covariance structures. 7.6 Estimation of parameters and efficiency. 7.7 Bone mineral density example. 7.8 Cost function. 7.9 D-optimal designs for linear mixed effects models with autocorrelated errors. 7.10 Miscellanea. 7. 11 Matrix formulation of the linear mixed effects model. 7. 12 Summary. 8 Two-treatment crossover designs . 8.1 Design problem for crossover studies. 8.2 The design. 8.3 Confounding treatment effects with nuisance effects. 8.4 The linear model for crossover designs. 8.5 Estimation of parameters and efficiency. 8.6 Cost and efficiency of the crossover design. 8.7 Optimal crossover designs for two treatments. 8.8 Matrix formulation of the mixed model for crossover designs. 8.9 Summary. 9 Alternative optimal designs for linear models . 9.1 Introduction. 9.2 Information matrix. 9.3 D A - or Ds-optimal designs. 9.4 Extrapolation optimal design. 9.5 L-optimal designs. 9.6 Bayesian optimal designs. 9.7 Minimax optimal design. 9.8 Multiple-objective optimal designs. 9.9 Summary. 10 Optimal designs for nonlinear models . 10.1 Introduction. 10.2 Linear models versus nonlinear models. 10.3 Design issues for nonlinear models. 10.4 Alternative optimal designs with examples. 10.5 Bayesian optimal designs. 10.6 Minimax optimal design. 10.7 Multiple-objective optimal designs. 10.8 Optimal design for model discrimination. 10.9 Summary. 11 Resources for the construction of optimal designs . 11.1 Introduction. 11.2 Sequential construction of optimal designs. 11.3 Exchange of design points. 11.4 Other algorithms. 11.5 Optimal design software. 11.6 A web site for finding optimal designs. 11.7 Summary. References . Author Index. Subject Index.

When optimizing an accelerated degradation testing (ADT) plan, the initial values of unknown model parameters must be pre-specified. However, it is usually difficult to obtain the exact values, since many uncertainties are embedded in these parameters. Bayesian ADT optimal design was presented to address this problem by using prior distributions to capture these uncertainties. Nevertheless, when the difference between a prior distribution and actual situation is large, the existing Bayesian optimal design might cause some over-testing or under-testing issues. For example, the implemented ADT following the optimal ADT plan consumes too much testing resources or few accelerated degradation data are obtained during the ADT. To overcome these obstacles, a Bayesian sequential step-down-stress ADT design is proposed in this article. During the sequential ADT, the test under the highest stress level is firstly conducted based on the initial prior information to quickly generate degradation data. Then, the data collected under higher stress levels are employed to construct the prior distributions for the test design under lower stress levels by using the Bayesian inference. In the process of optimization, the inverse Gaussian (IG) process is assumed to describe the degradation paths, and the Bayesian D-optimality is selected as the optimal objective. A case study on an electrical connector’s ADT plan is provided to illustrate the application of the proposed Bayesian sequential ADT design method. Compared with the results from a typical static Bayesian ADT plan, the proposed design could guarantee more stable and precise estimations of different reliability measures.

Accelerated degradation testing (ADT) is commonly used to obtain degradation data under the hasher than normal usage conditions, and to assess lifetime and reliability for high reliable and long life products in a short time. The optimal design for ADT is to propose the test plan based on the performance degradation process with a reasonable optimal criteria and the constraints of test resources. Both the traditional ADT optimal design and Bayesian ADT optimal design are similar that the historical data is needed to determine the initial values or the prior distributions of the model parameters. However, it may be inaccurate to estimate the parameter values or the prior distributions only according to the historical information, which will result in large deviations between the parameter values or the prior distributions and the actual ones. Therefore, the optimal plan could not meet the statistical accuracy. In this paper, we propose a Bayesian ADT dynamic optimal design method with sequential stress. The degradation information under the most recent stress level will be treated as the prior of the next stress level. Under the assumption of Wiener process, an optimal model with the objective of quadratic loss function is built with the decision variables, e.g. stress levels and inspection times. Finally, the simulation study is introduced to illustrate the proposed method, and the results show that Bayesian ADT optimal design with sequential stress can adjust the test plan dynamically to improve the test efficiency.

We are honored to be guest editors for this special issue of Applied Stochastic Models in Business and Industry, dedicated to Kathryn Chaloner's life and achievements. Contributions from her colleagues and friends in this volume are devoted to Kathryn's early research interest in Bayesian optimal design. She was a pioneer in this field and had a broad impact on the subsequent literature on experimental design. Kathryn was internationally known for her research in Bayesian Statistics, which included design of experiments, outlier detection, prior elicitation, and clinical trials, and for her advancements in the study of HIV/AIDS. She was an accomplished researcher, teacher, and mentor. She actively encouraged women and under-represented minority students to join the field of biostatistics and provided mentorship, support, and encouragement. The special issue contains seven papers. The first three papers present some of the recent work in Bayesian optimal designs for nonlinear models, binary responses, and experiments with adversarial components. This is followed by two articles on Bayesian designs in clinical trials. The last two articles of the special issue involve engineering design problems. The paper by Giovagnoli considers the binary response models of Chaloner and Larntz (1989, Journal of Statistical Planning and Inference, 21:191–208) and develops an adaptive version of A-optimal Bayesian designs that are revised as additional data become available during the experiment. Adaptive Bayesian compound designs is also discussed as another extension. Konstantinou and Dette develop approximate Bayesian D-optimal designs for nonlinear regression models that involve covariates that cannot be observed directly. Applications to specific nonlinear models including the exponential regression model are considered and characterizations of the D-optimal saturated designs are presented. Optimal sample size selection for experiments is considered by De Santis and Gubbiotti in an adversarial setting where the decision makers have different prior opinions while having common utility functions and data. Using a Bayesian decision approach with a quadratic loss function, the authors present results for the one-parameter exponential family and investigate effect of the priors on the optimal sample size. Müller, Xu, and Thall discuss advantages and limitations of Bayesian decision theoretic approach in design of clinical trials. In doing so, the authors present a specific case study and point out the limitations such as the computational difficulties involved in solving sequential problems as well as selection of utility functions capturing different stakeholder interests. The article by Ventz, Parmigiani, and Trippa also consider limitations of the Bayesian framework in design of clinical trials and propose a strategy to alleviate these. The proposed strategy involves combining Bayesian designs with frequentist metrics such as error rates, confidence intervals, etc. that are commonly used by medical community and regulatory agencies. The authors show that this can be achieved by an inclusion of frequentist constraints into the Bayesian formulation. The constrained decision theoretic framework is illustrated via several applications and computational algorithms are presented for implementing the proposed approach. Nakamura, Seepaul, Kadane, and Reeja-Jayan discuss design of experiments in materials science. The authors consider use of a batch-sequential experiment and a regression model to determine optimal levels of input variables for synthesis of ceramic materials. It is illustrated that the optimal settings produce more reliable results than other experiments. The final article by Polson and Soyer considers Bayesian design of accelerated life tests for reliability assessment. An augmented probability simulation (APS) approach using Lindley's (1976, Annals of Statistics, 4:1–10) conjugate utility functions is proposed to compute optimal designs in an efficient manner. The proposed approach is illustrated using single and multiple point designs. Use of particle based methods for APS is also discussed as an alternative to Markov chain Monte Carlo methods.

In pediatric oncology, a Phase II trial often utilizes a safety run-in phase followed by an efficacy phase that enrolls at the dose level selected based on the safety run-in. Different from a Phase I trial, a Phase II safety run-in often assesses a very small number of dose levels. In the context of a safety run-in that assesses two or three dose levels, this article aims to compare three design methods, including the algorithm-based designs 3 + 3 and Rolling 6, and the model-assisted designs such as the Bayesian optimal interval design. Extensive simulations were conducted to evaluate and compare operating characteristics of the three design methods for a safety run-in with two or three dose levels, varying the starting dose level. The performance of algorithm-based and model-assisted designs can be influenced by selection of the starting dose level, with trials starting at a lower dose level having a higher probability of selecting a low dose or considering all doses as toxic. The impact is larger for 3 + 3 and Rolling 6 but to a lesser extent for Bayesian optimal interval design. For a safety run-in with two dose levels, using 3 + 3 or Rolling 6 and starting at the higher dose often lead to similar performance to Bayesian optimal interval design. For safety run-in with three dose levels, starting at the middle dose with 3 + 3, Rolling 6 or Bayesian optimal interval design is a good compromise between improving correct dose selection and imposing a toxic dose to less patients. Despite being sensitive to the starting dose level, the 3 + 3, Rolling 6 and Bayesian optimal interval designs overall demonstrate reasonable performance, which can be further improved with wise selection of the starting dose level. The Rolling 6 design remains the recommended design method especially if pharmacokinetics is important or required with this design having the feature of treating six patients per dose level. When designing a safety run-in, selection of a design method or selection of a starting dose should consider both the performance of the design approaches with different choices of a starting dose level and the magnitude of safety concerns with the dose levels under investigation.

We consider a dose-finding trial in phase IIB of drug development. For choosing an appropriate design for this trial the specification of two points is critical: an appropriate model for describing the dose-effect relationship, and the specification of the aims of the trial (objectives), which will be the focus in the present paper. For many situations it is essential to have a robust trial objective that has little risk of changing during the complete trial due to external information. An important and realistic objective of a dose-finding trial is to obtain precise information about key parts of the dose-effect curve. We reflect this goal in a statistical optimality criterion and derive efficient designs using optimal design theory. In particular, we determine nonadaptive Bayesian optimal designs, i.e., designs which are not changed by information obtained from an interim analysis. Compared with a traditional balanced design for this trial, it is shown that the optimal design is substantially more efficient. This implies either a gain in information, or essential savings in sample size. Further, we investigate an adaptive Bayesian optimal design that uses different optimal designs before and after an interim analysis, and we compare the adaptive with the nonadaptive Bayesian optimal design. The basic concept is illustrated using a modification of a recent AstraZeneca trial.

In this paper 3 criteria to design experiments for Bayesian estimation of the parameters of nonlinear models with respect to their parameters, when a prior distribution is available, are presented: the determinant of the Bayesian information matrix, the determinant of the pre-posterior covariance matrix, and the expected information provided by an experiment. A procedure to simplify the computation of these criteria is proposed in the case of continuous prior distributions and is compared with the criterion obtained from a linearization of the model about the mean of the prior distribution for the parameters. This procedure is applied to two models commonly encountered in the area of pharmacokinetics and pharmacodynamics: the one-compartment open model with bolus intravenous single-dose injection and the Emax model. They both involve two parameters. Additive as well as multiplicative gaussian measurement errors are considered with normal prior distributions. Various combinations of the variances of the prior distribution and of the measurement error are studied. Our attention is restricted to designs with limited numbers of measurements (1 or 2 measurements). This situation often occurs in practice when Bayesian estimation is performed. The optimal Bayesian designs that result vary with the variances of the parameter distribution and with the measurement error. The two-point optimal designs sometimes differ from the D-optimal designs for the mean of the prior distribution and may consist of replicating measurements. For the studied cases, the determinant of the Bayesian information matrix and its linearized form lead to the same optimal designs. In some cases, the pre-posterior covariance matrix can be far from its lower bound, namely, the inverse of the Bayesian information matrix, especially for the Emax model and a multiplicative measurement error. The expected information provided by the experiment and the determinant of the pre-posterior covariance matrix generally lead to the same designs except for the Emax model and the multiplicative measurement error. Results show that these criteria can be easily computed and that they could be incorporated in modules for designing experiments.

SUMMARY Monitoring networks increasingly aim to assimilate data from a large number of diverse sensors covering many sensing modalities. Bayesian optimal experimental design (OED) seeks to identify data, sensor configurations or experiments which can optimally reduce uncertainty and hence increase the performance of a monitoring network. Information theory guides OED by formulating the choice of experiment or sensor placement as an optimization problem that maximizes the expected information gain (EIG) about quantities of interest given prior knowledge and models of expected observation data. Therefore, within the context of seismo-acoustic monitoring, we can use Bayesian OED to configure sensor networks by choosing sensor locations, types and fidelity in order to improve our ability to identify and locate seismic sources. In this work, we develop the framework necessary to use Bayesian OED to optimize a sensor network’s ability to locate seismic events from arrival time data of detected seismic phases at the regional-scale. This framework requires five elements: (i) A likelihood function that describes the distribution of detection and traveltime data from the sensor network, (ii) A prior distribution that describes a priori belief about seismic events, (iii) A Bayesian solver that uses a prior and likelihood to identify the posterior distribution of seismic events given the data, (iv) An algorithm to compute EIG about seismic events over a data set of hypothetical prior events, (v) An optimizer that finds a sensor network which maximizes EIG. Once we have developed this framework, we explore many relevant questions to monitoring such as: how to trade off sensor fidelity and earth model uncertainty; how sensor types, number and locations influence uncertainty; and how prior models and constraints influence sensor placement.