Bayesian Optimal Experimental Design Research Articles

Designing optimal behavioral experiments using machine learning.

Computational models are powerful tools for understanding human cognition and behavior. They let us express our theories clearly and precisely and offer predictions that can be subtle and often counter-intuitive. However, this same richness and ability to surprise means our scientific intuitions and traditional tools are ill-suited to designing experiments to test and compare these models. To avoid these pitfalls and realize the full potential of computational modeling, we require tools to design experiments that provide clear answers about what models explain human behavior and the auxiliary assumptions those models must make. Bayesian optimal experimental design (BOED) formalizes the search for optimal experimental designs by identifying experiments that are expected to yield informative data. In this work, we provide a tutorial on leveraging recent advances in BOED and machine learning to find optimal experiments for any kind of model that we can simulate data from, and show how by-products of this procedure allow for quick and straightforward evaluation of models and their parameters against real experimental data. As a case study, we consider theories of how people balance exploration and exploitation in multi-armed bandit decision-making tasks. We validate the presented approach using simulations and a real-world experiment. As compared to experimental designs commonly used in the literature, we show that our optimal designs more efficiently determine which of a set of models best account for individual human behavior, and more efficiently characterize behavior given a preferred model. At the same time, formalizing a scientific question such that it can be adequately addressed with BOED can be challenging and we discuss several potential caveats and pitfalls that practitioners should be aware of. We provide code to replicate all analyses as well as tutorial notebooks and pointers to adapt the methodology to different experimental settings.

Bayesian experimental design and parameter estimation for ultrafast spin dynamics

Advanced experimental measurements are crucial for driving theoretical developments and unveiling novel phenomena in condensed matter and materials physics, which often suffer from the scarcity of large-scale facility resources, such as x-ray or neutron scattering centers. To address these limitations, we introduce a methodology that leverages the Bayesian optimal experimental design paradigm to efficiently uncover key quantum spin fluctuation parameters from x-ray photon fluctuation spectroscopy (XPFS) data. Our method is compatible with existing theoretical simulation pipelines and can also be used in combination with fast machine learning surrogate models in the event that real-time simulations are unfeasible. Our numerical benchmarks demonstrate the superior performance in predicting model parameters and in delivering more informative measurements within limited experimental time. Our method can be adapted to many different types of experiments beyond XPFS and spin fluctuation studies, facilitating more efficient data collection and accelerating scientific discoveries.

Bayesian optimal experimental design for constitutive model calibration

Computational simulation is increasingly relied upon for high/consequence engineering decisions, which necessitates a high confidence in the calibration of and predictions from complex material models. However, the calibration and validation of material models is often a discrete, multi-stage process that is decoupled from material characterization activities, which means the data collected does not always align with the data that is needed. To address this issue, an integrated workflow for delivering an enhanced characterization and calibration procedure—Interlaced Characterization and Calibration (ICC)—is introduced and demonstrated. This framework leverages Bayesian optimal experimental design (BOED), which creates a line of communication between model calibration needs and data collection capabilities in order to optimize the information content gathered from the experiments for model calibration.Eventually, the ICC framework will be used in quasi real-time to actively control experiments of complex specimens for the calibration of a high-fidelity material model. This work presents the critical first piece of algorithm development and a demonstration in determining the optimal load path of a cruciform specimen with simulated data. Calibration results, obtained via Bayesian inference, from the integrated ICC approach are compared to calibrations performed by choosing the load path a priori based on human intuition, as is traditionally done. The calibration results are communicated through parameter uncertainties which are propagated to the model output space (i.e. stress–strain). In these exemplar problems, data generated within the ICC framework resulted in calibrated model parameters with reduced measures of uncertainty compared to the traditional approaches.

Bayesian design of measurements for magnetorelaxometry imaging *

The aim of magnetorelaxometry imaging is to determine the distribution of magnetic nanoparticles inside a subject by measuring the relaxation of the superposition magnetic field generated by the nanoparticles after they have first been aligned using an external activation magnetic field that has subsequently been switched off. This work applies techniques of Bayesian optimal experimental design to (sequentially) selecting the positions for the activation coil in order to increase the value of data and enable more accurate reconstructions in a simplified measurement setup. Both Gaussian and total variation (TV) prior models are considered for the distribution of the nanoparticles. The former allows simultaneous offline computation of optimized designs for multiple consecutive activations, while the latter introduces adaptability into the algorithm by using previously measured data in choosing the position of the next activation. The TV prior has a desirable edge-enhancing characteristic, but with the downside that the computationally attractive Gaussian form of the posterior density is lost. To overcome this challenge, the lagged diffusivity iteration is used to provide an approximate Gaussian posterior model and allow the use of the standard Bayesian A- and D-optimality criteria for the TV prior as well. Two-dimensional numerical experiments are performed on a few sample targets, with the conclusion that the optimized activation positions lead, in general, to better reconstructions than symmetric reference setups when the target distribution or region of interest are nonsymmetric in shape.

Stability estimates for the expected utility in Bayesian optimal experimental design

We study stability properties of the expected utility function in Bayesian optimal experimental design. We provide a framework for this problem in a non-parametric setting and prove a convergence rate of the expected utility with respect to a likelihood perturbation. This rate is uniform over the design space and its sharpness in the general setting is demonstrated by proving a lower bound in a special case. To make the problem more concrete we proceed by considering non-linear Bayesian inverse problems with Gaussian likelihood and prove that the assumptions set out for the general case are satisfied and regain the stability of the expected utility with respect to perturbations to the observation map. Theoretical convergence rates are demonstrated numerically in three different examples.

Bayesian Optimal Experimental Design for Race Tracking in Resin Transfer Moulding

A Bayesian inference formulation is applied to the Resin Transfer Moulding process to estimate bulk permeability and race-tracking effects using measured values of pressure at discrete sensor locations throughout a preform. The algorithm quantifies uncertainty in both the permeability and race-tracking effects, which decreases when more sensors are used or the preform geometry is less complex. We show that this approach becomes less reliable with a smaller resin exit vent. Numerical experiments show that the formulation can accurately predict race-tracking effects with few measurements. A Bayesian A-optimality formulation is used to develop a method for producing optimal sensor locations that reduce the uncertainty in the permeability and race-tracking estimates the most. This method is applied to two numerical examples which show that optimal designs reduce uncertainty by up to an order of magnitude compared to a random design.

Reduction of testing effort for fatigue tests: Application of Bayesian optimal experimental design

AbstractS‐N curve parameter determination is a time‐ and cost‐intensive procedure. A standardized method for simultaneously determining all S‐N curve parameters with minimum testing effort is still missing. The Bayesian optimal experimental design (BOED) approach can reduce testing effort and accelerates uncertainty reduction during fatigue testing for S‐N curve parameters. The concept is applicable to all S‐N curve models and is exemplary illustrated for a bilinear S‐N curve model. We demonstrate the fatigue testing workflow for the bilinear S‐N curve in detail while discussing steps and challenges when generalizing to other S‐N curve models. Applying the BOED to the bilinear S‐N curve models, minor errors and uncertainties for all S‐N curve parameters are obtained after only 10 experiments for data scatter values below 1.1. For such, the relative error in fatigue limit estimation was less than 1% after five tests. When S‐N data scatter higher than 1.2 is concerned, 17 tests were required for robust analysis. The BOED methodology should be applied to other S‐N curve models in the future. The high computational effort and the approximation of the posterior distribution with a normal distribution are the limitations of the presented BOED approach.

Bayesian Optimal Experimental Design for Inferring Causal Structure

Inferring the causal structure of a system typically requires interventional data, rather than just observational data. Since interventional experiments can be costly, it is preferable to select interventions that yield the maximum amount of information about a system. We propose a novel Bayesian method for optimal experimental design by sequentially selecting interventions that minimize the expected posterior entropy as rapidly as possible. A key feature is that the method can be implemented by computing simple summaries of the current posterior, avoiding the computationally burdensome task of repeatedly performing posterior inference on hypothetical future datasets drawn from the posterior predictive. After deriving the method in a general setting, we apply it to the problem of inferring causal networks. We present a series of simulation studies, in which we find that the proposed method performs favorably compared to existing alternative methods. Finally, we apply the method to real data from two gene regulatory networks.

Bayesian sequential optimal experimental design for nonlinear models using policy gradient reinforcement learning

We present a mathematical framework and computational methods for optimally designing a finite sequence of experiments. This sequential optimal experimental design (sOED) problem is formulated as a finite-horizon partially observable Markov decision process (POMDP) under a Bayesian setting and with information-theoretic utilities. The formulation is general and may accommodate continuous random variables, non-Gaussian posteriors, and nonlinear forward models. The sOED design policy incorporates elements of feedback and lookahead simultaneously, and we show it to generalize the commonly-used batch and greedy design strategies. We solve for the sOED policy using the policy gradient (PG) method from reinforcement learning, and provide a derivation for the PG expression in the sOED context. Adopting an actor–critic approach, the policy and value functions are parameterized using deep neural networks and improved via PG estimates produced from simulated episodes of designs and observations. The new PG-sOED algorithm is first validated on a linear-Gaussian benchmark, and then compared against other design baselines on a sensor movement problem for contaminant source inversion in a convection–diffusion field. We provide explanation for the policy behaviors using knowledge of the underlying physical process.

Design Amortization for Bayesian Optimal Experimental Design

Bayesian optimal experimental design is a sub-field of statistics focused on developing methods to make efficient use of experimental resources. Any potential design is evaluated in terms of a utility function, such as the (theoretically well-justified) expected information gain (EIG); unfortunately however, under most circumstances the EIG is intractable to evaluate. In this work we build off of successful variational approaches, which optimize a parameterized variational model with respect to bounds on the EIG. Past work focused on learning a new variational model from scratch for each new design considered. Here we present a novel neural architecture that allows experimenters to optimize a single variational model that can estimate the EIG for potentially infinitely many designs. To further improve computational efficiency, we also propose to train the variational model on a significantly cheaper-to-evaluate lower bound, and show empirically that the resulting model provides an excellent guide for more accurate, but expensive to evaluate bounds on the EIG. We demonstrate the effectiveness of our technique on generalized linear models, a class of statistical models that is widely used in the analysis of controlled experiments. Experiments show that our method is able to greatly improve accuracy over existing approximation strategies, and achieve these results with far better sample efficiency.

A polynomial chaos efficient global optimization approach for Bayesian optimal experimental design

This paper proposes a global optimization framework to address the high computational cost and non convexity of Optimal Experimental Design (OED) problems. To reduce the computational burden and the presence of noise in the evaluation of the Shannon expected information gain (SEIG), this framework proposes the coupling of Laplace approximation and polynomial chaos expansions (PCE). The advantage of this procedure is that PCE allows large samples to be employed for the SEIG estimation, practically vanishing the noisy introduced by the sampling procedure. Consequently, the resulting optimization problem may be treated as deterministic. Then, an optimization approach based on Kriging surrogates is employed as the optimization engine to search for the global solution with limited computational budget. Four numerical examples are investigated and their results are compared to state-of-the-art stochastic gradient descent algorithms. The proposed approach obtained better results than the stochastic gradient algorithms in all situations, indicating its efficiency and robustness in the solution of OED problems.

Large-Scale Bayesian Optimal Experimental Design with Derivative-Informed Projected Neural Network

We address the solution of large-scale Bayesian optimal experimental design (OED) problems governed by partial differential equations (PDEs) with infinite-dimensional parameter fields. The OED problem seeks to find sensor locations that maximize the expected information gain (EIG) in the solution of the underlying Bayesian inverse problem. Computation of the EIG is usually prohibitive for PDE-based OED problems. To make the evaluation of the EIG tractable, we approximate the (PDE-based) parameter-to-observable map with a derivative-informed projected neural network (DIPNet) surrogate, which exploits the geometry, smoothness, and intrinsic low-dimensionality of the map using a small and dimension-independent number of PDE solves. The surrogate is then deployed within a greedy algorithm-based solution of the OED problem such that no further PDE solves are required. We analyze the EIG approximation error in terms of the generalization error of the DIPNet and show they are of the same order. Finally, the efficiency and accuracy of the method are demonstrated via numerical experiments on OED problems governed by inverse scattering and inverse reactive transport with up to 16,641 uncertain parameters and 100 experimental design variables, where we observe up to three orders of magnitude speedup relative to a reference double loop Monte Carlo method.

A Fast and Scalable Computational Framework for Large-Scale High-Dimensional Bayesian Optimal Experimental Design

We develop a fast and scalable computational framework to solve Bayesian optimal experimental design problems governed by partial differential equations (PDEs) with application to optimal sensor placement by maximizing expected information gain (EIG). Such problems are particularly challenging due to the curse of dimensionality for high-dimensional parameters and the expensive solution of large-scale PDEs. To address these challenges, we exploit two fundamental properties: (1) the low-rank structure of the Jacobian of the parameter-to-observable map, to extract the intrinsically low-dimensional data-informed subspace, and (2) a series of approximations of the EIG that reduce the number of PDE solves while retaining high correlation with the true EIG. Based on these properties, we propose an efficient offline-online decomposition for the optimization problem. The offline stage dominates the cost and entails precomputing all components that require PDE solves. The online stage optimizes sensor placement and does not require any PDE solves. For the online stage, we propose a new greedy algorithm that first places an initial set of sensors using leverage scores and then swaps the selected sensors with other candidates until certain convergence criteria are met, which we call a swapping greedy algorithm. We demonstrate the efficiency and scalability of the proposed method by both linear and nonlinear inverse problems. In particular, we show that the number of required PDE solves is small, independent of the parameter dimension, and only weakly dependent on the data dimension for both problems.

An Offline-Online Decomposition Method for Efficient Linear Bayesian Goal-Oriented Optimal Experimental Design: Application to Optimal Sensor Placement

Bayesian optimal experimental design (OED) plays an important role in minimizing model uncertainty with limited experimental data in a Bayesian framework. In many applications, rather than minimizing the uncertainty in the inference of model parameters, one seeks to minimize the uncertainty of a model-dependent quantity of interest (QoI). This is known as goal-oriented OED (GOOED). Here, we consider GOOED for linear Bayesian inverse problems governed by large-scale models represented by partial differential equations (PDE) that are computationally expensive to solve. In particular, we consider optimal sensor placement by maximizing an expected information gain (EIG) for the QoI. We develop an efficient method to solve such problems by deriving a new formulation of the goal-oriented EIG. Based on this formulation we propose an offline-online decomposition scheme that achieves significant computational reduction by computing all of the PDE-dependent quantities in an offline stage just once, and optimizing the sensor locations in an online stage without solving any PDEs. Moreover, in the offline stage we need only to compute low-rank approximations of two Hessian-related operators. The computational cost of these low-rank approximations, measured by the number of PDE solves, does not depend on the parameter or data dimensions for a large class of elliptic, parabolic, and sufficiently dissipative hyperbolic inverse problem that exhibit dimension-independent rapid spectra decay. We carry out detailed error analysis for the approximate goal-oriented EIG due to the low-rank approximations of the two operators. Furthermore, in the online stage we extend a swapping greedy method to optimize the sensor locations developed in our recent work that is demonstrated to be more efficient than a standard greedy method. We conduct a numerical experiment for a contaminant transport inverse problem with an infinite-dimensional parameter field to demonstrate the efficiency, accuracy, and both data- and parameter-dimension independence of the proposed algorithm.

Optimal Array Geometric Structures for Direct Position Determination Systems

Millimeter-wave (mmWave) fifth-generation (5G) and beyond 5G localization enables the provisioning of extremely accurate positioning information, a feature that has attracted substantial research efforts. In this paper, we contribute to this effort by exploring optimal array geometric structures of direct position determination (DPD) systems, inspired by sensor placement problems that predominantly focus on two-step localization and have not yet been extended to DPD. Specifically, we research an optimal array placement and orientation strategy for two-dimensional (2-D) DPD systems that use sensors equipped with uniform linear arrays (ULA) to localize an agent. The A-optimality criterion in Bayesian optimal (experimental) design theory is invoked to formulate this problem. We use an optimization subproblem that optimizes array orientations when array locations are arbitrary but fixed to tackle this high-dimensional optimization problem. Then the optimization problem is converted into a typical optimal angular separation problem in two-step localization. Experimental results show that judiciously designed array geometric structures can lead to significant performance improvements.

Comparing Well and Geophysical Data for Temperature Monitoring Within a Bayesian Experimental Design Framework

AbstractTemperature logs are an important tool in the geothermal industry. Temperature measurements from boreholes are used for exploration, system design, and monitoring. The number of observations, however, is not always sufficient to fully determine the temperature field or explore the entire parameter space of interest. Drilling in the best locations is still difficult and expensive. It is therefore critical to optimize the number and location of boreholes. Due to its higher spatial resolution and lower cost, four‐dimensional (4D) temperature field monitoring via time‐lapse Electrical Resistivity Tomography has been investigated as a potential alternative. We use Bayesian Evidential Learning (BEL), a Monte Carlo‐based training approach, to optimize the design of a 4D temperature field monitoring experiment. We demonstrate how BEL can take into account various data source combinations (temperature logs combined with geophysical data) in the Bayesian optimal experimental design (BOED). To determine the optimal data source combination, we use the Root Mean Squared Error of the predicted target in the low dimensional latent space where BEL is solving the prediction problem. The parameter estimates are accurate enough to use in BOED. Furthermore, the method is not limited to monitoring temperature fields and can be applied to other similar experimental design problems. The method is computationally efficient and requires little training data. For the considered optimal design problem, a training set of only 200 samples and a test set of 50 samples is sufficient.

Small-noise approximation for Bayesian optimal experimental design with nuisance uncertainty

Calculating the expected information gain in optimal Bayesian experimental design typically relies on nested Monte Carlo sampling. When the model also contains nuisance parameters, which are parameters that contribute to the overall uncertainty of the system but are of no interest in the Bayesian design framework, this introduces a second inner loop. We propose and derive a small-noise approximation for this additional inner loop. The computational cost of our method can be further reduced by applying a Laplace approximation to the remaining inner loop. Thus, we present two methods, the small-noise double-loop Monte Carlo and small-noise Monte Carlo Laplace methods. Moreover, we demonstrate that the total complexity of these two approaches remains comparable to the case without nuisance uncertainty. To assess the efficiency of these methods, we present three examples, and the last example includes the partial differential equation for the electrical impedance tomography experiment for composite laminate materials.

Reinforcement Learning-Based Sequential Batch-Sampling for Bayesian Optimal Experimental Design

Abstract Engineering problems that are modeled using sophisticated mathematical methods or are characterized by expensive-to-conduct tests or experiments are encumbered with limited budget or finite computational resources. Moreover, practical scenarios in the industry, impose restrictions, based on logistics and preference, on the manner in which the experiments can be conducted. For example, material supply may enable only a handful of experiments in a single-shot or in the case of computational models one may face significant wait-time based on shared computational resources. In such scenarios, one usually resorts to performing experiments in a manner that allows for maximizing one’s state-of-knowledge while satisfying the above-mentioned practical constraints. Sequential design of experiments (SDOE) is a popular suite of methods that have yielded promising results in recent years across different engineering and practical problems. A common strategy that leverages Bayesian formalism is the Bayesian SDOE, which usually works best in the one-step-ahead or myopic scenario of selecting a single experiment at each step of a sequence of experiments. In this work, we aim to extend the SDOE strategy, to query the experiment or computer code at a batch of inputs. To this end, we leverage deep reinforcement learning (RL)-based policy gradient methods, to propose batches of queries that are selected taking into account the entire budget in hand. The algorithm retains the sequential nature, inherent in the SDOE while incorporating elements of reward based on task from the domain of deep RL. A unique capability of the proposed methodology is its ability to be applied to multiple tasks, for example, optimization of a function, once its trained. We demonstrate the performance of the proposed algorithm on a synthetic problem and a challenging high-dimensional engineering problem.

Stochastic Learning Approach for Binary Optimization: Application to Bayesian Optimal Design of Experiments

We present a novel stochastic approach to binary optimization for optimal experimental design (OED) for Bayesian inverse problems governed by mathematical models such as partial differential equations. The OED utility function, namely, the regularized optimality criterion, is cast into a stochastic objective function in the form of an expectation over a multivariate Bernoulli distribution. The probabilistic objective is then solved by using a stochastic optimization routine to find an optimal observational policy. The proposed approach is analyzed from an optimization perspective and also from a machine learning perspective with correspondence to policy gradient reinforcement learning. The approach is demonstrated numerically by using an idealized two-dimensional Bayesian linear inverse problem, and validated by extensive numerical experiments carried out for sensor placement in a parameter identification setup.

Active learning of continuous-time Bayesian networks through interventions* *This article is an updated version of: Linzner D and Koeppl H 2021 Active learning of continuous-time Bayesian networks through interventions Proc. 38th Int. Conf. on Machine Learning vol 139 ed M Meila and T Zhang pp 6692–701.

We consider the problem of learning structures and parameters of continuous-time Bayesian networks (CTBNs) from time-course data under minimal experimental resources. In practice, the cost of generating experimental data poses a bottleneck, especially in the natural and social sciences. A popular approach to overcome this is Bayesian optimal experimental design (BOED). However, BOED becomes infeasible in high-dimensional settings, as it involves integration over all possible experimental outcomes. We propose a novel criterion for experimental design based on a variational approximation of the expected information gain. We show that for CTBNs, a semi-analytical expression for this criterion can be calculated for structure and parameter learning. By doing so, we can replace sampling over experimental outcomes by solving the CTBNs master-equation, for which scalable approximations exist. This alleviates the computational burden of integrating over possible experimental outcomes in high-dimensions. We employ this framework in order to recommend interventional sequences. In this context, we extend the CTBN model to conditional CTBNs in order to incorporate interventions. We demonstrate the performance of our criterion on synthetic and real-world data.