Bayesian Emulation Research Articles

Partial Hepatectomy and Ablation for Survival of Early-Stage Hepatocellular Carcinoma Patients: A Bayesian Emulation Analysis.

Partial hepatectomy and ablation therapy are two widely used surgical procedures for localized early-stage hepatocellular carcinoma (HCC) patients. This article aimed to evaluate their relative effectiveness in terms of overall survival. An emulation analysis approach was first developed based on the Bayesian technique. We estimated propensity scores via Bayesian logistic regression and adopted a weighted Bayesian Weibull accelerated failure time (AFT) model incorporating prior information contained in the published literature. With the Surveillance, Epidemiology, and End Results (SEER)-Medicare data, an emulated target trial with rigorously defined inclusion/exclusion criteria and treatment regimens for early-stage HCC patients over 66 years old was developed. For the main cohort with tumor size less than or equal to 5 cm, a total of 1146 patients were enrolled in the emulated trial, with 301 and 845 in the partial hepatectomy and ablation arms, respectively. The analysis suggested ablation to be significantly associated with inferior overall survival (hazard ratio [HR] = 1.35; 95% credible interval [CrI]: 1.14, 1.60). For the subgroup with tumor size less than or equal to 3 cm, there was no significant difference in overall survival between the two arms (HR = 1.15; 95% CrI: 0.88, 1.52). Overall, the comparative treatment effect of ablation and partial hepatectomy on survival remains inconclusive. This finding may provide further insight into HCC clinical treatment.

Bayesian Emulation for Computer Models with Multiple Partial Discontinuities

Bayesian Emulation for Computer Models with Multiple Partial Discontinuities

A Bayesian Computer Model Analysis of Robust Bayesian Analyses

We harness the power of Bayesian emulation techniques, designed to aid the analysis of complex computer models, to examine the structure of complex Bayesian analyses themselves. These techniques facilitate robust Bayesian analyses and/or sensitivity analyses of complex problems, and hence allow global exploration of the impacts of choices made in both the likelihood and prior specification. We show how previously intractable problems in robustness studies can be overcome using emulation techniques, and how these methods allow other scientists to quickly extract approximations to posterior results corresponding to their own particular subjective specification. The utility and flexibility of our method is demonstrated on a reanalysis of a real application where Bayesian methods were employed to capture beliefs about river flow. We discuss the obvious extensions and directions of future research that such an approach opens up.

Flood hazard model calibration using multiresolution model output

AbstractRiverine floods pose a considerable risk to many communities. Improving flood hazard projections has the potential to inform the design and implementation of flood risk management strategies. Current flood hazard projections are uncertain, especially due to uncertain model parameters. Calibration methods use observations to quantify model parameter uncertainty. With limited computational resources, researchers typically calibrate models using either relatively few expensive model runs at high spatial resolutions or many cheaper runs at lower spatial resolutions. This leads to an open question: is it possible to effectively combine information from the high and low resolution model runs? We propose a Bayesian emulation–calibration approach that assimilates model outputs and observations at multiple resolutions. As a case study for a riverine community in Pennsylvania, we demonstrate our approach using the LISFLOOD‐FP flood hazard model. The multiresolution approach results in improved parameter inference over the single resolution approach in multiple scenarios. Results vary based on the parameter values and the number of available models runs. Our method is general and can be used to calibrate other high dimensional computer models to improve projections.

Bayesian emulation and history matching of JUNE.

We analyze JUNE: a detailed model of COVID-19 transmission with high spatial and demographic resolution, developed as part of the RAMP initiative. JUNE requires substantial computational resources to evaluate, making model calibration and general uncertainty analysis extremely challenging. We describe and employ the uncertainty quantification approaches of Bayes linear emulation and history matching to mimic JUNE and to perform a global parameter search, hence identifying regions of parameter space that produce acceptable matches to observed data, and demonstrating the capability of such methods.This article is part of the theme issue ‘Technical challenges of modelling real-life epidemics and examples of overcoming these’.

Residual Gaussian process: A tractable nonparametric Bayesian emulator for multi-fidelity simulations

Challenges in multi-fidelity modelling relate to accuracy, uncertainty estimation and high-dimensionality. A novel additive structure is introduced in which the highest fidelity solution is written as a sum of the lowest fidelity solution and residuals between the solutions at successive fidelity levels, with Gaussian process priors placed over the low fidelity solution and each of the residuals. The resulting model is equipped with a closed-form solution for the predictive posterior, making it applicable to advanced, high-dimensional tasks that require uncertainty estimation. Its advantages are demonstrated on univariate benchmarks and on three challenging multivariate problems. It is shown how active learning can be used to enhance the model, especially with a limited computational budget. Furthermore, error bounds are derived for the mean prediction in the univariate case.

Probabilistic forecast of wind speed based on Bayesian emulator using monitoring data

Wind speed forecasting can serve a wide spectrum of purposes, including scheduling of a power system and dynamic control of structures. A lot of models are widely used to forecast wind speed, consisting of deterministic models (e.g., physical models, statistical models, and artificial intelligence models) and probabilistic models (e.g., Bayesian model). The wind speed has the characteristics of random, nonlinear, and uncertainty, which highlights the importance of using Bayesian model to predict the wind speed. In this study, a Bayesian emulator with Gaussian process prior is adopted for probabilistic forecast of wind speed. The present Bayesian emulator approach not only maintains the data-driven property which guarantees its high flexibility in modeling the complexity of the target system but also allows for the efficient, probabilistic evaluation of the wind speed in terms of the predictive mean and variance. Nevertheless, the modeling performance of the Bayesian emulator directly depends on the selected covariance function. The influence of different types of covariance functions, which include squared-exponential (SE) covariance function, Matern (MA) covariance function, periodic (PE) covariance function, and composite covariance function, on forecasting performance of the wind speed is studied. One-month wind monitoring data collected by structural health monitoring (SHM) system installed on Jiubao bridge are employed to demonstrate the effectiveness of Bayesian emulator for forecasting the wind speed.

Understanding hormonal crosstalk in Arabidopsis root development via emulation and history matching.

A major challenge in plant developmental biology is to understand how plant growth is coordinated by interacting hormones and genes. To meet this challenge, it is important to not only use experimental data, but also formulate a mathematical model. For the mathematical model to best describe the true biological system, it is necessary to understand the parameter space of the model, along with the links between the model, the parameter space and experimental observations. We develop sequential history matching methodology, using Bayesian emulation, to gain substantial insight into biological model parameter spaces. This is achieved by finding sets of acceptable parameters in accordance with successive sets of physical observations. These methods are then applied to a complex hormonal crosstalk model for Arabidopsis root growth. In this application, we demonstrate how an initial set of 22 observed trends reduce the volume of the set of acceptable inputs to a proportion of 6.1×10-7 of the original space. Additional sets of biologically relevant experimental data, each of size 5, reduce the size of this space by a further three and two orders of magnitude respectively. Hence, we provide insight into the constraints placed upon the model structure by, and the biological consequences of, measuring subsets of observations.

Efficient Selection of Reservoir Model Outputs within an Emulation-Based Bayesian History-Matching Uncertainty Analysis

Summary When performing classic uncertainty reduction according to dynamic data, a large number of reservoir simulations need to be evaluated at high computational cost. As an alternative, we construct Bayesian emulators that mimic the dominant behavior of the reservoir simulator, and which are several orders of magnitude faster to evaluate. We combine these emulators within an iterative procedure that involves substantial but appropriate dimensional reduction of the output space (which represents the reservoir physical behavior, such as production data), enabling a more effective and efficient uncertainty reduction on the input space (representing uncertain reservoir parameters) than traditional methods, and with a more comprehensive understanding of the associated uncertainties. This study uses the emulation-based Bayesian history-matching (BHM) uncertainty analysis for the uncertainty reduction of complex models, which is designed to address problems with a high number of both input and output parameters. We detail how to efficiently choose sets of outputs that are suitable for emulation and that are highly informative to reduce the input-parameter space and investigate different classes of outputs and objective functions. We use output emulators and implausibility analysis iteratively to perform uncertainty reduction in the input-parameter space, and we discuss the strengths and weaknesses of certain popular classes of objective functions in this context. We demonstrate our approach through an application to a benchmark synthetic model (built using public data from a Brazilian offshore field) in an early stage of development using 4 years of historical data and four producers. This study investigates traditional simulation outputs (e.g., production data) and also novel classes of outputs, such as misfit indices and summaries of outputs. We show that despite there being a large number (2,136) of possible outputs, only very few (16) were sufficient to represent the available information; these informative outputs were used using fast and efficient emulators at each iteration (or wave) of the history match to perform the uncertainty-reduction procedure successfully. Using this small set of outputs, we were able to substantially reduce the input space by removing 99.8% of the original volume. We found that a small set of physically meaningful individual production outputs were the most informative at early waves, which once emulated, resulted in the highest uncertainty reduction in the input-parameter space, while more complex but popular objective functions that combine several outputs were only modestly useful at later waves. The latter point is because objective functions such as misfit indices have complex surfaces that can lead to low-quality emulators and hence result in noninformative outputs. We present an iterative emulator-based Bayesian uncertainty-reduction process in which all possible input-parameter configurations that lead to statistically acceptable matches between the simulated and observed data are identified. This methodology presents four central characteristics: incorporation of a powerful dimension reduction on the output space, resulting in significantly increased efficiency; effective reduction of the input space; computational efficiency, and provision of a better understanding of the complex geometry of the input and output spaces.

On the Construction of Uncertain Time Series Surrogates Using Polynomial Chaos and Gaussian Processes

The analysis of time series is a fundamental task in many flow simulations such as oceanic and atmospheric flows. A major challenge is the design of a faithful and accurate time-dependent surrogate built with a tractable sample set and a manageable number of degrees of freedom. Several techniques are implemented to handle the time-dependent aspect of the quantity of interest including uncoupled approaches, low-rank approximations, auto-regressive models and global Bayesian emulators. These approaches rely on two popular methods for uncertainty quantification: polynomial chaos and Gaussian process regression. The different techniques are tested and compared on the uncertain evolution of the sea surface height forecast at two locations exhibiting contrasting levels of variance. Two ensemble sizes are considered as well as two versions of polynomial chaos (ordinary least squares or ridge regression) and Gaussian processes (squared exponential or Matern covariance function) in order to assess their impact on the results. The conclusions focus on the advantages and the drawbacks, in terms of accuracy, flexibility and computational costs of the different techniques.

Bayesian Emulation for Multi-Step Optimization in Decision Problems

We discuss the Bayesian emulation approach to computational solution of multi-step portfolio studies in financial time series. "Bayesian emulation for decisions" involves mapping the technical structure of a decision analysis problem to that of Bayesian inference in a purely synthetic "emulating" statistical model. This provides access to standard posterior analytic, simulation and optimization methods that yield indirect solutions of the decision problem. We develop this in time series portfolio analysis using classes of economically and psychologically relevant multi-step ahead portfolio utility functions. Studies with multivariate currency, commodity and stock index time series illustrate the approach and show some of the practical utility and benefits of the Bayesian emulation methodology.

Bayesian emulator optimisation for cosmology: application to the Lyman-alpha forest

The Lyman-alpha forest provides strong constraints on both cosmological parameters and intergalactic medium astrophysics, which are forecast to improve further with the next generation of surveys including eBOSS and DESI. As is generic in cosmological inference, extracting this information requires a likelihood to be computed throughout a high-dimensional parameter space. Evaluating the likelihood requires a robust and accurate mapping between the parameters and observables, in this case the 1D flux power spectrum. Cosmological simulations enable such a mapping, but due to computational time constraints can only be evaluated at a handful of sample points; “emulators” are designed to interpolate between these. The problem then reduces to placing the sample points such that an accurate mapping is obtained while minimising the number of expensive simulations required. To address this, we introduce an emulation procedure that employs Bayesian optimisation of the training set for a Gaussian process interpolation scheme. Starting with a Latin hypercube sampling (other schemes with good space-filling properties can be used), we iteratively augment the training set with extra simulations at new parameter positions which balance the need to reduce interpolation error while focussing on regions of high likelihood. We show that smaller emulator error from the Bayesian optimisation propagates to smaller widths on the posterior distribution. Even with fewer simulations than a Latin hypercube, Bayesian optimisation shrinks the 95% credible volume by 90% and, e.g., the 1σ error on the amplitude of small-scale primordial fluctuations by 38%. This is the first demonstration of Bayesian optimisation applied to large-scale structure emulation, and we anticipate the technique will generalise to many other probes such as galaxy clustering, weak lensing and 21cm.

Bayesian emulation and calibration of an individual-based model of microbial communities

Individual-based (IB) modelling has been widely used for studying the emergence of complex interactions of bacterial biofilms and their environment. We describe the emulation and calibration of an expensive dynamic simulator of an IB model of microbial communities. We used a combination of multivariate dynamic linear models (DLM) and a Gaussian process to estimate the model parameters of our dynamic emulators. The emulators incorporate a smoothly varying and nonstationary trend that is modelled as a deterministic function of explanatory variables while the Gaussian process (GP) is allowed to capture the remaining intrinsic local variations. We applied this emulation strategy for parameter calibration of a newly developed model for simulation of microbial communities against the iDynoMiCS model. The percentage of variance explained for the four outputs biomass concentration, the total number of particles, biofilm average height and surface roughness range between 84—92% and 97–99% for univariate and multivariate emulators respectively. The simulation-based sensitivity analysis identified carbon substrate, oxygen concentration and maximum specific growth rate for heterotrophic bacteria as the most critical variables for predictions. The calibration results also indicated a general reduction of uncertainty levels in most of the parameters. The study has helped us identify the tradeoff in using different types of models for microbial simulation. The approach illustrated here provides a tractable and computationally efficient technique for calibrating the parameters of an expensive computer model.

Scalable Bayesian Modeling, Monitoring, and Analysis of Dynamic Network Flow Data

ABSTRACTTraffic flow count data in networks arise in many applications, such as automobile or aviation transportation, certain directed social network contexts, and Internet studies. Using an example of Internet browser traffic flow through site-segments of an international news website, we present Bayesian analyses of two linked classes of models which, in tandem, allow fast, scalable, and interpretable Bayesian inference. We first develop flexible state-space models for streaming count data, able to adaptively characterize and quantify network dynamics efficiently in real-time. We then use these models as emulators of more structured, time-varying gravity models that allow formal dissection of network dynamics. This yields interpretable inferences on traffic flow characteristics, and on dynamics in interactions among network nodes. Bayesian monitoring theory defines a strategy for sequential model assessment and adaptation in cases when network flow data deviate from model-based predictions. Exploratory and sequential monitoring analyses of evolving traffic on a network of web site-segments in e-commerce demonstrate the utility of this coupled Bayesian emulation approach to analysis of streaming network count data. Supplementary materials for this article are available online.

Bayesian uncertainty analysis for complex systems biology models: emulation, global parameter searches and evaluation of gene functions

BackgroundMany mathematical models have now been employed across every area of systems biology. These models increasingly involve large numbers of unknown parameters, have complex structure which can result in substantial evaluation time relative to the needs of the analysis, and need to be compared to observed data of various forms. The correct analysis of such models usually requires a global parameter search, over a high dimensional parameter space, that incorporates and respects the most important sources of uncertainty. This can be an extremely difficult task, but it is essential for any meaningful inference or prediction to be made about any biological system. It hence represents a fundamental challenge for the whole of systems biology.MethodsBayesian statistical methodology for the uncertainty analysis of complex models is introduced, which is designed to address the high dimensional global parameter search problem. Bayesian emulators that mimic the systems biology model but which are extremely fast to evaluate are embeded within an iterative history match: an efficient method to search high dimensional spaces within a more formal statistical setting, while incorporating major sources of uncertainty.ResultsThe approach is demonstrated via application to a model of hormonal crosstalk in Arabidopsis root development, which has 32 rate parameters, for which we identify the sets of rate parameter values that lead to acceptable matches between model output and observed trend data. The multiple insights into the model’s structure that this analysis provides are discussed. The methodology is applied to a second related model, and the biological consequences of the resulting comparison, including the evaluation of gene functions, are described.ConclusionsBayesian uncertainty analysis for complex models using both emulators and history matching is shown to be a powerful technique that can greatly aid the study of a large class of systems biology models. It both provides insight into model behaviour and identifies the sets of rate parameters of interest.

Quantifying uncertainty in wholesale electricity price projections using Bayesian emulation of a generation investment model

Abstract Policy-makers need to be confident that decisions based on the outputs of energy system models will be robust in the real-world. To make robust decisions it is critical that the consequences of uncertainty in model outputs are assessed. This paper presents statistical methodology for quantifying uncertainty associated with the output of a computer model of the long-term GB electricity supply. The output of the computer model studied is the projection of wholesale electricity prices from 2016 to 2030. The effect on wholesale prices of both uncertainty in input parameters and structural discrepancy is modelled. A probability distribution is used to model uncertainty over four inputs of the model: gas price, demand, EU ETS price and future offshore deployment. Estimates of the structural discrepancy introduced by the use of smoothed gas price projections and assuming that coal prices out to 2030 are known are obtained from experimentation with the computer model. A statistical model, known as an emulator, is fitted to a set of computer model evaluations and used to model uncertainty in the output of the computer model at inputs that have not been tested. The emulator is combined with the probability distribution over the inputs and the estimate of structural discrepancy to make an assessment of the overall uncertainty in the wholesale electricity price projections. A sensitivity analysis is also performed to investigate the effect of each of the four inputs on the trajectory of wholesale electricity prices.

Flexible Correlation Structure for Accurate Prediction and Uncertainty Quantification in Bayesian Gaussian Process Emulation of a Computer Model

Gaussian processes are widely used in the analysis of data from a computer model. Ideally, the analysis will yield accurate predictions with correct coverage probabilities of credible intervals. In this paper, we first review several existing Bayesian implementations in the literature. We show that Bayesian approaches with squared-exponential correlation structure do not always quantify well the uncertainty in prediction. Thus, we propose new Bayesian approaches with power-exponential or Matern correlation structure, which have more flexibility. Through application examples and a simulation study, we show that the proposed Bayesian methods not only have superior prediction accuracy but are closer to having the correct coverage probability.

Constraints on galaxy formation models from the galaxy stellar mass function and its evolution

We explore the parameter space of the semi-analytic galaxy formation model GALFORM, studying the constraints imposed by measurements of the galaxy stellar mass function (GSMF) and its evolution. We use the Bayesian Emulator method to quickly eliminate vast implausible volumes of the parameter space and zoom in on the most interesting regions, allowing us to identify a set of models that match the observational data within model uncertainties. We find that the GSMF strongly constrains parameters related to quiescent star formation in discs, stellar and AGN feedback and threshold for disc instabilities, but weakly restricts other parameters. Constraining the model using local data alone does not usually select models that match the evolution of the GSMF well. Nevertheless, we show that a small subset of models provides acceptable match to GSMF data out to redshift 1.5. We explore the physical significance of the parameters of these models, in particular exploring whether the model provides a better description if the mass loading of the galactic winds generated by starbursts ($\beta_{0,\text{burst}}$) and quiescent disks ($\beta_{0,\text{disc}}$) is different. Performing a principal component analysis of the plausible volume of the parameter space, we write a set of relations between parameters obeyed by plausible models with respect to GSMF evolution. We find that while $\beta_{0,\text{disc}}$ is strongly constrained by GSMF evolution data, constraints on $\beta_{0,\text{burst}}$ are weak. Although it is possible to find plausible models for which $\beta_{0,\text{burst}} = \beta_{0,\text{disc}}$, most plausible models have $\beta_{0,\text{burst}}>\beta_{0,\text{disc}}$, implying - for these - larger SN feedback efficiency at higher redshifts.

Multivariate emulation of computer simulators: model selection and diagnostics with application to a humanitarian relief model.

SummaryWe present a common framework for Bayesian emulation methodologies for multivariate output simulators, or computer models, that employ either parametric linear models or non‐parametric Gaussian processes. Novel diagnostics suitable for multivariate covariance separable emulators are developed and techniques to improve the adequacy of an emulator are discussed and implemented. A variety of emulators are compared for a humanitarian relief simulator, modelling aid missions to Sicily after a volcanic eruption and earthquake, and a sensitivity analysis is conducted to determine the sensitivity of the simulator output to changes in the input variables. The results from parametric and non‐parametric emulators are compared in terms of prediction accuracy, uncertainty quantification and scientific interpretability.

TRANSITIONAL ANNEALED ADAPTIVE SLICE SAMPLING FOR GAUSSIAN PROCESS HYPER-PARAMETER ESTIMATION

Surrogate models have become ubiquitous in science and engineering for their capability of emulating expensive computer codes, necessary to model and investigate complex phenomena. Bayesian emulators based on Gaussian processes adequately quantify the uncertainty that results from the cost of the original simulator, and thus the inability to evaluate it on the whole input space. However, it is common in the literature that only a partial Bayesian analysis is carried out, whereby the underlying hyper-parameters are estimated via gradient-free optimisation or genetic algorithms, to name a few methods. On the other hand, maximum a posteriori (MAP) estimation could discard important regions of the hyper-parameter space. In this paper, we carry out a more complete Bayesian inference, that combines Slice Sampling with some recently developed Sequential Monte Carlo samplers. The resulting algorithm improves the mixing in the sampling through delayed-rejection, the inclusion of an annealing scheme akin to Asymptotically Independent Markov Sampling and parallelisation via Transitional Markov Chain Monte Carlo. Examples related to the estimation of Gaussian process hyper-parameters are presented. For the purpose of reproducibility, further development, and use in other applications, the code to generate the examples in this paper is freely available for download at http://github.com/agarbuno/ta2s2_codes