Computer Model Output Research Articles

Varying Coefficient Models and Design Choice for Bayes Linear Emulation of Complex Computer Models with Limited Model Evaluations

Computer models are widely used to help make decisions about real-world systems. As computer models of large and complex systems can have long run-times and high-dimensional input spaces, it is often necessary to use emulation to assess uncertainties in computer model output. This paper presents methodology for emulation of complex computer models motivated by a real-world example in energy policy. The computer model studied is an economic model of investment in electricity generation in Great Britain. The computer model was used to select parameters in a government policy designed to incentivize investment in renewable technologies to meet government targets. Limited computing time meant that few runs of the computer model were available to fit an emulator. The statistical methodology developed was therefore focused on accurately capturing the uncertainty in computer model output arising from the small number of available model runs. A varying coefficient emulator is proposed to model uncertainty in model output when extrapolating away from model runs. To maximize use of the small number of runs available, this varying coefficient emulator is paired with a criterion-based procedure for design selection.

Does auditory deprivation impairs statistical learning in the auditory modality?

Early sensory deprivation allows assessing the extent of reorganisation of cognitive functions, well beyond sensory processing. As such, it is a good model to explore the links between sensory experience and cognitive functions. One of these functions, statistical learning – the ability to extract and use regularities present in the environment – is suspected to be impaired in prelingually deaf children with a cochlear implant. However, empirical evidence supporting this claim is very scarce and studies have reported contradictory results. This might be because previous studies have tested statistical learning only in the visual modality and did not make clear distinctions between multiple types of statistical regularities. To overcome these problems, we designed a modified serial reaction time task where cochlear implanted children and normal hearing children had to react to auditory sequences that embed multiple statistical regularities, namely transition probabilities of 0th, 1st or 2nd order. We compared the reaction times of the children with the output of a simple computational model that learns transition probabilities. First, 6–12 years old children were able to learn 0th and 1st order transition probabilities but not 2nd order ones. Second, there were no differences between cochlear implanted children and their normal hearing peers. These results indicate that auditory statistical learning is preserved in congenitally deaf children with cochlear implants. This suggests in turn that early auditory deprivation might not be crucially detrimental for the normal development of statistical learning.

EFFICIENT CALIBRATION FOR HIGH-DIMENSIONAL COMPUTER MODEL OUTPUT USING BASIS METHODS

Calibration of expensive computer models with high-dimensional output fields can be approached via history matching. If the entire output field is matched, with patterns or correlations between locations or time points represented, calculating the distance metric between observational data and model output for a single input setting requires a time intensive inversion of a high-dimensional matrix. By using a low-dimensional basis representation rather than emulating each output individually, we define a metric in the reduced space that allows the implausibility for the field to be calculated efficiently, with only small matrix inversions required, using projection that is consistent with the variance specifications in the implausibility. We show that projection using the $L_2$ norm can result in different conclusions, with the ordering of points not maintained on the basis, with implications for both history matching and probabilistic methods. We demonstrate the scalability of our method through history matching of the Canadian atmosphere model, CanAM4, comparing basis methods to emulation of each output individually, showing that the basis approach can be more accurate, whilst also being more efficient.

Complexity and data mining in dental research: A network medicine perspective on interceptive orthodontics.

Procedures and models of computerized data analysis are becoming researchers' and practitioners' thinking partners by transforming the reasoning underlying biomedicine. Complexity theory, Network analysis and Artificial Intelligence are already approaching this discipline, intending to provide support for patient's diagnosis, prognosis and treatments. At the same time, due to the sparsity, noisiness and time‐dependency of medical data, such procedures are raising many unprecedented problems related to the mismatch between the human mind's reasoning and the outputs of computational models. Thanks to these computational, non‐anthropocentric models, a patient's clinical situation can be elucidated in the orthodontic discipline, and the growth outcome can be approximated. However, to have confidence in these procedures, orthodontists should be warned of the related benefits and risks. Here we want to present how these innovative approaches can derive better patients' characterization, also offering a different point of view about patient's classification, prognosis and treatment.

Electrodynamic Computer Model of a Metal Rod in a Concrete Medium Detection

The main aim of the present work is to describe a generalized electrodynamic model of the specific device operation, which provides detection and measurement of the geometric characteristics of the reinforcement cage in a concrete structure. The result of this study is the design of the device’s receiving-transmitting path that is optimal in a number of parameters, as well as a signal processing algorithm based on the operation of an artificial neural network trained on the generalized computer model output data. In addition to the main functionality of the device being developed, which consists in structureroscopy by the method of radar holography, useful target characteristics may be obtained during its operation: electrophysical parameters of concrete, structural defects, visualization of an object, etc. To solve this problem, on the basis of general radar principles, frequency-modulated continuous wave was chosen as the operating mode of the device. To create an electrodynamic model, the computer-aided design environment Pathwave EM Design (EMPro) 2021 was used. The developed generalized model may be optimized for a large number of parameters. In addition to the position and number of receiving antennas, the list of optimization variables may include parameters of the transmitting antenna (ray width, directivity, near-field distance), their number (i.e., the capacity of the MIMO system), power on the transmitting side, etc. The proposed scheme of the device is presented as the main result.

A comparison of constitutive models for describing the flow of uncured styrene-butadiene rubber

Uncured styrene-butadiene rubber (SBR) can be modelled as a viscoelastic material with at least two different relaxation mechanisms. In this paper we compare multi-mode constitutive models combining two viscoelastic modes (linear and/or nonlinear) in three possible ways. Our particular choice of the two modes was inspired by models originally developed to describe the response of asphalt binders. We select the model that best fits the experimental data obtained from a modified stress relaxation experiment in the torsional configuration of the plate–plate rheometer. The optimisation of the five model parameters for each model is achieved by minimising the weighted least-squares distance between experimental observations and the computer model output using a tree-structured Parzen estimator algorithm to find an initial guess, followed by further optimisation using the Nelder–Mead simplex algorithm. The results show that the model combining the linear mode and the nonlinear mode is the most suitable variant to describe the observed behaviour of SBR in the given regime. The predictive capabilities of the three models are further examined in changed experimental and numerical configurations. Full data and code to produce the figures in this article are included as supplementary material.

Diagnostics-Driven Nonstationary Emulators Using Kernel Mixtures

Weakly stationary Gaussian processes (GPs) are the principal tools in the statistical approaches to the design and analysis of computer experiments (or uncertainty quantification). Such processes are fitted to computer model output using a set of training runs to learn the parameters of the process covariance kernel. The stationarity assumption is often adequate, yet can lead to poor predictive performance when the model response exhibits nonstationarity, for example, if its smoothness varies across the input space. In this paper, we introduce a diagnostic-led approach to fitting nonstationary GP emulators by specifying finite mixtures of region-specific covariance kernels. Our method first fits a stationary GP and, if traditional diagnostics exhibit nonstationarity, those diagnostics are used to fit appropriate mixing functions for a covariance kernel mixture designed to capture the nonstationarity, ensuring an emulator that is continuous in parameter space and readily interpretable. We compare our approach to the principal nonstationary GP models in the literature and illustrate its performance on a number of idealized test cases and in an application to modeling the cloud parameterization of the French climate model.

Fast parameter inference in a biomechanical model of the left ventricle by using statistical emulation.

SummaryA central problem in biomechanical studies of personalized human left ventricular modelling is estimating the material properties and biophysical parameters from in vivo clinical measurements in a timeframe that is suitable for use within a clinic. Understanding these properties can provide insight into heart function or dysfunction and help to inform personalized medicine. However, finding a solution to the differential equations which mathematically describe the kinematics and dynamics of the myocardium through numerical integration can be computationally expensive. To circumvent this issue, we use the concept of emulation to infer the myocardial properties of a healthy volunteer in a viable clinical timeframe by using in vivo magnetic resonance image data. Emulation methods avoid computationally expensive simulations from the left ventricular model by replacing the biomechanical model, which is defined in terms of explicit partial differential equations, with a surrogate model inferred from simulations generated before the arrival of a patient, vastly improving computational efficiency at the clinic. We compare and contrast two emulation strategies: emulation of the computational model outputs and emulation of the loss between the observed patient data and the computational model outputs. These strategies are tested with two interpolation methods, as well as two loss functions. The best combination of methods is found by comparing the accuracy of parameter inference on simulated data for each combination. This combination, using the output emulation method, with local Gaussian process interpolation and the Euclidean loss function, provides accurate parameter inference in both simulated and clinical data, with a reduction in the computational cost of about three orders of magnitude compared with numerical integration of the differential equations by using finite element discretization techniques.

A Sequential Design Approach for Calibrating Dynamic Computer Simulators

A comprehensive understanding of the population growth of a variety of pests is often crucial for efficient crop management. Our motivating application comes from calibrating a two-delay blowfly (TDB) model which is used to simulate the population growth of Panonychus ulmi (Koch) or European red mites that infest on apple leaves and diminish the yield. We focus on the inverse problem, that is, to estimate the set of parameters/inputs of the TDB model that produces the computer model output matching the field observation as closely as possible. The time series nature of both the field observation and the TDB outputs makes the inverse problem significantly more challenging than in the scalar valued simulator case. In spirit, we follow the popular sequential design framework of computer experiments. However, due to the time-series response, a singular value decomposition based Gaussian process model is used for the surrogate model, and subsequently, a new expected improvement criterion is developed for choosing the follow-up points. We also propose a new criterion for extracting the optimal inverse solution from the final surrogate. Three simulated examples and the real-life TDB calibration problem have been used to demonstrate higher accuracy of the proposed approach as compared to popular existing techniques.

Adaptation reveals sensory and decision components in the visual estimation of locomotion speed

Locomotion speed provides important social information about an individual’s fitness, mood and intent. Visual estimation of locomotion speed is a complex task for the visual system because viewing distance must be taken into account, and the estimate has to be calibrated by recent experience of typical speeds. Little is known about how locomotion speed judgements are made. Previous research indicates that the human visual system possesses neurons that respond specifically to moving human forms. This research used point-light walker (PLW) displays that are known to activate these cells, in order to investigate the process mediating locomotion speed judgements. The results of three adaptation experiments show that these judgements involve both a low-level sensory component and a high-level decision component. A simple theoretical scheme is proposed, in which neurons sensitive to image flicker rate (temporal frequency) provide a sensory speed code, and a benchmark ‘norm’ value of the speed code, based on prevailing locomotion speeds, is used to make decisions about objective speed. The output of a simple computational model of the scheme successfully captured variations in locomotion speed in the stimuli used in the experiments. The theory offers a biologically-motivated account of how locomotion speed can be visually estimated.

Inverse uncertainty quantification using the modular Bayesian approach based on Gaussian process, Part 1: Theory

In nuclear reactor system design and safety analysis, the Best Estimate plus Uncertainty (BEPU) methodology requires that computer model output uncertainties must be quantified in order to prove that the investigated design stays within acceptance criteria. "Expert opinion" and "user self-evaluation" have been widely used to specify computer model input uncertainties in previous uncertainty, sensitivity and validation studies. Inverse Uncertainty Quantification (UQ) is the process to inversely quantify input uncertainties based on experimental data in order to more precisely quantify such ad-hoc specifications of the input uncertainty information. In this paper, we used Bayesian analysis to establish the inverse UQ formulation, with systematic and rigorously derived metamodels constructed by Gaussian Process (GP). Due to incomplete or inaccurate underlying physics, as well as numerical approximation errors, computer models always have discrepancy/bias in representing the realities, which can cause over-fitting if neglected in the inverse UQ process. The model discrepancy term is accounted for in our formulation through the "model updating equation". We provided a detailed introduction and comparison of the full and modular Bayesian approaches for inverse UQ, as well as pointed out their limitations when extrapolated to the validation/prediction domain. Finally, we proposed an improved modular Bayesian approach that can avoid extrapolating the model discrepancy that is learnt from the inverse UQ domain to the validation/prediction domain.

Calibrating a Stochastic, Agent-Based Model Using Quantile-Based Emulation

In a number of cases, the quantile Gaussian process has proven effective in emulating stochastic, univariate computer model output [M. Plumlee and R. Tuo, Technometrics, 56 (2014), pp. 466--473]. In this paper, we develop an approach that uses this emulation approach within a Bayesian model calibration framework to calibrate an agent-based model of an epidemic. In addition, this approach is extended to handle the multivariate nature of the model output, which gives a time series of the count of infected individuals. The basic modeling approach is adapted from [D. Higdon et al., J. Amer. Statist. Assoc., 103 (2008), pp. 570--583], using a basis representation to capture the multivariate model output. The approach is motivated with an example taken from the 2015 Ebola challenge workshop which simulated an Ebola epidemic to evaluate methodology.

Temporal weighting of azimuthal cues in the free field and in headphone listening

Dynamic aspects of binaural cue processing can be captured via several types of psychophysical measurements in human listeners. These include sensitivity measures with spatially dynamic stimuli, temporal integration measurements, and perceptual weighting measurement with stochastic spatial-cue variation. The latter approach has been adopted in a series of papers and ongoing studies. These assessed the time course of binaural lateralization and azimuthal localization as a function of stimulus frequency, bandwidth, modulation rate, temporal regularity, duration, envelope shape, and reverberation. The detailed temporal profile of perceptual weights, or “temporal weighting function,” can be captured psychophysically and through Monte Carlo simulations based on computational binaural model outputs. Among the results that will be reviewed in this talk are (1) demonstration of sparse temporal sampling of binaural cues at moments of positive envelope slope, (2) impacts of cochlear interference on low-frequency bi...

On the Bayesian calibration of computer model mixtures through experimental data, and the design of predictive models

For many real systems, several computer models may exist with different physics and predictive abilities. To achieve more accurate simulations/predictions, it is desirable for these models to be properly combined and calibrated. We propose the Bayesian calibration of computer model mixture method which relies on the idea of representing the real system output as a mixture of the available computer model outputs with unknown input dependent weight functions. The method builds a fully Bayesian predictive model as an emulator for the real system output by combining, weighting, and calibrating the available models in the Bayesian framework. Moreover, it fits a mixture of calibrated computer models that can be used by the domain scientist as a mean to combine the available computer models, in a flexible and principled manner, and perform reliable simulations. It can address realistic cases where one model may be more accurate than the others at different input values because the mixture weights, indicating the contribution of each model, are functions of the input. Inference on the calibration parameters can consider multiple computer models associated with different physics. The method does not require knowledge of the fidelity order of the models. We provide a technique able to mitigate the computational overhead due to the consideration of multiple computer models that is suitable to the mixture model framework. We implement the proposed method in a real-world application involving the Weather Research and Forecasting large-scale climate model.

Gaussian Process Emulators for Computer Experiments with Inequality Constraints

Physical phenomena are observed in many fields (science and engineering) and are often studied by time-consuming computer codes. These codes are analyzed with statistical models, often called emulators. In many situations, the physical system (computer model output) may be known to satisfy inequality constraints with respect to some or all input variables. The aim is to build a model capable of incorporating both data interpolation and inequality constraints into a Gaussian process emulator. By using a functional decomposition, a finite-dimensional approximation of Gaussian processes such that all conditional simulations satisfy the inequality constraints in the entire domain is proposed. To show the performance of the proposed model, some conditional simulations with inequality constraints such as boundedness, monotonicity or convexity conditions in one and two dimensions are given. A simulation study to investigate the efficiency of the method in terms of prediction is included.

Superior colliculus neurons encode a visual saliency map during free viewing of natural dynamic video

Models of visual attention postulate the existence of a saliency map whose function is to guide attention and gaze to the most conspicuous regions in a visual scene. Although cortical representations of saliency have been reported, there is mounting evidence for a subcortical saliency mechanism, which pre-dates the evolution of neocortex. Here, we conduct a strong test of the saliency hypothesis by comparing the output of a well-established computational saliency model with the activation of neurons in the primate superior colliculus (SC), a midbrain structure associated with attention and gaze, while monkeys watched video of natural scenes. We find that the activity of SC superficial visual-layer neurons (SCs), specifically, is well-predicted by the model. This saliency representation is unlikely to be inherited from fronto-parietal cortices, which do not project to SCs, but may be computed in SCs and relayed to other areas via tectothalamic pathways.

Addressing factors fixing setting from given data: A comparison of different methods

This paper deals with global sensitivity analysis of computer model output. Given an independent input sample and associated model output vector with possibly the vector of output derivatives with respect to the input variables, we show that it is possible to evaluate the following global sensitivity measures: (i) the Sobol’ indices, (ii) the Borgonovo's density-based sensitivity measure, and (iii) the derivative-based global sensitivity measure of Sobol’ and Kucherenko. We compare the efficiency of the different methods to address factors fixing setting, an important issue in global sensitivity analysis. First, global sensitivity analysis of the Ishigami function is performed with the different methods. Then, they are applied to two different responses of a soil drainage model. The results show that the polynomial chaos expansion for estimating Sobol’ indices is the most efficient approach.

The wrapped skew Gaussian process for analyzing spatio-temporal data

We consider modeling of angular or directional data viewed as a linear variable wrapped onto a unit circle. In particular, we focus on the spatio-temporal context, motivated by a collection of wave directions obtained as computer model output developed dynamically over a collection of spatial locations. We propose a novel wrapped skew Gaussian process which enriches the class of wrapped Gaussian process. The wrapped skew Gaussian process enables more flexible marginal distributions than the symmetric ones arising under the wrapped Gaussian process and it allows straightforward interpretation of parameters. We clarify that replication through time enables criticism of the wrapped process in favor of the wrapped skew process. We formulate a hierarchical model incorporating this process and show how to introduce appropriate latent variables in order to enable efficient fitting to dynamic spatial directional data. We also show how to implement kriging and forecasting under this model. We provide a simulation example as a proof of concept as well as a real data example. Both examples reveal consequential improvement in predictive performance for the wrapped skew Gaussian specification compared with the earlier wrapped Gaussian version.

Predicting Ecological Roles in the Rhizosphere Using Metabolome and Transportome Modeling.

The ability to obtain complete genome sequences from bacteria in environmental samples, such as soil samples from the rhizosphere, has highlighted the microbial diversity and complexity of environmental communities. However, new algorithms to analyze genome sequence information in the context of community structure are needed to enhance our understanding of the specific ecological roles of these organisms in soil environments. We present a machine learning approach using sequenced Pseudomonad genomes coupled with outputs of metabolic and transportomic computational models for identifying the most predictive molecular mechanisms indicative of a Pseudomonad’s ecological role in the rhizosphere: a biofilm, biocontrol agent, promoter of plant growth, or plant pathogen. Computational predictions of ecological niche were highly accurate overall with models trained on transportomic model output being the most accurate (Leave One Out Validation F-scores between 0.82 and 0.89). The strongest predictive molecular mechanism features for rhizosphere ecological niche overlap with many previously reported analyses of Pseudomonad interactions in the rhizosphere, suggesting that this approach successfully informs a system-scale level understanding of how Pseudomonads sense and interact with their environments. The observation that an organism’s transportome is highly predictive of its ecological niche is a novel discovery and may have implications in our understanding microbial ecology. The framework developed here can be generalized to the analysis of any bacteria across a wide range of environments and ecological niches making this approach a powerful tool for providing insights into functional predictions from bacterial genomic data.

Full scale multi-output Gaussian process emulator with nonseparable auto-covariance functions

Gaussian process emulator with separable covariance function has been utilized extensively in modeling large computer model outputs. The assumption of separability imposes constraints on the emulator and may negatively affect its performance in some applications where separability may not hold. We propose a multi-output Gaussian process emulator with a nonseparable auto-covariance function to avoid limitations of using separable emulators. In addition, to facilitate the computation of nonseparable emulator, we introduce a new computational method, referred to as the Full-Scale approximation method with block modulating function (FSA-Block) approach. The FSA-Block is an effective and accurate covariance approximation method to reduce computations for Gaussian process models, which applies to both nonseparable and partially separable covariance models. We illustrate the effectiveness of our method through simulation studies and compare it with emulators with separable covariances. We also apply our method to a real computer code of the carbon capture system.