Stochastic Kriging Research Articles

Kriging metamodels and experimental design for Bermudan option pricing

We investigate two new strategies for the numerical solution of optimal stopping problems in the regression Monte Carlo (RMC) framework proposed by Longstaff and Schwartz in 2001. First, we propose using stochastic kriging (Gaussian process) metamodels for fitting the continuation value. Kriging offers a flexible, non-parametric regression approach that quantifies approximation quality. Second, we connect the choice of stochastic grids used in RMC with the design of experiments (DoE) paradigm. We examine space-filling and adaptive experimental designs; we also investigate the use of batching with replicated simulations at design sites to improve the signal-to-noise ratio. Numerical case studies for valuing Bermudan puts and max calls under a variety of asset dynamics illustrate that our methods offer a significant reduction in simulation budgets compared with existing approaches.

ASTRO-DF: A Class of Adaptive Sampling Trust-Region Algorithms for Derivative-Free Stochastic Optimization

We consider unconstrained optimization problems where only “stochastic” estimates of the objective function are observable as replicates from a Monte Carlo oracle. The Monte Carlo oracle is assumed to provide no direct observations of the function gradient. We present ASTRO-DF---a class of derivative-free trust-region algorithms, where a stochastic local model is constructed, optimized, and updated iteratively. Function estimation and model construction within ASTRO-DF is adaptive in the sense that the extent of Monte Carlo sampling is determined by continuously monitoring and balancing measures of sampling error (or variance) and structural error (or model bias) within ASTRO-DF. Such balancing of errors is designed to ensure that Monte Carlo effort within ASTRO-DF is sensitive to algorithm trajectory: sampling is higher whenever an iterate is inferred to be close to a critical point and lower when far away. We demonstrate the almost sure convergence of ASTRO-DF's iterates to first-order critical points when using stochastic polynomial interpolation models. The question of using more complicated models, e.g., regression or stochastic kriging, in combination with adaptive sampling is worth further investigation and will benefit from the methods of proof presented here. We speculate that ASTRO-DF's iterates achieve the canonical Monte Carlo convergence rate, although a proof remains elusive.

Green Simulation

We introduce a new paradigm in simulation experiment design and analysis, called “green simulation,” for the setting in which experiments are performed repeatedly with the same simulation model. Green simulation means reusing outputs from previous experiments to answer the question currently being asked of the simulation model. As one method for green simulation, we propose estimators that reuse outputs from previous experiments by weighting them with likelihood ratios, when parameters of distributions in the simulation model differ across experiments. We analyze convergence of these estimators as more experiments are repeated, while a stochastic process changes the parameters used in each experiment. As another method for green simulation, we propose an estimator based on stochastic kriging. We find that green simulation can reduce mean squared error by more than an order of magnitude in examples involving catastrophe bond pricing and credit risk evaluation.

Estimation of the Pareto front in stochastic simulation through stochastic Kriging

Multi-objective optimization of simulated stochastic systems aims at estimating a representative set of Pareto optimal solutions and a common approach is to rely on metamodels to alleviate computational costs of the optimization process. In this article, both the objective and constraint functions are assumed to be smooth, highly non-linear and computationally expensive and are emulated by stochastic Kriging models. Then a novel global optimization algorithm, combing the expected hypervolume improvement of approximated Pareto front and the probability of feasibility of new points, is proposed to identify the Pareto front (set) with a minimal number of expensive simulations. The algorithm is suitable for the situations of having disconnected feasible regions and of having no feasible solution in initial design. Then, we also quantify the variability of estimated Pareto front caused by the intrinsic uncertainty of stochastic simulation using nonparametric bootstrapping method to better support decision making. One test function and an (s, S) inventory system experiment illustrate the potential and efficiency of the proposed sequential optimization algorithm for constrained multi-objective optimization problems in stochastic simulation, which is especially useful in Operations Research and Management Science.

An Empirical Validation Protocol for Large-Scale Agent-Based Models

Despite recent advances in bringing agent-based models (ABMs) to the data, the estimation or calibration of model parameters remains a challenge, especially when it comes to large-scale agent-based macroeconomic models. Most methods, such as the method of simulated moments (MSM), require in-the-loop simulation of new data, which may not be feasible for such computationally heavy simulation models. The purposes of this paper is to provide a proof-of-concept of a generic empirical validation methodology for such large-scale simulation models. We introduce an alternative 'large-scale' empirical validation approach, and apply it to the Eurace@Unibi macroeconomic simulation model (Dawid, et al., 2016). This model was selected because it displays strong emergent behaviour and is able to generate a wide variety on nonlinear economic dynamics, including endogenous business- and financial cycles. In addition, it is a computationally heavy simulation mode, so it fits our targeted use-case. The validation protocol consists of three stages. At the first stage we use Nearly-Orthogonal Latin Hypercube sampling (NOLH) in order to generate a set of 513 parameter combinations with good space-filling properties. At the second stage we use the recently developed Markov Information Criterion (MIC) to score the simulated data against empirical data. Finally, at the third stage we use stochastic kriging to construct a surrogate model if the MIC response surface, resulting in an interpolation of the response surface as a function of the parameters. The parameter combinations providing the best fit to the data are then identified as the local minima of the interpolated MIC response surface. The Model Confidence Set (MCS) procedure of Hansen et al. (2011) is used to restrict the set of model calibrations to those models that cannot be rejected to have equal predictive ability, at a given confidence level. Validation of the surrogate model is carried out by re-running the second stage of the analysis on the so identified optima and cross-checking that the realised MIC scores equal the MIC scores predicted by the surrogate model. The results we obtain so far look promising as a first proof-of-concept for the empirical validation methodology since we are able to validate the model using empirical data series for 30 OECD countries and the euro area. The internal validation procedure of the surrogate model also suggests that the combination of NOLH sampling, MIC measurement and stochastic kriging yields reliable predictions of the MIC scores for samples not included in the original NOLH sample set. In our opinion, this is a strong indication that the method we propose could provide a viable statistical machine learning technique for the empirical validation of (large-scale) ABMs.

Sequential design strategies for mean response surface metamodeling via stochastic kriging with adaptive exploration and exploitation

Stochastic kriging (SK) methodology has been known as an effective metamodeling tool for approximating a mean response surface implied by a stochastic simulation. In this paper we provide some theoretical results on the predictive performance of SK, in light of which novel integrated mean squared error-based sequential design strategies are proposed to apply SK for mean response surface metamodeling with a fixed simulation budget. Through numerical examples of different features, we show that SK with the proposed strategies applied holds great promise for achieving high predictive accuracy by striking a good balance between exploration and exploitation.

Robust Optimization of Variable-Camber Continuous Trailing-Edge Flap Static Aeroelastic Action

A robust optimization method is described in this paper, focusing on the optimization of a variable-camber continuous trailing-edge flap system for drag reduction in transonic cruise. The variable-camber continuous trailing-edge flap, through the rich combination of shapes it can attain, has the potential to provide greater design and operation freedom than conventional flap systems as well as greater efficiency in drag reduction and performance improvement. The deformation shape optimization of such a system, however, when based on rigorous computational fluid dynamics simulations, presents a significant computational challenge because of the high cost of repetitive analyses required. The construction of surrogate models and their use in the optimization process is one common way to tackle this challenge. Here, stochastic kriging is used, extending the commonly used deterministic kriging. Accounting for the intrinsic uncertainty of data, stochastic kriging and the optimization based on it are well suited for addressing uncertain flight conditions as well as math modeling limitations. An optimization process for the variable-camber continuous trailing-edge flap system that couples shape synthesis with high-fidelity aerodynamic simulations, accounting for static aeroelastic interactions and load distribution constraints, is presented. A potential for robust optimal drag subject to flight Mach number uncertainties is shown.

Efficient VaR and CVaR Measurement via Stochastic Kriging

In this paper, stochastic kriging (SK) is considered as a metamodeling tool to capture risk-related properties of complex stochastic system outputs. To better assess the tail behavior of the underlying distribution of a system output, we specifically focus on global prediction of value-at-risk and conditional value-at-risk. Going beyond the standard SK framework intended for metamodeling of mean response surfaces, we rethink the original formulation to allow for the flexibility of utilizing different estimation methods for metamodel construction. The resulting impact on the predictive performance of SK is examined in detail. In parallel with the study by Chen et al. [Chen X, Ankenman BE, Nelson BL (2013) Enhancing stochastic kriging metamodels with gradient estimators. Oper. Res. 61(2):512–528], we further consider the situation in which noisy gradient information can be incorporated into SK metamodel construction and prediction. The theoretical results are illustrated by two numerical examples.

Efficient VaR and CVaR Measurement via Stochastic Kriging

In this paper, stochastic kriging SK is considered as a metamodeling tool to capture risk-related properties of complex stochastic system outputs. To better assess the tail behavior of the underlyi...

Multivariate Input Uncertainty in Output Analysis for Stochastic Simulation

When we use simulations to estimate the performance of stochastic systems, the simulation is often driven by input models estimated from finite real-world data. A complete statistical characterization of system performance estimates requires quantifying both input model and simulation estimation errors. The components of input models in many complex systems could be dependent. In this paper, we represent the distribution of a random vector by its marginal distributions and a dependence measure: either product-moment or Spearman rank correlations. To quantify the impact from dependent input model and simulation estimation errors on system performance estimates, we propose a metamodel-assisted bootstrap framework that is applicable to cases when the parametric family of multivariate input distributions is known or unknown. In either case, we first characterize the input models by their moments that are estimated using real-world data. Then, we employ the bootstrap to quantify the input estimation error, and an equation-based stochastic kriging metamodel to propagate the input uncertainty to the output mean, which can also reduce the influence of simulation estimation error due to output variability. Asymptotic analysis provides theoretical support for our approach, while an empirical study demonstrates that it has good finite-sample performance.

A reduced form model for ozone based on two decades of CMAQ simulations for the continental United States

A Reduced Form Model (RFM) is a mathematical relationship between the inputs and outputs of an air quality model, permitting estimation of additional modeling without costly new regional-scale simulations. A 21-year Community Multiscale Air Quality (CMAQ) simulation for the continental United States provided the basis for the RFM developed in this study. Predictors included the principal component scores (PCS) of emissions and meteorological variables, while the predictand was the monthly mean of daily maximum 8-h CMAQ ozone for the ozone season at each model grid. The PCS form an orthogonal basis for RFM inputs. A few PCS incorporate most of the variability of emissions and meteorology, thereby reducing the dimensionality of the source-receptor problem. Stochastic kriging was used to estimate the model.The RFM was used to separate the effects of emissions and meteorology on ozone concentrations. By running the RFM with emissions constant (ozone dependent on meteorology), or constant meteorology (ozone dependent on emissions). Years with ozone-conducive meteorology were identified, and meteorological variables best explaining meteorology-dependent ozone were identified. Meteorology accounted for 19%–55% of ozone variability in the eastern US, and 39%–92% in the western US. Temporal trends estimated for original CMAQ ozone data and emission-dependent ozone were mostly negative, but the confidence intervals for emission-dependent ozone are much narrower. Emission-driven changes in monthly mean ozone levels for the period 2000–2010 ranged from 6.4 to 10.9 ppb for the eastern US and from 1.4 to 2.5 ppb for the western US.

Estimating the variance of the predictor in stochastic Kriging

We study the estimation of the true variance of the predictor in stochastic Kriging (SK). First, we obtain macroreplications for a SK metamodel that approximates a single-server simulation model; these macroreplications give independently and identically distributed predictions. This simulation may use common random numbers (CRN). From these macroreplications we conclude that the usual plug-in estimator of the variance significantly underestimates the true variance. Because macroreplications of practical simulation models are computationally expensive, we next formulate two bootstrap methods that use a single macroreplication: (i) a distribution-free method that resamples simulation replications (within the single macroreplication), and (ii) a parametric method that assumes a Gaussian distribution for the SK predictor, and estimates the (hyper)parameters of that distribution from the single macroreplication. Altogether we recommend distribution-free bootstrapping for the estimation of the SK predictor variance in practical simulation experiments.

A Critical Assessment of Kriging Model Variants for High-Fidelity Uncertainty Quantification in Dynamics of composite Shells

This paper presents a critical comparative assessment of Kriging model variants for surrogate based uncertainty propagation considering stochastic natural frequencies of composite doubly curved shells. The five Kriging model variants studied here are: Ordinary Kriging, Universal Kriging based on pseudo-likelihood estimator, Blind Kriging, Co-Kriging and Universal Kriging based on marginal likelihood estimator. First three stochastic natural frequencies of the composite shell are analysed by using a finite element model that includes the effects of transverse shear deformation based on Mindlin’s theory in conjunction with a layer-wise random variable approach. The comparative assessment is carried out to address the accuracy and computational efficiency of five Kriging model variants. Comparative performance of different covariance functions is also studied. Subsequently the effect of noise in uncertainty propagation is addressed by using the Stochastic Kriging. Representative results are presented for both individual and combined stochasticity in layer-wise input parameters to address performance of various Kriging variants for low dimensional and relatively higher dimensional input parameter spaces. The error estimation and convergence studies are conducted with respect to original Monte Carlo Simulation to justify merit of the present investigation. The study reveals that Universal Kriging coupled with marginal likelihood estimate yields the most accurate results, followed by Co-Kriging and Blind Kriging. As far as computational efficiency of the Kriging models is concerned, it is observed that for high-dimensional problems, CPU time required for building the Co-Kriging model is significantly less as compared to other Kriging variants.

Efficient Global Optimization for Black-Box Simulation Via Sequential Intrinsic Kriging

Efficient Global Optimization (EGO) is a popular method that searches sequentially for the global optimum of a simulated system. EGO treats the simulation model as a black-box, and balances local and global searches. In deterministic simulation, EGO uses ordinary Kriging (OK), which is a special case of universal Kriging (UK). In our EGO variant we use intrinsic Kriging (IK), which eliminates the need to estimate the parameters that quantify the trend in UK. In random simulation, EGO uses stochastic Kriging (SK), but we use stochastic IK (SIK). Moreover, in random simulation, EGO needs to select the number of replications per simulated input combination, accounting for the heteroscedastic variances of the simulation outputs. A popular selection method uses optimal computer budget allocation (OCBA), which allocates the available total number of replications over simulated combinations. We derive a new allocation algorithm. We perform several numerical experiments with deterministic simulations and random simulations. These experiments suggest that (1) in deterministic simulations, EGO with IK outperforms classic EGO; (2) in random simulations, EGO with SIK and our allocation rule does not differ significantly from EGO with SK combined with the OCBA allocation rule.

A method for the updating of stochastic kriging metamodels

Two standard approaches to predicting the expected values of simulation outputs are either execution of the simulation itself or the use of a metamodel. In this work we propose a methodology that enables both approaches to be combined. When a prediction for a new input is required the procedure is to augment the metamodel forecast with additional simulation outputs for a given input. The key benefit of the method is that it is possible to reach the desired prediction accuracy at a new input faster than in the case when no initial metamodel is present. We show that such a procedure is computationally simple and can be applied to, for instance, web-based simulations, where response time to user actions is often crucial.In this analysis we focus on stochastic kriging metamodels. We show that if this type of metamodel is used and we assume that its metaparameters are fixed, then updating such a metamodel with new observations is equivalent to a Bayesian forecast combination under the known variance assumption. Additionally we observe that using metamodel predictions of variance instead of point estimates for estimation of stochastic kriging metamodes can lead to improved metamodel performance.

Estimating the Variance of the Predictor in Stochastic Kriging

Estimating the Variance of the Predictor in Stochastic Kriging

Gradient Extrapolated Stochastic Kriging

We introduce an approach for enhancing stochastic kriging in the setting where additional direct gradient information is available (e.g., provided by techniques such as perturbation analysis or the likelihood ratio method). The new approach, called gradient extrapolated stochastic kriging (GESK) , incorporates direct gradient estimates by extrapolating additional responses. For two simplified settings, we show that GESK reduces mean squared error (MSE) compared to stochastic kriging under certain conditions on step sizes. Since extrapolation step sizes are crucial to the performance of the GESK model, we propose two different approaches to determine the step sizes: maximizing penalized likelihood and minimizing integrated mean squared error. Numerical experiments are conducted to illustrate the performance of the GESK model and to compare it with alternative approaches.

A New Stochastic Kriging Method for Modeling Multi-Source Exposure-Response Data in Toxicology Studies.

One of the most fundamental steps in risk assessment is to quantify the exposure–response relationship for the material/chemical of interest. This work develops a new statistical method, referred to as SKQ (stochastic kriging with qualitative factors), to synergistically model exposure–response data, which often arise from multiple sources (e.g., laboratories, animal providers, and shapes of nanomaterials) in toxicology studies. Compared to the existing methods, SKQ has several distinct features. First, SKQ integrates data across multiple sources and allows for the derivation of more accurate information from limited data. Second, SKQ is highly flexible and able to model practically any continuous response surfaces (e.g., dose–time–response surface). Third, SKQ is able to accommodate variance heterogeneity across experimental conditions and to provide valid statistical inference (i.e., quantify uncertainties of the model estimates). Through empirical studies, we have demonstrated SKQ’s ability to efficiently model exposure–response surfaces by pooling information across multiple data sources. SKQ fits into the mosaic of efficient decision-making methods for assessing the risk of a tremendously large variety of nanomaterials and helps to alleviate safety concerns regarding the enormous amount of new nanomaterials.

Stochastic kriging with biased sample estimates

Stochastic kriging has been studied as an effective metamodeling technique for approximating response surfaces in the context of stochastic simulation. In a simulation experiment, an analyst typically needs to estimate relevant metamodel parameters and further do prediction; therefore, the impact of parameter estimation on the performance of the metamodel-based predictor has drawn some attention in the literature. However, how the standard stochastic kriging predictor is affected by the presence of bias in finite-sample estimates has not yet been fully investigated. In this article, we study the predictive performance and investigate optimal budget allocation rules subject to a fixed computational budget constraint. Furthermore, we extend the analysis to two-level or nested simulation, which has been recently documented in the risk management literature, with biased estimators.

Enhancing Stochastic Kriging Metamodels with Gradient Estimators

Stochastic kriging is a new metamodeling technique for effectively representing the mean response surface implied by a stochastic simulation; it takes into account both stochastic simulation noise and uncertainty about the underlying response surface of interest. We show theoretically, through some simplified models, that incorporating gradient estimators into stochastic kriging tends to significantly improve surface prediction. To address the issue of which type of gradient estimator to use, when there is a choice, we briefly review stochastic gradient estimation techniques; we then focus on the properties of infinitesimal perturbation analysis and likelihood ratio/score function gradient estimators and make recommendations. To conclude, we use simulation experiments with no simplifying assumptions to demonstrate that the use of stochastic kriging with gradient estimators provides more reliable prediction results than stochastic kriging alone.