Stochastic Kriging Research Articles

Bi-objective ranking and selection using stochastic kriging

Bi-objective ranking and selection using stochastic kriging

Life cycle assessment of metal powder production: a Bayesian stochastic Kriging model-based autonomous estimation

Metal powder contributes to the environmental burdens of additive manufacturing (AM) substantially. Current life cycle assessments (LCAs) of metal powders present considerable variations of lifecycle environmental inventory due to process divergence, spatial heterogeneity, or temporal fluctuation. Most importantly, the amounts of LCA studies on metal powder are limited and primarily confined to partial material types. To this end, based on the data surveyed from a metal powder supplier, this study conducted an LCA of titanium and nickel alloy produced by electrode-inducted and vacuum-inducted melting gas atomization, respectively. Given that energy consumption dominates the environmental burden of powder production and is influenced by metal materials’ physical properties, we proposed a Bayesian stochastic Kriging model to estimate the energy consumption during the gas atomization process. This model considered the inherent uncertainties of training data and adaptively updated the parameters of interest when new environmental data on gas atomization were available. With the predicted energy use information of specific powder, the corresponding lifecycle environmental impacts can be further autonomously estimated in conjunction with the other surveyed powder production stages. Results indicated the environmental impact of titanium alloy powder is slightly higher than that of nickel alloy powder and their lifecycle carbon emissions are around 20 kg CO2 equivalency. The proposed Bayesian stochastic Kriging model showed more accurate predictions of energy consumption compared with conventional Kriging and stochastic Kriging models. This study enables data imputation of energy consumption during gas atomization given the physical properties and producing technique of powder materials.

A Uniform Error Bound for Stochastic Kriging: Properties and Implications on Simulation Experimental Design

A Uniform Error Bound for Stochastic Kriging: Properties and Implications on Simulation Experimental Design

Sequential metamodel‐based approaches to level‐set estimation under heteroscedasticity

AbstractThis paper proposes two sequential metamodel‐based methods for level‐set estimation (LSE) that leverage the uniform bound built on stochastic kriging: predictive variance reduction (PVR) and expected classification improvement (ECI). We show that PVR and ECI possess desirable theoretical performance guarantees and provide closed‐form expressions for their respective sequential sampling criteria to seek the next design point for performing simulation runs, allowing computationally efficient one‐iteration look‐ahead updates. To enhance understanding, we reveal the connection between PVR and ECI's sequential sampling criteria. Additionally, we propose integrating a budget allocation feature with PVR and ECI, which improves computational efficiency and potentially enhances robustness to the impacts of heteroscedasticity. Numerical studies demonstrate the superior performance of the proposed methods compared to state‐of‐the‐art benchmarking approaches when given a fixed simulation budget, highlighting their effectiveness in addressing LSE problems.

Real-Time Derivative Pricing and Hedging with Consistent Metamodels

In derivative pricing and hedging, the consistency between the price and Greek surfaces (i.e., the Greek surfaces can be obtained by differentiating the price surface) is important in stabilizing the balance sheet and reducing the hedging cost. To build consistent surfaces of the price and Greeks for real-time decisions, we propose to use the gradient-enhanced stochastic kriging method, based on the data collected through extensive simulation experiments conducted when the market is closed. In addition to the naturally guaranteed consistency, we prove that the constructed price and Greek surfaces are more accurate than those constructed separately using stochastic kriging. Besides the consistency between the price and Greeks, we show that the partial differential equation relation between the price and Greeks, implied by the famous Feynman-Kac formula, can also be used to further improve the accuracy of the constructed surfaces. The numerical studies show that our proposed metamodeling methods work well for derivative pricing and hedging. History: Accepted by Bruno Tuffin, Area Editor for Simulation. Funding: This work was supported by the National Natural Science Foundation of China [Grants 72161160340, 72293562, 72121001, 72031006, and 72171060]. Supplemental Material: The software that supports the findings of this study is available within the paper and its Supplemental Information ( https://pubsonline.informs.org/doi/suppl/10.1287/ijoc.2023.0292 ) as well as from the IJOC GitHub software repository ( https://github.com/INFORMSJoC/2023.0292 ). The complete IJOC Software and Data Repository is available at https://informsjoc.github.io/ .

Robust parameter design for 3D printing process using stochastic computer model

3D printing technology has been developing rapidly in recent years, but product quality control has become one of the main obstacles to its widespread use in manufacturing. A new stochastic computer model and robust optimization method are proposed for the highly fluctuating 3D printing process to improve the stability of the printed product quality. Firstly, the signal and noise are jointly modeled, and the idea of latent variables in machine learning is incorporated to overcome the limitation that the replication times of the stochastic Kriging model must be greater than one. Then, the chain rule and Woodbury identity are utilized to reduce the time required for hyperparameter estimation of the model. Finally, the optimization objective function is constructed based on the Taguchi quality loss function, and optimal process parameters are found using a genetic algorithm. The numerical simulation results demonstrate that the robust optimization method based on heteroskedasticity Gaussian process model proposed in this paper can estimate model hyperparameters faster and predict results more accurately. Furthermore, the prediction and validation results of 3D printing experiments verify the effectiveness of the proposed method.

Robust Optimization Design of the Aerodynamic Shape and External Ballistics of a Pulse Trajectory Correction Projectile

To improve the tactical and technical performance of pulse correction projectiles while maintaining stability in uncertain conditions and considering practical engineering constraints, this study performs a multi-objective robust optimization design of the aerodynamic shape and external ballistics of a projectile. The study utilizes an aerodynamic force engineering algorithm and numerical trajectory calculations to obtain the projectile’s performance responses within the Latin hypercube design space. To enhance optimization efficiency, a stochastic Kriging surrogate model is established to capture the inherent uncertainty of limited input data. Ultimately, a Pareto optimal solution for the projectile is obtained using a non-dominated sorting multi-objective sparrow search algorithm. The results of this study demonstrate that the consideration of design uncertainty in the robust optimization of pulse correction projectiles leads to significant enhancements in both lateral correction ability and range while satisfying flight stability requirements. Moreover, when compared to deterministic optimization, the performance variability of the design is markedly improved. This research methodology provides valuable insights for optimizing the performance of pulse correction projectiles.

Convergence Analysis of Stochastic Kriging-Assisted Simulation with Random Covariates

We consider performing simulation experiments in the presence of covariates. Here, covariates refer to some input information other than system designs to the simulation model that can also affect the system performance. To make decisions, decision makers need to know the covariate values of the problem. Traditionally in simulation-based decision making, simulation samples are collected after the covariate values are known; in contrast, as a new framework, simulation with covariates starts the simulation before the covariate values are revealed and collects samples on covariate values that might appear later. Then, when the covariate values are revealed, the collected simulation samples are directly used to predict the desired results. This framework significantly reduces the decision time compared with the traditional way of simulation. In this paper, we follow this framework and suppose there are a finite number of system designs. We adopt the metamodel of stochastic kriging (SK) and use it to predict the system performance of each design and the best design. The goal is to study how fast the prediction errors diminish with the number of covariate points sampled. This is a fundamental problem in simulation with covariates and helps quantify the relationship between the offline simulation efforts and the online prediction accuracy. Particularly, we adopt measures of the maximal integrated mean squared error (IMSE) and integrated probability of false selection (IPFS) for assessing errors of the system performance and the best design predictions. Then, we establish convergence rates for the two measures under mild conditions. Last, these convergence behaviors are illustrated numerically using test examples. History: Accepted by Bruno Tuffin, area editor for simulation. Funding: This work was supported in part by Singapore Ministry of Education Academic Research Funds [Tier 1 Grants R-155-000-201-114 and A-0004822-00-00], the City University of Hong Kong [Grants 7005269 and 7005568], and the National Natural Science Foundation of China [Grant 72091211]. Supplemental Material: The software that supports the findings of this study is available within the paper and its Supplemental Information ( https://pubsonline.informs.org/doi/suppl/10.1287/ijoc.2022.1263 ) as well as from the IJOC GitHub software repository ( https://github.com/INFORMSJoC/2021.0329 ) at ( http://dx.doi.org/10.5281/zenodo.7344997 ).

A collaborative model calibration framework under uncertainty considering parameter distribution

Model calibration is critical to update unknown parameters and model differences between simulation and experiment. While some of the current methods can quantify response uncertainty and estimate the likelihood function of unknown parameters, calibration remains challenging when the parameters are uncertainties of unknown distribution. Considering model uncertainty inversion, we propose a collaborative calibration framework that quantifies and calibrates the distribution of unknown parameters. We used the Nested Stochastic Kriging (NSK) model to estimate the global trend and uncertainty of response data. The unknown parameters are then updated using the optimization-based calibration (OBC) method, where an infill sampling criterion is proposed. Finally, the variance of the parameter is calibrated. The calibrated model can reflect the uncertainty of the response data and can be further used to estimate the distribution of a new design. The proposed approach is tested using two numerical instances, and the bolt simulation model under tension-bending condition is calibrated using 30 experimental data points. The results demonstrate that the proposed method can achieve the best performance in terms of accuracy and efficiency.

A novel Bayesian approach for multi-objective stochastic simulation optimization

Multi-objective stochastic simulation optimization plays an important role in designing complex engineering systems. To identify optimal solutions via simulations, Bayesian optimization, which utilizes metamodels and an acquisition function to determine the next design point, has been popular in machine learning. However, studies on Bayesian optimization of multi-objective stochastic simulation are limited and can give undesired performances in terms of convergence and diversity metrics. Moreover, direct extension of Bayesian optimization algorithms of deterministic responses is unrealistic as stochastic responses are observed with heterogeneous noise. To handle these issues, we propose a novel framework for Bayesian optimization of multiple stochastic responses. Stochastic kriging metamodels are constructed independently for multiple stochastic responses considering heterogeneous simulation noise. To remove undesired effects of random simulation noise on comparing solutions, quantiles of stochastic kriging metamodels are employed to define the Pareto dominance. Acquisition functions are proposed by integrating the hypervolume-based and probability-of-improvement (PoI)-based acquisition functions from deterministic multi-objective optimization with typical acquisition functions from univariate stochastic response optimization. This not only enhances the convergence and diversity of Pareto-optimal solutions but also properly handles heterogeneous noise. Exact calculation formulas of the proposed algorithms are further developed. Typical numerical cases and two engineering examples (a rocket injector design example and a real inventory design scenario) demonstrate that the proposed algorithms give better Pareto-optimal solutions of which the proximity to true Pareto fronts and the diversity are improved.

Experimental Case Study of Stochastic Surrogate-Assisted Engine Calibration

Reducing experimental calibration cost to achieve optimal system performance is challenging, especially for highly nonlinear engine systems. With the advancement and widespread adoption of machine learning methods for control applications, it is now possible to use an intelligent black-box model to efficiently optimize nonlinear control systems without the detailed knowledge of system dynamics. The surrogate-assisted optimization approach has been recently proven to work effectively to reduce the overall experimental budget (cost). However, its application to calibrate and optimize a practical control system experimentally is a challenge due to stochastic system and measurement noises. This article experimentally calibrates and optimizes the control parameters of a 6.7L Ford diesel engine using the stochastic surrogate-assisted optimization automatically based on a stochastic Kriging model, which is the first step toward automating the engine calibration process. Two engine control variables, namely, exhaust gas recirculation valve and variable geometry turbocharger vane positions, are calibrated to obtain a tradeoff between engine efficiency, in terms of brake-specific fuel consumption, and <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"><tex-math notation="LaTeX">$\text{NO}_{x}$</tex-math></inline-formula> emissions, utilizing both the deterministic and stochastic surrogate-assisted learning algorithms. Experimental results show a much better representation of the output response surfaces with the stochastic surrogate-assisted optimization than that from the deterministic approach. Note that the stochastic approach helps direct the search toward the global optimal region without wasting the evaluation budget in exploring the local optima, and the deterministic approach cannot effectively handle stochastic system and measurement noises, resulting in overfitting the data. Therefore, to solve a practical optimization problem with noises, stochastic surrogate-assisted optimization is recommended.

Sample and Computationally Efficient Stochastic Kriging in High Dimensions

High-dimensional Simulation Metamodeling Stochastic kriging has been widely employed for simulation metamodeling to predict the response surface of complex simulation models. However, its use is limited to cases where the design space is low-dimensional because the sample complexity (i.e., the number of design points required to produce an accurate prediction) grows exponentially in the dimensionality of the design space. The large sample size results in both a prohibitive sample cost for running the simulation model and a severe computational challenge due to the need to invert large covariance matrices. To address this long-standing challenge, Liang Ding and Xiaowei Zhang, in their recent paper “Sample and Computationally Efficient Stochastic Kriging in High Dimensions”, develop a novel methodology — based on tensor Markov kernels and sparse grid experimental designs — that dramatically alleviates the curse of dimensionality. The proposed methodology has theoretical guarantees on both sample complexity and computational complexity and shows outstanding performance in numerical problems of as high as 16,675 dimensions.

Time-of-day pricing for toll roads under traffic demand uncertainties: A distributionally robust simulation-based optimization method

Road pricing has received increasing attention for traffic congestion mitigation, but traffic demand uncertainties cause a profound impact on its effectiveness. This study proposes a link-based time-of-day pricing distributionally robust simulation-based optimization (DRSO) model to minimize the worst-case expected total travel time (TTT). This model aims to answer where and how much to charge under origin–destination (OD) traffic demand uncertainties that are described by an ambiguity set of probability distributions, and is solved by a DRSO method with two stages. The first stage searches an infill sample after building two stochastic kriging models (i.e., SKG-I and SKG-II) to map the relation between the traffic demand uncertainty parameter and the average TTT for each decision sample, and that between the decision variable and the worst-case average TTT, respectively. The second stage introduces the optimal computational budget allocation (OCBA) method to distribute more simulation replications to more promising combination samples, which helps to improve the prediction accuracy of SKGs in promising regions. The DRSO method is first tested with an M/M/1 queuing problem, and numerical results show its outperformance compared with one counterpart algorithm. After that, it is applied to address the link-based time-of-day pricing DRSO problem on Anaheim network and two solution algorithms are also added for comparison. Numerical results show that the robust toll plan solved by the DRSO method outperforms no toll plan and two other toll plans solved by the comparison algorithms in handling traffic dynamics and different levels of traffic demand uncertainties. In conclusion, the DRSO method is promising to address the costly simulation-based optimization (SO) problems under uncertainties characterized by an ambiguity set of probability distributions.

A Gaussian process-driven worst realistic imperfection method for cylindrical shells by limited data

In this study, we propose a Gaussian process-driven worst imperfection method to generate simulated geometric imperfections and find the lower limit of buckling capability. First, we utilize the nested stochastic Kriging to estimate the global trend and envelopes of the geometric imperfections. Second, the expansion optimal linear estimation and direct weighting construction method are used to expand limited data, which allow the simulated geometric imperfections to have the same characteristics as the initial data. Third, two magnitude-based constraints are provided, which can make the simulated geometric imperfections more consistent with the realistic data and can represent more design space. Finally, for the cylindrical shells, the data pre-processing method is further provided in this paper. Two numerical examples with assumed geometric imperfections are used to validate the proposed method, and the minimum buckling capacity of a cylindrical shell with 8 realistic geometric imperfections data is estimated. The results show that the proposed method can achieve the best performance in terms of accuracy, efficiency, and robustness.

Uncertainty Quantification and Optimal Robust Design for Machining Operations

Abstract In this study, we carry out robust optimal design for the machining operations, one key process in wafer polishing in chip manufacturing, aiming to avoid the peculiar regenerative chatter and maximize the material removal rate (MRR) considering the inherent material and process uncertainty. More specifically, we characterize the cutting tool dynamics using a delay differential equation (DDE) and enlist the temporal finite element method (TFEM) to derive its approximate solution and stability index given process settings or design variables. To further quantify the inherent uncertainty, replications of TFEM under different realizations of random uncontrollable variables are performed, which however incurs extra computational burden. To eschew the deployment of such a crude Monte Carlo (MC) approach at each design setting, we integrate the stochastic TFEM with a stochastic surrogate model, stochastic kriging, in an active learning framework to sequentially approximate the stability boundary. The numerical result suggests that the nominal stability boundary attained from this method is on par with that from the crude MC, but only demands a fraction of the computational overhead. To further ensure the robustness of process stability, we adopt another surrogate, the Gaussian process, to predict the variance of the stability index at unexplored design points and identify the robust stability boundary per the conditional value at risk (CVaR) criterion. Therefrom, an optimal design in the robust stable region that maximizes the MRR can be identified.

Gaussian Processes with Input Location Error and Applications to the Composite Parts Assembly Process

This paper investigates Gaussian process modeling with input location error, where the inputs are corrupted by noise. Here, the best linear unbiased predictor for two cases is considered, according to whether there is noise at the target location or not. We show that the mean squared prediction error converges to a nonzero constant if there is noise at the target location, and we provide an upper bound of the mean squared prediction error if there is no noise at the target location. We investigate the use of stochastic Kriging in the prediction of Gaussian processes with input location error and show that stochastic Kriging is a good approximation when the sample size is large. Several numerical examples are given to illustrate the results, and a case study on the assembly of composite parts is presented. Technical proofs are provided in the appendices.

Addressing the different sources of excitation variability in seismic response distribution estimation using kriging metamodeling

AbstractThis paper investigates the impact of the different sources of excitation variability within the stochastic kriging framework recently developed by the authors for the estimation of the distribution of engineering demand parameters (EDPs) in applications that the seismic hazard is described through stochastic ground motion models. For a given seismic event, described by seismicity characteristics such as the moment magnitude and the distance to source, one can distinguish two type of uncertainties in the excitation description: (g.i) the stochastic sequence utilized within the excitation model; (g.ii) the predictive models relating the ground motion parameters, such as significant duration, arias intensity, or frequency properties of seismic waves, to the seismicity characteristics. The original formulation of the authors treats (g.i) as aleatoric uncertainty and (g.ii) as parametric uncertainty, including the latter in the model parameters for the risk characterization, representing the metamodel input. Implementation establishes the EDP distribution approximation by utilizing a database of EDP estimates for a set of different input parameters, considering replications of these estimates for different stochastic sequences for some small part of this database. The database with replications is first leveraged to approximate the heteroscedastic behavior with respect to the aleatoric uncertainty, and the entire database is subsequently used, coupled with the previous heteroscedasticity approximation, to establish the stochastic kriging predictions for the EDP distribution. For excitation models with a large number of ground motion parameters, the original formulation leads to a significant increase of the input dimensionality for the metamodel development, requiring a larger database for facilitating accurate EDP approximations. Here, an alternative formulation is investigated, considering some (or perhaps even all) of the ground motion parameters to be part of the aleatoric uncertainty of the excitation description. It is shown that careful selection is needed for the exact ground motion parameters that can be treated as part of this aleatoric uncertainty, to accommodate an accurate overall EDP approximation. Similarities are discussed for applications that consider the description of the seismic hazard through the selection of ground motions based on intensity measures. An extensive validation of this new approach is presented considering two different stochastic ground motion models.

Simulation optimization using stochastic kriging with robust statistics

Metamodels are widely used as fast surrogates to facilitate the optimization of simulation models. Stochastic kriging (SK) is an effective metamodeling tool for a mean response surface implied by stochastic simulation. In SK, it is usually assumed that the experimental data are normally distributed and uncontaminated. However, these assumptions can be easily violated in many practical applications. This paper proposes a new type of SK for simulation models that may have non-Gaussian responses; this new SK uses robust estimators of location (or central tendency) and scale (or variability) that are well-known in the literature on robust statistics. Statistical properties of the robust estimators used in this paper are briefly analyzed and the performances of the proposed methods are compared through numerical examples of different features. The comparison results show that the proposed robust SK with the robust estimators is quite efficient, no matter whether the standard assumptions hold or not.

Statistical model calibration and design optimization under aleatory and epistemic uncertainty

Statistical model calibration is a framework for inference on unknown model parameters and modeling discrepancy between simulation and experiment through an inverse method in the presence of uncertainty. Most of the existing approaches cannot treat aleatory uncertainty of model parameters and model discrepancy simultaneously, and thus reliability analysis and design optimization using a calibrated simulation model accounting for uncertainty have been limitedly applied. Therefore, a statistical model calibration using stochastic Kriging accounting for the aleatory uncertainty of model parameters is proposed in this paper. The probability distributions of the model parameters and corresponding discrepancy are quantified and estimated through the maximum likelihood estimation (MLE). Since it may be difficult to secure sufficient experimental data for model calibration due to limited resources, the quantification of epistemic uncertainty on the calibrated model and propagation to reliability are also presented. Then, the design optimization accounting for aleatory and epistemic uncertainty, which is called confidence-based design optimization (CBDO), can yield the conservative optimum to prevent the unexpected failure. In conclusion, the proposed framework facilitates the statistical model calibration and design optimization under aleatory uncertainty of model parameters and model discrepancy and epistemic uncertainty caused by insufficient data.

Stochastic Bayesian optimization for predicting borderline knock

Engine knock is an undesirable combustion that could damage the engine mechanically. On the other hand, it is often desired to operate the engine close to its borderline knock limit to optimize combustion efficiency. Traditionally, borderline knock limit is detected by sweeping tests of related control parameters for the worst knock, which is expensive and time consuming, and also, the detected borderline knock limit is often used as a feedforward control without considering its stochastic characteristics without compensating current engine operational condition and type of fuel used. In this paper, stochastic Bayesian optimization method is used to obtain a tradeoff between stochastic knock intensity and fuel economy. The log-nominal distribution of knock intensity signal is converted to Gaussian one using a proposed map to satisfy the assumption for Kriging model development. Both deterministic and stochastic Kriging surrogate models are developed based on test data using the Bayesian iterative optimization process. This study focuses on optimizing two competing objectives, knock intensity and indicated specific fuel consumption using two control parameters: spark and intake valve timings. Test results at two different operation conditions show that the proposed learning algorithm not only reduces required time and cost for predicting knock borderline but also provides control parameters, based on trained surrogate models and the corresponding Pareto front, with the best fuel economy possible.