Stochastic Kriging Research Articles

Stochastic optimization of high-altitude airship envelopes based on kriging method

High-altitude airships can be used to transport substantial payloads to the stratosphere and remain there over long periods of time. In this paper, an algorithm for the design of high-altitude airship envelopes, accounting for uncertainties, is developed and applied. The algorithm is based on the non-intrusive polynomial chaos expansion scheme, which is employed to build a stochastic kriging metamodel. Two uncertainties are examined and characterized: 1) the stratospheric wind fluctuations using reanalysis datasets and 2) the variability in the turbulence levels. The method results are discussed to address the relevancy of the uncertainties. It is found that the drag coefficient of stratospheric envelopes can vary by as much as 30 percent. As a case of study, an ideal stratospheric airship is considered, operating at an altitude of 20 km, at a latitude of 30cN and carrying a payload of 250 kg. The baseline design follows the shape of the ZHIYUAN-1 envelope and the cost function to be minimized is the average mission drag coefficient. Due to the new method, a significant reduction (4%) of the average drag of the aircraft is achieved.

Kriging metamodeling for seismic response distribution estimation

AbstractThis paper revisits the implementation of surrogate modeling (metamodeling) techniques within seismic risk assessment, for applications that the seismic hazard is described through stochastic ground motion models. Emphasis is placed on how to efficiently address the aleatoric uncertainty in the ground motions, stemming in this case from the stochastic sequence utilized within the excitation model. Previous work has accommodated this uncertainty by approximating the statistics of the engineering demand parameters (EDPs), something that required a large number of replication simulations (for different stochastic sequences) for each training point that was used to inform the metamodel calibration. Using kriging (Gaussian Process regression) as surrogate model, an alternative formulation is discussed here, aiming to minimize the replications for each training point. This is achieved by approximating directly the EDP distribution. It is shown that accommodating heteroscedastic behavior with respect to the aleatoric uncertainty is absolutely critical for achieving an accurate approximation, and two different approaches are presented for establishing this objective. The first approach adopts a stochastic kriging formulation, utilizing a small number of replications for judicially selected inputs, leveraging a secondary surrogate model over the latter inputs to address the heteroscedastic behavior. The second approach uses no replications, establishing a heteroscedastic nugget formulation to accommodate the EDP distribution estimation. A functional relationship is introduced between the nugget and the excitation intensity features to approximate the heteroscedastic behavior. This relationship is explicitly optimized during the metamodel calibration.

Generalized polynomial chaos-informed efficient stochastic Kriging

Stochastic kriging (SK) offers an explicit way to characterize heterogeneous noise variance in stochastic computer simulations and has gained considerable traction recently as a surrogate model. Nevertheless, SK relies on tedious Monte Carlo (MC) method to estimate the intrinsic variance at each design input. For computationally expensive simulations, the substantial replication effort has essentially rendered SK intractable. To this end, we develop generalized polynomial chaos (gPC)-informed efficient stochastic kriging (gPC-SK) to ameliorate the computational cost. At its core, gPC supplants the tedious repetitive MC simulations, instead resting on a much smaller set of sampling points to estimate the intrinsic uncertainty, thus applicable to those prohibitively expensive simulations. We present the gPC-SK in sequential optimal design on the borehole function and stability of time-delay dynamic systems.

A novel Nested Stochastic Kriging model for response noise quantification and reliability analysis

Surrogate models and adaptive methods can release the huge computational burden of structural reliability analysis. However, it is very difficult to guarantee the accuracy of uncertainty quantification, especially when noises are contained in samples, which may greatly reduce the confidence of reliability analysis. In this study, we propose a novel Nested Stochastic Kriging (NSK) model method with response noise parameters decoupled from other surrogate model parameters, which can significantly improve the accuracy of uncertainty quantification and modeling efficiency. The proposed NSK is for deterministic data with response noise, aiming at reliability analysis. Various types of uncertainty can be identified by the NSK, including the conditions of known and unknown, constant and variable variances. Moreover, a local sample verification method is established to improve the accuracy of uncertainty quantification. To further improve the accuracy of reliability analysis, an adaptive NSK framework is established with new samples added, and then reliability analysis can be conducted. Through several numerical examples, it can be seen that the NSK always provides the most accurate results with the fewest analysis calls.

A model validation framework based on parameter calibration under aleatory and epistemic uncertainty

Model validation methods have been widely used in engineering design to evaluate the accuracy and reliability of simulation models with uncertain inputs. Most of the existing validation methods for aleatory and epistemic uncertainty are based on the Bayesian theorem, which needs a vast number of data to update the posterior distribution of the model parameter. However, when a single simulation is time-consuming, the required simulation cost for the validation of a simulation model may be unaffordable. To overcome this difficulty, a new model validation framework based on parameter calibration under aleatory and epistemic uncertainty is proposed. In the proposed method, a stochastic kriging model is constructed to predict the validity of the candidate simulation model under different uncertainty input parameters. Then, an optimization problem is defined to calibrate the epistemic uncertainty parameters to minimize the discrepancy between the simulation model and the experimental model. K–S test finally decides whether to accept or reject the calibrated simulation model. The performance of the proposed approach is illustrated through a cantilever beam example and a turbine blade validation problem. Results show that the proposed framework can identify the most appropriate parameters to calibrate the simulation model and provide a correct judgment about the validity of the candidate model, which is useful for the validation of simulation models in practical engineering design.

Statistical inference of the equivalent initial flaw size for assembled plate structures with the dual boundary element method

The statistical inference of the Equivalent Initial Flaw Size Distribution (EIFSD) is developed for the first time using the Dual Boundary Element Method (DBEM) for assembled shear deformable plate structures. As part of this inference procedure, Bayesian updating is employed to enable the continuous refinement of the EIFSD via data obtained by many simulated routine inspections of a stiffened panel from a fleet of aircraft. Fatigue crack growth is modelled using an incremental crack growth procedure that only requires modelling of the boundary of the 2.5D structure with line elements and requires no remeshing during crack growth simulations. Stochastic Kriging is employed to account for the stochastic nature of fatigue crack growth and to offset the high computational cost associated with modelling complex built-up structures. To demonstrate the efficiency of the proposed inference methodology, a numerical example featuring a stiffened panel subjected to complex loading in the form of combined tension and bending is presented. Once the EIFSD has been inferred, it can be used to optimise the intervals between routine aircraft inspections via the use of reliability analysis techniques as part of a combined reliability-EIFS approach. It is demonstrated that the proposed methodology offers the capability to reduce the costs associated with inspections.

A generalized hierarchical co-Kriging model for multi-fidelity data fusion

Multi-fidelity (MF) surrogate models have shown great potential in simulation-based design since they can make a trade-off between high prediction accuracy and low computational cost by augmenting the small number of expensive high-fidelity (HF) samples with a large number of cheap low-fidelity (LF) data. In this work, a generalized hierarchical co-Kriging (GCK) surrogate model is proposed for MF data fusion with both nested and non-nested sampling data. Specifically, a comprehensive Gaussian process (GP) Bayesian framework is developed by aggregating calibrated LF Kriging model and discrepancy stochastic Kriging model. The stochastic Kriging model enables the GCK model to consider the predictive uncertainty from the LF Kriging model at HF sampling points, making it possible to estimate the model parameter separately under both nested and non-nested sampling data. The performance of the GCK model is compared with three well-known Kriging-based MF surrogates, i.e., hybrid Kriging–scaling (HKS) model, KOH autoregressive (KOH) model, and hierarchical Kriging (HK) model, by testing them on two numerical examples and two real-life cases. The influence of correlations between LF and HF samples and the cost ratio between them are also analyzed. Comparison results on the illustrated cases demonstrate that the proposed GCK model shows great potential in MF modeling under non-nested sampling data, especially when the correlations between LF and HF samples are weak.

Hierarchical surrogate model with dimensionality reduction technique for high‐dimensional uncertainty propagation

SummaryIn this article, hierarchical surrogate model combined with dimensionality reduction technique is investigated for uncertainty propagation of high‐dimensional problems. In the proposed method, a low‐fidelity sparse polynomial chaos expansion model is first constructed to capture the global trend of model response and exploit a low‐dimensional active subspace (AS). Then a high‐fidelity (HF) stochastic Kriging model is built on the reduced space by mapping the original high‐dimensional input onto the identified AS. The effective dimensionality of the AS is estimated by maximum likelihood estimation technique. Finally, an accurate HF surrogate model is obtained for uncertainty propagation of high‐dimensional stochastic problems. The proposed method is validated by two challenging high‐dimensional stochastic examples, and the results demonstrate that our method is effective for high‐dimensional uncertainty propagation.

A multiobjective stochastic simulation optimization algorithm

The use of kriging metamodels in simulation optimization has become increasingly popular during recent years. The majority of the algorithms so far uses the ordinary (deterministic) kriging approach for constructing the metamodel, assuming that solutions have been sampled with infinite precision. This is a major issue when the simulation problem is stochastic: ignoring the noise in the outcomes may not only lead to an inaccurate metamodel, but also to potential errors in identifying the optimal points among those sampled. Moreover, most algorithms so far have focused on single-objective problems. In this article, we test the performance of a multiobjective simulation optimization algorithm that contains two crucial elements: the search phase implements stochastic kriging to account for the inherent noise in the outputs when constructing the metamodel, and the accuracy phase uses a well-known multiobjective ranking and selection procedure in view of maximizing the probability of selecting the true Pareto-optimal points by allocating extra replications on competitive designs. We evaluate the impact of these elements on the search and identification effectiveness, for a set of test functions with different Pareto front geometries, and varying levels of heterogeneous noise. Our results show that the use of stochastic kriging is essential in improving the search efficiency; yet, the allocation procedure appears to lose effectiveness in settings with high noise. This emphasizes the need for further research on multiobjective ranking and selection methods.

Managing component degradation in series systems for balancing degradation through reallocation and maintenance

In a physical system, components are usually installed in fixed positions that are known as operating slots. Due to such reasons as user behavior and imbalanced workload, a component’s degradation can be affected by the corresponding installation position in the system. As a result, components degradation levels can be significantly different even when the components come from a homogeneous population. Dynamic reallocation of the components among the installation positions is a feasible way to balance the extent of the degradation, and hence, extend the time from system installation to its replacement. In this study, we quantify the benefit of incorporating reallocation into the condition-based maintenance framework for series systems. The degradation of components in the system is modeled as a multivariate Wiener process, where the correlation between the degradation is considered. Under the periodic inspection framework, the optimal control limits for reallocation and preventive replacement are investigated. We first propose a reallocation policy of two-component systems, where the degradation process with reallocation and replacement is formulated as a semi-regenerative process. Then the long-run average operational cost is computed based on the stationary distribution of its embedded Markov chain. We then generalize the model to general series systems and use Monte Carlo simulations to approximate the maintenance cost. The optimal thresholds for reallocation and replacement are obtained from a stochastic response surface method using a stochastic kriging model. We further generalize the model to the scenario of an unknown degradation rate associated with each slot. The proposed model is applied to the tire system of a car and the battery system of hybrid-electric vehicles, where we show that the reallocation policy is capable of significantly reducing the system’s long-run average operational cost.

Optimization of Energy Harvesting From Stall-Induced Oscillations Using the Multidimensional Kriging Metamodel

The power harvested from stall-induced oscillations of airfoils has been analyzed as a potential source of electric energy for microsystems. Previous works have indicated that the energy harvested from such oscillations is affected by key parameters of the structural configuration. In this sense, this work proposes the optimization of such parameters by considering the use of a stochastic multidimensional Kriging metamodel. The metamodel was built using a database created with simulations of an electro-aeroelastic model. Such model considers aerodynamics loads given by the Beddoes–Leishman model as input for the system of differential equations which governs the pitching motion of an airfoil attached to an electric generator. The results of the optimization process have indicated an optimum point for the elastic axis of the structure and the need for reducing the mass, the moment of inertia, and the stiffness for increasing the harvested power in a range of wind speeds.

A Bayesian Stochastic Kriging Optimization Model Dealing with Heteroscedastic Simulation Noise for Freeway Traffic Management

Since advanced traveler information and traffic management systems have become popular, it is vital to capture the joint impact of various strategies on transportation systems. Developing analytical models to incorporate traffic dynamics and travel behavior is challenging. Based on observation of the heteroscedasticity of the simulation noise in the stochastic simulator, this paper develops the Bayesian stochastic Kriging (BSK) model to adapt for the heteroscedastic noise and metamodel parameter uncertainty in a Bayesian framework, which enhances the existing surrogate-based optimization methods. This paper presents a metamodel for large scale simulation-based freeway traffic management optimization problems. Simulation-based optimization combines the advantages of simulation models and mathematical optimization methods. The parameter estimation of the BSK model is accomplished by the Bayesian inference. The proposed methodology enables the efficient use of large scale high-resolution traffic simulation models for simulation-based optimization, while accounting for travelers’ behavioral responses to information provision. We demonstrate the advantages of BSK compared to other existing metamodels using a numerical example of a synthetic network and a mathematical example. In a work zone scenario on a real-world freeway/arterial corridor of I-270 and MD-355 in the State of Maryland, the BSK model is applied to the freeway traffic management via optimizing the high-occupancy/toll rate and deploying dynamic message signs. Field traffic measurements by loop/microwave detections are used to calibrate travel demand and to supply the simulation parameters. The optimization results are promising in reducing the corridor-wide travel delay and enhancing the vehicle throughput. The online appendix is available at https://doi.org/10.1287/trsc.2018.0819 .

Monte Carlo integration with adaptive variance selection for improved stochastic efficient global optimization

In this paper, the minimization of computational cost on evaluating multi-dimensional integrals is explored. More specifically, a method based on an adaptive scheme for error variance selection in Monte Carlo integration (MCI) is presented. It uses a stochastic Efficient Global Optimization (sEGO) framework to guide the optimization search. The MCI is employed to approximate the integrals, because it provides the variance of the error in the integration. In the proposed approach, the variance of the integration error is included into a Stochastic Kriging framework by setting a target variance in the MCI. We show that the variance of the error of the MCI may be controlled by the designer and that its value strongly influences the computational cost and the exploration ability of the optimization process. Hence, we propose an adaptive scheme for automatic selection of the target variance during the sEGO search. The robustness and efficiency of the proposed adaptive approach were evaluated on global optimization stochastic benchmark functions as well as on a tuned mass damper design problem. The results showed that the proposed adaptive approach consistently outperformed the constant approach and a multi-start optimization method. Moreover, the use of MCI enabled the method application in problems with high number of stochastic dimensions. On the other hand, the main limitation of the method is inherited from sEGO coupled with the Kriging metamodel: the efficiency of the approach is reduced when the number of design variables increases.

Controlling Sources of Inaccuracy in Stochastic Kriging

ABSTRACTScientists and engineers commonly use simulation models to study real systems for which actual experimentation is costly, difficult, or impossible. Many simulations are stochastic in the sense that repeated runs with the same input configuration will result in different outputs. For expensive or time-consuming simulations, stochastic kriging is commonly used to generate predictions for simulation model outputs subject to uncertainty due to both function approximation and stochastic variation. Here, we develop and justify a few guidelines for experimental design, which ensure accuracy of stochastic kriging emulators. We decompose error in stochastic kriging predictions into nominal, numeric, parameter estimation, and parameter estimation numeric components and provide means to control each in terms of properties of the underlying experimental design. The design properties implied for each source of error are weakly conflicting and broad principles are proposed. In brief, space-filling properties, “small fill distance” and “large separation distance,” should be balanced with replication at distinct input configurations, with number of replications depending on the relative magnitudes of stochastic and process variability. Nonstationarity implies higher input density in more active regions, while regression functions imply a balance with traditional design properties. A few examples are presented to illustrate the results. Supplementary materials providing proofs of the theoretical results and code for comparisons are available online.

Constrained optimization of black-box stochastic systems using a novel feasibility enhanced Kriging-based method

Stochastically constrained simulation optimization problems are challenging because the inherent noise terms to a black-box system lead to the need of considering the uncertainty at both the objective and the constraint function. To address this problem, we propose a Kriging-based optimization framework, which uses stochastic Kriging models to approximate the objective and the constraint functions and an adaptive sampling approach to sequentially search for the next point that is promising to have a better objective. The main contribution of this work is to incorporate a feasibility-enhanced term to the infill sampling criterion, which improves the Kriging-based algorithm's capabilities of returning a truly feasible near-optimal solution for stochastic systems. The efficacy of the Kriging-based algorithm is demonstrated with eight benchmark problems and a case study from the pharmaceutical manufacturing domain.

Accurate Prediction of the Weld Bead Characteristic in Laser Keyhole Welding Based on the Stochastic Kriging Model

As an important index of weld quality, the weld bead geometry is closely related to the welding process parameters (WPP). Therefore, it is crucial to establish the relationships between the WPP and weld bead shape to serve as an indicator of the weld quality. However, it is difficult to predict the weld bead shape accurately due to uncertainty. In this paper, laser keyhole welding (LKW) experiments are conducted on 2205 stainless steel at sample points generated by the optimal Latin hypercube sampling (OLHS). Then the relationships between WPP and weld width (WW) are constructed using stochastic kriging model (SKM), considering the randomness of the welding process. To verify the effectiveness of the SKM, two validation approaches, the additional experiments validation and k-fold cross-validation, are used to compare the prediction performance of SKM and the traditional kriging model. SKM is superior to the kriging model at the whole five additional test points with smaller relative error. As to k-fold cross-validation, SKM provides a smaller root mean square error at four in five groups of the data. In addition, SKM can provide the variations of the entire weld bead shape. Overall, the SKM is very prominent in predicting the weld bead shape, considering fluctuations of WPP.

Enhancing stochastic kriging for queueing simulation with stylized models

ABSTRACTStochastic kriging is a popular metamodeling technique to approximate computationally expensive simulation models. However, it typically treats the simulation model as a black box in practice and often fails to capture the highly nonlinear response surfaces that arise from queueing simulations. We propose a simple, effective approach to improve the performance of stochastic kriging by incorporating stylized queueing models that contain useful information about the shape of the response surface. We provide several statistical tools to measure the usefulness of the incorporated stylized models. We show that even a relatively crude stylized model can substantially improve the prediction accuracy of stochastic kriging.

Some Monotonicity Results for Stochastic Kriging Metamodels in Sequential Settings

Stochastic kriging (SK) and stochastic kriging with gradient estimators (SKG) are useful methods for effectively approximating the response surface of a simulation model. In this paper, we show that in a fully sequential setting when all model parameters are known, the mean squared errors of the optimal SK and SKG predictors are monotonically decreasing as the number of design points increases. In addition, we prove, under appropriate conditions, that the use of gradient information in the SKG framework generally improves the prediction performance of SK. Motivated by these findings, we propose a sequential procedure for adaptively choosing design points and simulation replications in obtaining SK (SKG) predictors with desired levels of fidelity. We justify the validity of the procedure and carry out numerical experiments to illustrate its performance. The online supplement is available at https://doi.org/10.1287/ijoc.2017.0779.

An adaptive two-stage dual metamodeling approach for stochastic simulation experiments

ABSTRACTIn this article we propose an adaptive two-stage dual metamodeling approach for stochastic simulation experiments, aiming at exploiting the benefits of fitting the mean and variance function models simultaneously to improve the predictive performance of Stochastic Kriging (SK). To this end, we study the effects of replacing the sample variances with smoothed variance estimates on the predictive performance of SK, articulate the links between SK and least-squares support vector regression, and provide some useful data-driven methods for identifying important design points. We argue that efficient data-driven experimental designs for stochastic simulation metamodeling can be “learned” through a “dense and shallow” initial design (i.e., relatively many design points with relatively little effort at each), and efficient budget allocation rules can be seamlessly incorporated into the proposed approach to intelligently spend the remaining simulation budget on the important design points identified. Two numerical examples are provided to demonstrate the promise held by the proposed approach in providing highly accurate mean response surface approximations.

Efficient global optimisation for black-box simulation via sequential intrinsic Kriging

Efficient global optimisation (EGO) is a popular method that searches sequentially for the global optimum of a simulated system. EGO treats the simulation model as a black-box, and balances local and global searches. In deterministic simulation, classic EGO uses ordinary Kriging (OK), which is a special case of universal Kriging (UK). In our EGO variant we use intrinsic Kriging (IK), which does not need to estimate the parameters that quantify the trend in UK. In random simulation, classic EGO uses stochastic Kriging (SK), but we replace SK by stochastic IK (SIK). Moreover, in random simulation, EGO needs to select the number of replications per simulated input combination, accounting for the heteroscedastic variances of the simulation outputs. A popular method uses optimal computer budget allocation (OCBA), which allocates the available total number of replications to simulated combinations. We replace OCBA by a new allocation algorithm. We perform several numerical experiments with deterministic simulations and random simulations. These experiments suggest that (1) in deterministic simulations, EGO with IK outperforms classic EGO; (2) in random simulations, EGO with SIK and our allocation rule does not perform significantly better than EGO with SK and OCBA.