Metamodel Estimation Research Articles

Queueing inspired feature engineering to improve and simplify patient flow simulation metamodels

ABSTRACT We explore the interplay between domain-informed feature engineering, model performance, and model interpretability. This is a hybrid modelling and simulation study that merges the application of discrete-event simulation with alternative metamodelling techniques for modelling patient flow in health care. We consider two cases: a tandem queueing system of obstetric hospital units and a transient analysis of an outpatient clinic in which a finite number of scheduled patients arrive for care. We use several metamodels including various types of linear models, random forests, and neural networks. We evaluate the performance improvement of metamodel estimation when empowered with supplementary queueing theory knowledge. We consider three knowledge levels: no knowledge (no queueing-inspired features), basic (simple queueing features), and advanced (sophisticated queueing approximations). Our results show that queueing-related inputs improve the accuracy for the metamodels, independent of the model type. Moreover, queueing-related inputs improve model explainability and can lead to more parsimonious models. This has positive practical implications for implementing these types of models in actual health-care analytic projects.

Frequency optimization of nozzle guide vane in the low-pressure turbine system

The aim of the article is to present the approach for modal optimisation of nozzle guide vane in the low-pressure turbine system of aircraft engine. Described analysis assumption related to the finite element model definition, meta-model estimation and optimisation process itself. Goal function has been determined basing on Campbell diagram. Typical approach to solve modal analysis for mentioned structure is full three-dimensional model that is time consuming. For optimisation purposes employed two dimensional one in order to make the process more robust (element types with support of Fourier series). This approach is reducing degree of freedom quantity and is still capable to consider non-axisymmetric structure behaviour. The article provides relationship between geometrical features of nozzle and frequency change.

Construction of stochastic simulation metamodels with segmented polynomials

Metamodels are an important tool in simulation analysis as they can provide insight about the behavior of the simulation response. Modeling the response with low-degree polynomial segments allows the identification of different behavior zones and the parameters still have relation with the physical world. The purpose of this paper is to extend the use of segmented polynomial functions for simulation metamodeling, where the segments have at most identical value and slope at the breaks. Our approach is to build segmented polynomials metamodels where the hypothesis of degree and continuity of splines are less exigent, allowing more flexibility of the approximation. When breaks are known, constrained least squares are used for metamodel estimation, taking into account the linear formulation of the problem. If breaks have to be estimated, the unconstrained nonlinear regression theory is used, when it can be applied. Otherwise, the estimation is performed using an iterative algorithm which is applied repeatedly in a cyclic manner for estimating the breaks, and jackknifing yields the confidence intervals.

Maximum probable life time analysis under the required time-dependent failure probability constraint and its meta-model estimation

Time-dependent reliability (failure probability) aims at measuring the probability of the normal (abnormal) operation for structure/mechanism within the given time interval. To analyze the maximum probable life time under a required time-dependent failure probability (TDFP) constraint, an inverse process corresponding to the time-dependent reliability is proposed by taking the randomness of the input variables into consideration. The proposed inverse process employs the monotonicity between the TDFP and the upper boundary of the given time interval which reflects the life time, and an adaptive single-loop sampling meta-model for the time-dependent limit state function is presented to estimate the TDFP at the given time interval flexibly. Since the TDFP is generally monotonic to the upper boundary of the given time interval, thus by adjusting the probable upper and lower boundaries of the time interval in which the corresponding TDFPs include the required TDFP constraint, the proposed approach can always search the maximum probable life time at the required TDFP by the dichotomy. By introducing the time variable as an input which is the same level as the input random variables and constructing the adaptive single-loop sampling meta-model for the time-dependent limit state function in a longer time interval with the TDFP bigger than the required TDFP, the TDFP in any subintervals of the time interval involved in the constructed meta-model can be estimated as a byproduct of the constructed meta-model without any additional actual limit state evaluations. Then the efficiency for analyzing the maximum probable life time is improved by the dichotomy and the unified meta-model of the time-dependent limit state function. Two examples are employed to illustrate the accuracy and the efficiency of the proposed approach.

Switching regression metamodels in stochastic simulation

Simulation models are frequently analyzed through a linear regression model that relates the input/output data behavior. However, in several situations, it happens that different data subsets may resemble different models. The purpose of this paper is to present a procedure for constructing switching regression metamodels in stochastic simulation, and to exemplify the practical use of statistical techniques of switching regression in the analysis of simulation results. The metamodel estimation is made using a mixture weighted least squares and the maximum likelihood method. The consistency and the asymptotic normality of the maximum likelihood estimator are establish. The proposed methods are applied in the construction of a switching regression metamodel. This paper gives special emphasis on the usefulness of constructing switching metamodels in simulation analysis.

Gaussian Process Metamodels for Sensitivity Analysis of Traffic Simulation Models

This study adopted a metamodel-based technique for model sensitivity analysis and applied it to the AIMSUN mesoscopic model. The application of sensitivity analysis is crucial for the true comprehension and correct use of the traffic simulation model, although the main obstacle to an extensive use of the most sophisticated techniques is the high number of model runs such techniques usually require. For this reason, the possibility of performing a sensitivity analysis was tested not on a model but on its metamodel approximation. Important issues concerning metamodel estimation were investigated and commented on in the specific application to the AIMSUN model. Among these issues are the importance of selecting a proper sampling strategy based on low-discrepancy random number sequences and the importance of selecting a class of metamodels able to reproduce the inputs–outputs relationship in a robust and reliable way. Sobol sequences and Gaussian process metamodels were recognized as the appropriate choices. The proposed methodology was assessed by comparing the results of the application of variance-based sensitivity analysis techniques with the simulation model and with a metamodel estimated with 512 model runs for a variety of traffic scenarios and model outputs. Results confirmed the power of the proposed methodology and also made a more extensive application of sensitivity analysis techniques available for complex traffic simulation models.

A pseudo-random number assignment strategy for the estimation of quadratic metamodels

Whereas an optimal Pseudo-Random Number (PRN) assignment strategy for simulation experiments involving the estimation of linear metamodels currently exists, no such optimal assignment strategy for quadratic metamodels has been proposed. This situation is now rectified by the introduction of a PRN assignment strategy for a quadratic metamodel for 3 k factorial designs. In addition to extending the theory from linear to quadratic metamodels, the proposed PRN strategy is shown to be superior to a number of existing and competing strategies in terms of various variance measures.

A metamodeling methodology involving both qualitative and quantitative input factors

This paper suggests a methodology for developing a simulation metamodel involving both quantitative and qualitative factors. The methodology mainly deals with various strategic issues involved in metamodel estimation, analysis, comparison, and validation. To illustrate how to apply the methodology, a regression metamodel is developed for a client–server computer system. In particular, we studied how the response time is affected by the quantum interval, the buffer size, and the total number of terminals when different queuing disciplines are employed in the operation of the round-robin queue with a limited buffer. The results of the study indicate that the relationship between the response time and the quantitative factors cannot be adequately described by a single metamodel for all queuing disciplines.

Small response surface designs for metamodel estimation

Abstract The primary objective of this study is to provide the novice researchers in simulation metamodeling with guidance on how to use small designs for metamodel estimation especially when cost effectiveness is a concern. This study was carried out in three phases: First, a group of second-order small designs were evaluated with respect to various criteria. Next, the metamodel of a time-shared computer system was estimated using these designs. Finally, the predictive capabilities of these small designs in giving the best metamodel fit were investigated, and also, the performance of small designs was compared with two large size standard designs. Results indicate that the metamodel estimated using the hybrid design has the best predictive capability among the small designs, and its performance competes with that of the standard designs studied.

Simulation metamodel estimation using a combined correlation-based variance reduction technique for first and higher-order metamodels

This paper develops a procedure for jointly applying the correlation-based variance reduction techniques of common random numbers and control variates in a simulation experiment that is designed to estimate a linear metamodel (that is, linear in the regression model parameters) for a single response variable expressed in terms of an input vector of design variables for the target system. This procedure combines (a) the method of common random numbers for metamodel estimation and (b) a metamodel estimation scheme based on the method of control variates. Under specified conditions on the dependency structure of the simulation outputs and with respect to a variety of optimality criteria, the combined procedure is shown to be superior to each of the following conventional correlation-based variance reduction techniques: independent random number streams, common random number streams, and control variates.

Simulation Designs for the Estimation of Quadratic Response Surface Gradients in the Presence of Model Misspecification

This article considers the construction of simulation designs for the ordinary least squares estimation of second-order metamodels. Two premises underlie the development of these experimental strategies. First it is assumed that the postulated metamodel may be misspecified due to the true model structure being of third-order. It is therefore important that the locations of the simulation experiments be specified to provide protection against bias, as well as variance, in the estimation of metamodel parameters. The second premise is based on the observation that, in many applications of metamodels, functions of the fitted model coefficients (such as the slope gradients) are of greater interest than the response function. The integrated mean squared error of slopes design criterion that is implemented here addresses both premises. This criterion finds application in various optimum seeking methods and sensitivity analysis procedures. Combinations of four important classes of response surface designs and three pseudorandom number assignment strategies constitute the basis structure of the simulation designs studied. The performance of these simulation designs is evaluated and, subsequently, compared to a similar set of experimental plans that have as their focus the estimation of the response function.

ESTIMATING SIMULATION METAMODELS USING COMBINED CORRELATION-BASED VARIANCE REDUCTION TECHNIQUES

This paper develops a procedure for jointly applying all of the correlation-based variance reduction techniques (namely, die methods of antithetic variates, common random numbers, and control variates) in a simulation experiment that is designed to estimate a linear metamodel (that is, a linear regression model) for a single response variable expressed in terms of an input vector of design variables for the target system. This procedure combines (a) the Schruben-Margolin strategy for metamodel estimation based on joint application of the methods of common random numbers and antithetic variates, and (b) a metamodel estimation scheme based on the method of control variates. Under specified conditions on the dependency structure of die simulation outputs and with respect to a variety of optimally criteria, the combined procedure is shown to be superior to each of the following conventional correlation-based variance reduction techniques: independent random number streams, common random number streams, contro...

Combined correlation methods for metamodel estimation in multipopulation simulation experiments

In this paper we examine two variance-reduction strategies for estimating the parameters of a simulation metamodel that are based on the reduction of the variances of the mean of the responses across the design points that comprise the simulation experiment through the effective combination of well known variance-reduction techniques. In the first strategy, Combined Method I, we extend a combined approach of antithetic variates and control variates that we have developed in an earlier paper for a single design point simulation experiment. In Combined Method I, we perform h independent paris of simulation runs at each design point; and on the second run of each such pair, we use random number streams to drive the non-control variate components that are antithetic to the corresponding streams of the first run of the pair. The random number streams that drive the control variate components are randomly selected across all simulation runs. In Combined Method II, we extend Combined Method I to incorporate the ...