Five-stage procedure for the evaluation of simulation models through statistical techniques

Abstract

This paper recommends the following sequence for the evaluation of simulation models. 1) Validation: the availability of data on the real system determines the proper type of statistical technique. 2) Screening: in the simulation's pilot phase the important inputs are identified through a novel technique, namely sequential bifurcation, which uses aggregation and sequential experimentation. 3) Sensitivity or what-if analysis: the important inputs are analyzed in more detail, including interactions between inputs; relevant techniques are design of experiments (DOE) and regression analysis. 4) Uncertainty or risk analysis: important environmental inputs may have values not precisely known, so the resulting uncertainties in the model outputs are quantified; techniques are Monte Carlo and Latin hypercube sampling. 5) Optimization: policy variables may be controlled, applying Response Surface Methodology (RSM), which combines DOE, regression analysis, and steepest-ascent hill-climbing. This paper summarizes case studies for each stage.