Design and Analysis of Simulation Experiments: Tutorial

Abstract

This tutorial reviews the design and analysis of simulation experiments. These experiments may have various goals: validation , prediction, sensitivity analysis, optimization (possibly robust), and risk or uncertainty analysis . These goals may be realized through metamodels. Two types of metamodels are the focus of this tutorial: (i) low-order polynomial regression , and (ii) Kriging (or Gaussian processes). The type of metamodel guides the design of the experiment; this design fixes the input combinations of the simulation model. However, before a regression or Kriging metamodel is applied, the many inputs of the underlying realistic simulation model should be screened; the tutorial focuses on sequential bifurcation . Optimization of the simulated system may use either a sequence of low-order polynomials—known as response surface methodology—or Kriging models fitted through sequential designs. Finally, “robust” optimization should account for uncertainty in simulation inputs. The tutorial includes references to earlier Winter Simulation Conference papers.