Enhanced Simulation Inference Using Bootstraps of Historical Inputs

Abstract

Simulation input analysis can present a dilemma: either use one historical trace that is realistic but captures only a small portion of the possible input variability, or use many unrealistic inputs generated by simplified parametric models of the trace. We demonstrate that a nonparametric time series bootstrap can ease the dilemma by converting a single trace into many realistic input scenarios. Bootstrapping inputs reduces by about one-third the inaccuracy in estimated standard errors for mean delay, standard deviation of delay, and probability of long delay in a G/M/1 queue. Such improvements can be achieved without an increase in the number of simulation runs by reducing the number of non-trace input replications to offset the increased number of inputs created by bootstrapping the trace. Bootstrapping can also be applied in problems lacking other means of estimating uncertainty in system performance. We show an example in system reliability where reasonable estimates are available over a wide range of bootstrap block sizes. We note that bootstrap estimates are always conditional on the traces from which they derive and thereby tend to underestimate the true level of uncertainty. Nevertheless, bootstrapping inputs offers a new way to enhance statistical inference in simulation experiments.