Calculation of confidence intervals for simulation output

Abstract

This article is concerned with the calculation of confidence intervals for simulation output that is dependent on two sources of variability. One, referred to as <i>simulation variability</i>, arises from the use of random numbers in the simulation itself; and the other, referred to as <i>parameter variability</i>, arises when the input parameters are unknown and have to be estimated from observed data. Three approaches to the calculation of confidence intervals are presented--the traditional asymptotic normality theory approach, a bootstrap approach and a new method which produces a conservative approximation based on performing just two simulation runs at carefully selected parameter settings. It is demonstrated that the traditional and bootstrap approaches provide similar degrees of accuracy and that whilst the new method may sometimes be very conservative, it can be calculated in a small fraction of the computational time of the exact methods.