Analyzing transformation-based simulation metamodels

Abstract

To analyze a simulation (response surface) metamodel that involves a variance-stabilizing transformation of the original simulation-generated response, we present two techniques. In the first technique we compute an approximate percentile-type confidence interval for the mean of the original response at a selected factor-level combination (design point) as follows: we compute the usual confidence interval for the mean of the transformed response at that design point; and then we untransform the corresponding endpoints to obtain the desired confidence interval for the untransformed metamodel. In the second technique we compute the Maximum Likelihood Estimator (MLE) for the mean of the untransformed response based on standard distributional properties of the transformed metamodel; then using the delta method to approximate the MLE's variance, we construct for the untransformed metamodel an asymptotically exact confidence interval centered on the MLE. We illustrate these techniques in a case study on manufacturing cell design, comparing them with a more conventional approach for analyzing transformed-based simulation metamodels. A Monte Carlo performance evaluation shows that significantly better confidence-interval coverage is maintained with the second proposed technique (called the "MLE-delta method") over a wide range of values for the residual variance of the transformed metamodel.