Combining regression diagnostics with simulation metamodels

Abstract

In a 1983 article in this journal Kleijnen proposed a lack-of-fit cross-validation test for validating regression metamodels in computer simulation. His procedure utilizes additional information frequently available in simulation to obtain independent estimates of the response variance at each design point. Kleijnen notes that one drawback of his strategy is the need to run a large number of regressions. We demonstrate that the statistic upon which this inference procedure is based is analogous to the R-Student statistic commonly used in regression diagnostics and show that only a single regression need be computed to implement the test. Kleijnen's procedure is then extended to include weighted least squares, and again, major computational simplifications are identified. For both ordinary least squares and weighted least squares, extensions of modern regression diagnostic tools also are developed to supplement traditional inferential model building strategies.