Calibration, Validation, and Prediction in Random Simulation Models

Abstract

Model calibration and validation are important processes in the development of stochastic computer models of real complex systems. This article introduces an integrated approach for model calibration, validation, and prediction based on Gaussian process metamodels and a Bayesian approach. Within this integrated approach, a sequential approach is further proposed for stochastic computer model calibration. Several design criteria for this sequential stage are proposed and studied, including an entropy-based criterion and one based on minimizing prediction error. To further use the data resources to improve the performance of both calibration and prediction, an adaptive procedure that combines these criteria is introduced to balance the resource allocation between the calibration and prediction. The accuracy and efficiency of the proposed sequential calibration approach and the integrated approach are illustrated with several numerical examples.