Analysis of Deterministic Simulation Models Using Methods Applicable to Two‐Way Treatment Structures without Replication

Abstract

Listen

Translate article

AbstractSeparate executions of deterministic simulation models are not independent replications in the strictest sense and, therefore, it is difficult to statistically analyze model output, since estimates of the error variance for the model are difficult to obtain. However, an analysis of simulation output is desirable. Presented here is a method that permits the statistical analysis of deterministic simulation output or its comparison to other output or experimental data. These analyses are possible when conditions imposed on the computer‐generated experiment conform to a two‐way treatment structure. The response data can be comprised of (i) model output, (ii) differences between experimental means and the model, or (iii) differences between individual output across treatment levels. The additive main effects and multiplicative interaction (AMMI) model for unreplicated data supplies several statistical tests. Foremost, the characteristic root test indicates if interaction exists between the two treatments. If interaction exists, the AMMI model can be used to determine which treatment combinations are involved. Once sources of interaction have been identified, it is then possible to obtain a good estimate of the error variance, even though these data are unreplicated. Using this estimate of the error variance, it is possible to test hypotheses of interest on model performance. An AMMI analysis comparing the output of a deterministic soil hydrology model to (replicated) experimental data is presented.