Experimental design principles to choose the number of Monte Carlo replicates for stochastic ecological models

Abstract

Ecologists often rely on computer models as virtual laboratories to evaluate alternative theories, make predictions, perform scenario analysis, and to aid in decision-making. The application of ecological models can have real-world consequences that drive ecological theory development and science-based decision and policy-making, so it is imperative that the conclusions drawn from ecological models have a strong, credible quantitative basis. In particular it is important to establish whether any predicted change in a model output has ecological and statistical significance. Ecological models may include stochastic components, using probability distributions to represent some modeled processes. An individual run of a stochastic ecological model is a random draw from an infinitely large population, requiring replicate simulations to estimate the distribution of model outcomes. An important consideration is the number of Monte Carlo replicates necessary to draw useful conclusions from the model analysis. A simple framework is presented that borrows from well-understood techniques for experimental design, including confidence interval estimation and sample size power analysis. The desired precision of interval estimates for model prediction, or the minimum desired detectable effect size between scenarios, is established by the researcher in the context of the model objectives and the ecological system. The number of replicates required to achieve that level of precision or detectable effect is computed given an estimate of the variability in the model outcomes of interest. If the number of replicates is computationally prohibitive, then the expected precision or detectable effect for that sample size should be reported. An example is given for a stochastic model of fire spread integrated with an eco-hydrological model.