Abstract
AbstractStochastic simulation models requiring many input parameters are widely used to inform the management of ecological systems. The interpretation of complex models is aided by global sensitivity analysis, using simulations for distinct parameter sets sampled from multidimensional space. Ecologists typically analyze such output using an “emulator”; that is, a statistical model used to approximate the relationship between parameter inputs and simulation outputs and to derive sensitivity measures. However, it is typical for ad hoc decisions to be made regarding: (1) trading off the number of parameter samples against the number of simulation iterations run per sample, (2) determining whether parameter sampling is sufficient, and (3) selecting an appropriate emulator. To evaluate these choices, we coupled different sensitivity‐analysis designs and emulators for a stochastic, 20‐parameter model that simulated the re‐introduction of a threatened species subject to predation and disease, and then validated the emulators against new output generated from the simulation model. Our results lead to the following sensitivity analysis‐protocol for stochastic ecological models. (1) Run a single simulation iteration per parameter sample generated, even if the focal response is a probabilistic outcome, while sampling extensively across the parameter space. In contrast to designs that invested in many model iterations (tens to thousands) per parameter sample, this approach allowed emulators to capture the input‐output relationship of the simulation model more accurately and also to produce sensitivity measures that were robust to variation inherent in the parameter‐sampling stage. (2) Confirm that parameter sampling is sufficient, by emulating subsamples of the sensitivity‐analysis output. As the subsample size is increased, the cross‐validatory performance of the emulator and sensitivity measures derived from it should exhibit asymptotic behavior. This approach can also be used to compare candidate emulators and select an appropriate interaction depth. (3) If required, conduct further simulations for additional parameter samples, and then report sensitivity measures and illustrate key response curves using the selected emulator. This protocol will generate robust sensitivity measures and facilitate the interpretation of complex ecological models, while minimizing simulation effort.
Highlights
Following an exponential rise in the availability of cheap computing power, simulation models are increasingly used to study ecological systems that are too difficult to investigate empirically or experimentally, or as planning tools in adaptive management (Green et al 2005, Hastings et al 2005, Valle et al 2009)
In contrast to designs that invested in many model iterations per parameter sample, this approach allowed emulators to capture the input-o utput relationship of the simulation model more accurately and to produce sensitivity measures that were robust to variation inherent in the parameter-s ampling stage
We introduce a standardized method for choosing an appropriate emulator and determining when parameter sampling is sufficient for that emulator to capture the input–output relationship of the simulation model
Summary
Following an exponential rise in the availability of cheap computing power, simulation models are increasingly used to study ecological systems that are too difficult to investigate empirically or experimentally, or as planning tools in adaptive management (Green et al 2005, Hastings et al 2005, Valle et al 2009). A simple population viability analysis to estimate extinction risk might only require information about the survival and fertility rates of a focal species that are readily derived from empirical studies (Boyce 1992, Brook et al 2000). Parameters governing the strength or functional form of ecological processes or interactions (e.g., density feedbacks, predator-p rey or disease-host dynamics) are typically more difficult to estimate accurately. Modelers are faced with a trade-off between the need to simulate important ecological processes adequately, which might necessitate many parameters, and the need to construct interpretable and computationally tractable model systems (Levins 1966, Ginzburg and Jensen 2004). Sensitivity analysis is the primary tool used to determine whether simulation models produce outputs that are robust to parameter uncertainty. A “global” sensitivity analysis varies all parameters simultaneously and can provide robust sensitivity measures in the presence of nonlinear responses and interactions among parameters (Drechsler 1998, Sobol 2001, Wainwright et al 2014)
Sign-in/Register to view highlights & summary Sign-in/Register to access full text options 
Free) 
Talk to us
Join us for a 30 min session where you can share your feedback and ask us any queries you have
Schedule a call
