Is mean squared error a consistent indicator of accuracy for spatially structured demographic models?

Abstract

Animal demographic models are used in many disciplines and it is well-understood that their results and interpretation depend greatly on their structure and complexity. However, when models are constructed using data compiled from multiple sources it is difficult to objectively assess optimal structure and complexity. We explore the use of mean squared error (MSE) to identify optimal model structure. To illustrate, we compare the performance of a single-patch model versus a two-patch model for a migratory songbird in a spatially structured environment. In the single-patch model, mean fecundity across patches is estimated, and movement between patches is ignored. In the two-patch model, patch-specific reproductive rates are estimated, which requires the estimation and use of rates of movement between patches. MSE performed well as an indicator of optimal model structure, consistently conforming to our intuition about which model should be preferred under different combinations of model parameters. We show that (1) when dispersal offsets patch-specific demography, the single-patch model will always be favored, (2) with good estimates of vital rates, only modest sample sizes for dispersal are required to make the two-patch model more accurate, and (3) the same factors that result in large bias in the simple model also result in large sampling error for the two-patch model. Finally, our analyses also suggest that erroneous conclusions about optimal model complexity can be reached when additional structure is added to an already poorly parameterized model and we recommend a stepwise approach to the study of model complexity.