Evaluating ensemble forecasts of plant species distributions under climate change

Abstract

Species distributions models (SDMs) are commonly used to assess potential species’ range shifts or extinction risk under climate change. It has been suggested that the use of ensemble forecasts, where a variety of model algorithms are used to generate consensus predictions, are preferred to individual SDMs by avoiding bias or prediction error inherent in a single modeling approach. Whereas several studies have assessed the performance of ensemble predictions using cross-validation or data-partitioning approaches, few studies have assessed the predictive accuracy of ensemble forecasts under climate change by using temporally independent model validation data. We used five SDM approaches to develop consensus forecasts of distributions of 145 vascular plant species from California in the 1930s and tested their projections against current distributions, a span of approximately 75 years. When evaluated with a portion of the model training data, consensus forecasts were highly accurate with an average AUC value of 0.97. False positive and false negative error rates were also low, exhibiting similar performance to random forest models. However, when evaluated with temporally independent data, the accuracy of consensus forecasts was similar to that of generalized linear and generalized additive models, with an average AUC value of 0.83. Our results suggest that the high levels of predictive accuracy exhibited by consensus forecasts when using data partitioning approaches may not reflect their performance when predicting temporally independent data. We contend that consensus forecasts may not represent the best approach for predicting species distributions under future climatic change, as they may not provide superior predictive accuracy in novel temporal domains compared to traditional modeling approaches that more readily lend themselves to ecological interpretation of model structure.