Method selection for species distribution modelling: are temporally or spatially independent evaluations necessary?

Abstract

To assess the realism of habitat projections in the context of climate change, we conduct independent evaluations of twelve species distribution models, including three novel ecosystem‐based modelling techniques. Habitat hindcasts for 24 western North American tree species were validated against 931 palaeoecological records from 6000, 11000, 14000, 16000 and 21000 yr before present. In addition, we evaluate regional extrapolations based on geographic splits of >55000 sample plots. Receiver operating characteristic analyses indicated excellent predictive accuracy for cross‐validations (median AUC of 0.90) and fair accuracy for independent regional and palaeoecological validations (0.78 and 0.75). Surprisingly, we found little evidence for over‐parameterisation in any method. Also, given high correlations found between model accuracies in non‐independent and independent evaluations, we conclude that non‐independent evaluations are effective model selection tools. Ecosystem‐based modelling approaches performed below average with respect to model sensitivity but excelled in specificity statistics and robustness against extrapolations far beyond training data, suggesting that they are well suited to reconstruct historical biogeographies and glacial refugia.