Evaluating temporal and spatial generality: How valid are species–habitat relationship models?

Abstract

Prior to making general inferences or predictions from habitat models, their generalizability requires thorough assessment. However, systematic testing of model generality is often claimed, but rarely done. We used existing models for phytophagous insects (grasshoppers and leafhoppers) from a study on urban brownfields. Data for model building had been collected in two major cities of Northern Germany, Berlin and Bremen. We transferred these models to test data from another year (Bremen, 30 model transfers), and to test data from different geographic regions (transfer from Berlin to Bremen and vice versa, 30 model transfers). We evaluated discriminatory ability as well as model calibration for the test data. Most transfers (28 in time, 27 in space) were successful, i.e. occupied sites within the test data were assigned higher occurrence probabilities than unoccupied sites, the threshold independent c-index for the test data exceeded chance. Our results indicated that models built on the larger dataset (147 plots, Bremen) were more general than the ones basing on the smaller dataset (89 plots, Berlin). The overall good transferability had three important drawbacks: (1) models were mostly not well calibrated to the test data, thus predicted occurrence probabilities may not be used as absolute values, but as ordinal ranks. (2) Model fit to the test data often decreased considerably compared to the training data. (3) Dichotomising occurrence probabilities to presence/absence predictions required prior information about species prevalence. Assigning presences to the sites with the highest predicted occurrence probabilities, with the number of presences corresponding to the prevalence, proofed to be a comparatively simple and reliable way of dichotomising predictions. Still, it only allowed predictions exceeding chance for 19 model transfers in time and 23 transfers in space, and required information about species’ prevalences. We qualitatively compared pairs of models for 10 species, with one model basing on the Bremen data, one on the Berlin data. Both models had been built with the same modeling technique. Vegetation structure variables were largely comparable between models. It seemed that they were more directly related to species’ occurrences and thus more general than landscape context variables and soil parameters.