Accuracy Of Species Distribution Models Research Articles

Past, present and future distributions of an Iberian Endemic, Lepus granatensis: ecological and evolutionary clues from species distribution models.

The application of species distribution models (SDMs) in ecology and conservation biology is increasing and assuming an important role, mainly because they can be used to hindcast past and predict current and future species distributions. However, the accuracy of SDMs depends on the quality of the data and on appropriate theoretical frameworks. In this study, comprehensive data on the current distribution of the Iberian hare (Lepus granatensis) were used to i) determine the species’ ecogeographical constraints, ii) hindcast a climatic model for the last glacial maximum (LGM), relating it to inferences derived from molecular studies, and iii) calibrate a model to assess the species future distribution trends (up to 2080). Our results showed that the climatic factor (in its pure effect and when it is combined with the land-cover factor) is the most important descriptor of the current distribution of the Iberian hare. In addition, the model’s output was a reliable index of the local probability of species occurrence, which is a valuable tool to guide species management decisions and conservation planning. Climatic potential obtained for the LGM was combined with molecular data and the results suggest that several glacial refugia may have existed for the species within the major Iberian refugium. Finally, a high probability of occurrence of the Iberian hare in the current species range and a northward expansion were predicted for future. Given its current environmental envelope and evolutionary history, we discuss the macroecology of the Iberian hare and its sensitivity to climate change.

Improving species distribution models using biotic interactions: a case study of parasites, pollinators and plants

Biotic interactions have been considered as an important factor to be included in species distribution modelling, but little is known about how different types of interaction or different strategies for modelling affect model performance. This study compares different methods for including interspecific interactions in distribution models for bees, their brood parasites, and the plants they pollinate. Host–parasite interactions among bumble bees (genus Bombus: generalist pollinators and brood parasites) and specialist plant–pollinator interactions between Centris bees and Krameria flowers were used as case studies. We used 7 different modelling algorithms available in the BIOMOD R package. For Bombus, the inclusion of interacting species distributions generally increased model predictive accuracy. The improvement was better when the interacting species was included with its raw distribution rather than with its modeled suitability. However, incorporating the distributions of non‐interacting species sometimes resulted in similarly increased model accuracy despite their being no significance of any interaction for the distribution. For the Centris‐Krameria system the best strategy for modelling biotic interactions was to include the interacting species model‐predicted values. However, the results were less consistent than those for Bombus species, and most models including biotic interactions showed no significant improvement over abiotic models. Our results are consistent with previous studies showing that biotic interactions can be important in structuring species distributions at regional scales. However, correlations between species distributions are not necessarily indicative of interactions. Therefore, choosing the correct biotic information, based on biological and ecological knowledge, is critical to improve the accuracy of species distribution models and forecast distribution change.

Does the interpolation accuracy of species distribution models come at the expense of transferability?

Model transferability (extrapolative accuracy) is one important feature in species distribution models, required in several ecological and conservation biological applications. This study uses 10 modelling techniques and nationwide data on both (1) species distribution of birds, butterflies, and plants and (2) climate and land cover in Finland to investigate whether good interpolative prediction accuracy for models comes at the expense of transferability – i.e. markedly worse performance in new areas. Models’ interpolation and extrapolation performance was primarily assessed using AUC (the area under the curve of a receiver characteristic plot) and Kappa statistics, with supplementary comparisons examining model sensitivity and specificity values. Our AUC and Kappa results show that extrapolation to new areas is a greater challenge for all included modelling techniques than simple filling of gaps in a well‐sampled area, but there are also differences among the techniques in the degree of transferability. Among the machine‐learning modelling techniques, MAXENT, generalized boosting methods (GBM), and artificial neural networks (ANN) showed good transferability while the performance of GARP and random forest (RF) decreased notably in extrapolation. Among the regression‐based methods, generalized additive models (GAM) and generalized linear models (GLM) showed good transferability. A desirable combination of good prediction accuracy and good transferability was evident for three modelling techniques: MAXENT, GBM, and GAM. However, examination of model sensitivity and specificity revealed that model types may differ in their tendencies to either increased over‐prediction of presences or absences in extrapolation, and some of the methods show contrasting changes in sensitivity vs specificity (e.g. ANN and GARP). Among the three species groups, the best transferability was seen with birds, followed closely by butterflies, whereas reliable extrapolation for plant species distribution models appears to be a major challenge at least at this scale. Overall, detailed knowledge of the behaviour of different techniques in various study settings and with different species groups is of utmost importance in predictive modelling.

How biotic interactions may alter future predictions of species distributions: future threats to the persistence of the arctic fox in Fennoscandia

AbstractAim With climate change, reliable predictions of future species geographic distributions are becoming increasingly important for the design of appropriate conservation measures. Species distribution models (SDMs) are widely used to predict geographic range shifts in response to climate change. However, because species communities are likely to change with the climate, accounting for biotic interactions is imperative. A shortcoming of introducing biotic interactions in SDMs is the assumption that biotic interactions remain the same under changing climatic factors, which is disputable. We explore the performance of SDMs while including biotic interactions.Location Fennoscandia, Europe.Methods We investigate the appropriateness of the inclusion of biotic factors (predator pressure and prey availability) in assessing the future distribution of the arctic fox (Alopex lagopus) in Fennoscandia by means of SDM, using the algorithm MaxEnt.Results Our results show that the inclusion of biotic interactions enhanced the accuracy of SDMs to predict the current arctic fox distribution, and we argue that the accuracy of future predictions might also be enhanced. While the range of the arctic fox is predicted to have decreased by 43% in 2080 because of temperature‐related variables, projected increases in predator pressure and reduced prey availability are predicted to constrain the potential future geographic range of the arctic fox in Fennoscandia 13% more.Main conclusions The results indicate that, provided one has a good knowledge of past changes and a clear understanding of interactions in the community involved, the inclusion of biotic interactions in modelling future geographic ranges of species increases the predictive power of such models. This likely has far‐reaching impacts upon the design and implementation of possible conservation and management plans. Control of competing predators and supplementary feeding are suggested as necessary management actions to preserve the Fennoscandian arctic fox population in the face of climate change.

Keep collecting: accurate species distribution modelling requires more collections than previously thought

Aim Species distribution models (SDMs) use the locations of collection records to map the distributions of species, making them a powerful tool in conservation biology, ecology and biogeography. However, the accuracy of range predictions may be reduced by temporally autocorrelated biases in the data. We assess the accuracy of SDMs in predicting the ranges of tropical plant species on the basis of different sample sizes while incorporating real-world collection patterns and biases. Location Tropical South American moist forests. Methods We use dated herbarium records to model the distributions of 65 Amazonian and Andean plant species. For each species, we use the first 25, 50, 100, 125 and 150 records collected and available for each species to analyse changes in spatial aggregation and climatic representativeness through time. We compare the accuracy of SDM range estimates produced using the time-ordered data subsets to the accuracy of range estimates generated using the same number of collections but randomly subsampled from all available records. Results We find that collections become increasingly aggregated through time but that additional collecting sites are added resulting in progressively better representations of the species’ full climatic niches. The range predictions produced using time-ordered data subsets are less accurate than predictions from random subsets of equal sample sizes. Range predictions produced using time-ordered data subsets consistently underestimate the extent of ranges while no such tendency exists for range predictions produced using random data subsets. Main conclusions These results suggest that larger sample sizes are required to accurately map species ranges. Additional attention should be given to increasing the number of records available per species through continued collecting, better distributed collecting, and/or increasing access to existing collections. The fact that SDMs generally under-predict the extent of species ranges means that extinction risks of species because of future habitat loss may be lower than previously estimated.

Ensemble modelling of species distribution: the effects of geographical and environmental ranges

The aim of this study was to analyse the effects of species geographical and environmental ranges on the predictive performances of species distribution models (SDMs). We explored the usefulness of ensemble modelling approaches and tested whether species attributes influenced the outcomes of such approaches. Eight SDMs were used to model the current distribution of 35 fish species at 1110 stream sections in France. We first quantified the consensus among the resulting set of predictions for each fish species. Next, we created an average model by taking the average of the individual model predictions and tested whether the average model improved the predictive performances of single SDMs. Lastly, we described the ranges of fish species along four gradients: latitudinal, thermal, stream gradient (i.e. upstream-downstream) and elevation. After accounting for the effects of phylogenetic relatedness and species prevalence, these four species attributes were related to the observed variations in both consensus among SDMs and predictive performances by using generalized estimation equations. Our results highlight the usefulness of ensemble approaches for identifying geographical areas of agreement among predictions. Although the geographical extent of species had no effect on the performances of SDMs, we demonstrated that more consensual and accurate predictions were obtained for species with low thermal and elevation ranges, validating the hypothesis that specialist species yield models with higher accuracy than generalist ones. We emphasized that significant improvements in the accuracy of SDMs can be achieved by using an average model. Furthermore, these improvements were higher for species with smaller ranges along the four gradients studied. The geographical extent and ranges of species along environmental gradients provide promising insights into our understanding of uncertainties in species distribution modelling.

Effects of species’ traits and data characteristics on distribution models of threatened invertebrates

The lack of information about the distribution of threatened species inhibits the development of strategies for their conservation. This is a particularly important problem when considering invertebrates. Here we evaluate the effects of species’ traits and data characteristics on the accuracy of species distribution models (SDM) of 20 threatened Iberian invertebrates. We found that the accuracy of the predictions was mostly affected by the characteristics of the data. Species whose distributions were most accurately modelled were those with a greater sample size or smaller relative occurrence area (ROA). Species in habitats that were difficult to detect using GIS data, such as riparian species, tended to be more difficult to predict.

A spatially explicit method for evaluating accuracy of species distribution models

AbstractAim Models predicting the spatial distribution of animals are increasingly used in wildlife management and conservation planning. There is growing recognition that common methods of evaluating species distribution model (SDM) accuracy, as a global overall value of predictive ability, could be enhanced by spatially evaluating the model thereby identifying local areas of relative predictive strength and weakness. Current methods of spatial SDM model assessment focus on applying local measures of spatial autocorrelation to SDM residuals, which require quantitative model outputs. However, SDM outputs are often probabilistic (relative probability of species occurrence) or categorical (species present or absent). The goal of this paper was to develop a new method, using a conditional randomization technique, which can be applied to directly spatially evaluate probabilistic and categorical SDMs.Location Eastern slopes, Rocky Mountains, Alberta, Canada.Methods We used predictions from seasonal grizzly bear (Ursus arctos) resource selection functions (RSF) models to demonstrate our spatial evaluation technique. Local test statistics computed from bear telemetry locations were used to identify areas where bears were located more frequently than predicted. We evaluated the spatial pattern of model inaccuracies using a measure of spatial autocorrelation, local Moran’s I.Results We found the model to have non‐stationary patterns in accuracy, with clusters of inaccuracies located in central habitat areas. Model inaccuracies varied seasonally, with the summer model performing the best and the least error in areas with high RSF values. The landscape characteristics associated with model inaccuracies were examined, and possible factors contributing to RSF error were identified.Main conclusions The presented method complements existing spatial approaches to model error assessment as it can be used with probabilistic and categorical model output, which is typical for SDMs. We recommend that SDM accuracy assessments be done spatially and resulting accuracy maps included in model metadata.

Measuring and comparing the accuracy of species distribution models with presence-absence data

Species distribution models have been widely used to predict species distributions for various purposes, including conservation planning, and climate change impact assessment. The success of these applications relies heavily on the accuracy of the models. Various measures have been proposed to assess the accuracy of the models. Rigorous statistical analysis should be incorporated in model accuracy assessment. However, since relevant information about the statistical properties of accuracy measures is scattered across various disciplines, ecologists find it difficult to select the most appropriate ones for their research. In this paper, we review accuracy measures that are currently used in species distribution modelling (SDM), and introduce additional metrics that have potential applications in SDM. For the commonly used measures (which are also intensively studied by statisticians), including overall accuracy, sensitivity, specificity, kappa, and area and partial area under the ROC curves, promising methods to construct confidence intervals and statistically compare the accuracy between two models are given. For other accuracy measures, methods to estimate standard errors are given, which can be used to construct approximate confidence intervals. We also suggest that as general tools, computer-intensive methods, especially bootstrap and randomization methods can be used in constructing confidence intervals and statistical tests if suitable analytic methods cannot be found. Usually, these computer-intensive methods provide robust results.

Ecological‐Niche Modeling and Prioritization of Conservation‐Area Networks for Mexican Herpetofauna

One of the most important tools in conservation biology is information on the geographic distribution of species and the variables determining those patterns. We used maximum-entropy niche modeling to run distribution models for 222 amphibian and 371 reptile species (49% endemics and 27% threatened) for which we had 34,619 single geographic records. The planning region is in southeastern Mexico, is 20% of the country's area, includes 80% of the country's herpetofauna, and lacks an adequate protected-area system. We used probabilistic data to build distribution models of herpetofauna for use in prioritizing conservation areas for three target groups (all species and threatened and endemic species). The accuracy of species-distribution models was better for endemic and threatened species than it was for all species. Forty-seven percent of the region has been deforested and additional conservation areas with 13.7% to 88.6% more native vegetation (76% to 96% of the areas are outside the current protected-area system) are needed. There was overlap in 26 of the main selected areas in the conservation-area network prioritized to preserve the target groups, and for all three target groups the proportion of vegetation types needed for their conservation was constant: 30% pine and oak forests, 22% tropical evergreen forest, 17% low deciduous forest, and 8% montane cloud forests. The fact that different groups of species require the same proportion of habitat types suggests that the pine and oak forests support the highest proportion of endemic and threatened species and should therefore be given priority over other types of vegetation for inclusion in the protected areas of southeastern Mexico.

Testing the accuracy of species distribution models using species records from a new field survey

Species distribution models are a very popular tool in ecology and biogeography and have great potential to help direct conservation efforts. Models are traditionally tested by using half the original species records to build the model and half to evaluate it. However, this can lead to overly optimistic estimates of model accuracy, particularly when there are systematic biases in the data. It is better to evaluate models using independent data. This study used independent species records from a new to survey to provide a more rigorous evaluation of distribution-model accuracy. Distribution models were built for reptile, amphibian, butterfly and mammal species. The accuracy of these models was evaluated using the traditional approach of partitioning the original species records into model-building and model-evaluating datasets, and using independent records collected during a new field survey of 21 previously unvisited sites in diverse habitat types. We tested whether variation in distribution-model accuracy among species could be explained by species detectability, range size, number of records used to build the models, and body size. Estimates of accuracy derived using the new species records correlated positively with estimates generated using the traditional data-partitioning approach, but were on average 22% lower. Model accuracy was negatively related to range size and number of records used to build the models, and positively related to the body size of butterflies. There was no clear relationship between species detectability and model accuracy. The field data generally validated the species distribution models. However, there was considerable variation in model accuracy among species, some of which could be explained by the characteristics of species.

Species traits affect the performance of species distribution models for plants in southern California

Questions: To what extent do plant species traits, including life history, life form, and disturbance response characteristics, affect the degree to which species distributions are determined by physical environmental factors? Is the strength of the relationship between species distribution and environment stronger in some disturbance-response types than in others? Location: California southwest ecoregion, USA. Methods: We developed species distribution models (SDMs) for 45 plant species using three primary modeling methods (GLMs, GAMs, and Random Forests). Using AUC as a performance measure of prediction accuracy, and measure of the strength of species–environment correlations, we used regression analyses to compare the effects of fire disturbance response type, longevity, dispersal mechanism, range size, cover, species prevalence, and model type. Results: Fire disturbance response type explained more variation in model performance than any other variable, but other species and range characteristics were also significant. Differences in prediction accuracy reflected variation in species life history, disturbance response, and rarity. AUC was significantly higher for longer-lived species, found at intermediate levels of abundance, and smaller range sizes. Models performed better for shrubs than sub-shrubs and perennial herbs. The disturbance response type with the highest SDM accuracy was obligate-seeding shrubs with ballistic dispersal that regenerate via fire-cued germination from a dormant seed bank. Conclusions: The effect of species characteristics on predictability of species distributions overrides any differences in modeling technique. Prediction accuracy may be related to how a suite of species characteristics co-varies along environmental gradients. Including disturbance response was important because SDMs predict the realized niche. Classification of plant species into disturbance response types may provide a strong framework for evaluating performance of SDMs.

Assessing the accuracy of species distribution models to predict amphibian species richness patterns

1. Evaluating the distribution of species richness where biodiversity is high but has been insufficiently sampled is not an easy task. Species distribution modelling has become a useful approach for predicting their ranges, based on the relationships between species records and environmental variables. Overlapping predictions of individual distributions could be a useful strategy for obtaining estimates of species richness and composition in a region, but these estimates should be evaluated using a proper validation process, which compares the predicted richness values and composition with accurate data from independent sources. 2. In this study, we propose a simple approach to estimate model performance for several distributional predictions generated simultaneously. This approach is particularly suitable when species distribution modelling techniques that require only presence data are used. 3. The individual distributions for the 370 known amphibian species of Mexico were predicted using maxent to model data on their known presence (66,113 presence-only records). Distributions were subsequently overlapped to obtain a prediction of species richness. Accuracy was assessed by comparing the overall species richness values predicted for the region with observed and predicted values from 118 well-surveyed sites, each with an area of c. 100 km(2), which were identified using species accumulation curves and nonparametric estimators. 4. The derived models revealed a remarkable heterogeneity of species richness across the country, provided information about species composition per site and allowed us to obtain a measure of the spatial distribution of prediction errors. Examining the magnitude and location of model inaccuracies, as well as separately assessing errors of both commission and omission, highlights the inaccuracy of the predictions of species distribution models and the need to provide measures of uncertainty along with the model results. 5. The combination of a species distribution modelling method like maxent and species richness estimators offers a useful tool for identifying when the overall pattern provided by all model predictions might be representing the geographical patterns of species richness and composition, regardless of the particular quality or accuracy of the predictions for each individual species.

Effect of species rarity on the accuracy of species distribution models for reptiles and amphibians in southern California

ABSTRACTAim Several studies have found that more accurate predictive models of species’ occurrences can be developed for rarer species; however, one recent study found the relationship between range size and model performance to be an artefact of sample prevalence, that is, the proportion of presence versus absence observations in the data used to train the model. We examined the effect of model type, species rarity class, species’ survey frequency, detectability and manipulated sample prevalence on the accuracy of distribution models developed for 30 reptile and amphibian species.Location Coastal southern California, USA.Methods Classification trees, generalized additive models and generalized linear models were developed using species presence and absence data from 420 locations. Model performance was measured using sensitivity, specificity and the area under the curve (AUC) of the receiver‐operating characteristic (ROC) plot based on twofold cross‐validation, or on bootstrapping. Predictors included climate, terrain, soil and vegetation variables. Species were assigned to rarity classes by experts. The data were sampled to generate subsets with varying ratios of presences and absences to test for the effect of sample prevalence. Join count statistics were used to characterize spatial dependence in the prediction errors.Results Species in classes with higher rarity were more accurately predicted than common species, and this effect was independent of sample prevalence. Although positive spatial autocorrelation remained in the prediction errors, it was weaker than was observed in the species occurrence data. The differences in accuracy among model types were slight.Main conclusions Using a variety of modelling methods, more accurate species distribution models were developed for rarer than for more common species. This was presumably because it is difficult to discriminate suitable from unsuitable habitat for habitat generalists, and not as an artefact of the effect of sample prevalence on model estimation.

Assessing the accuracy of species distribution models: prevalence, kappa and the true skill statistic (TSS)

Summary In recent years the use of species distribution models by ecologists and conservation managers has increased considerably, along with an awareness of the need to provide accuracy assessment for predictions of such models. The kappa statistic is the most widely used measure for the performance of models generating presence–absence predictions, but several studies have criticized it for being inherently dependent on prevalence, and argued that this dependency introduces statistical artefacts to estimates of predictive accuracy. This criticism has been supported recently by computer simulations showing that kappa responds to the prevalence of the modelled species in a unimodal fashion. In this paper we provide a theoretical explanation for the observed dependence of kappa on prevalence, and introduce into ecology an alternative measure of accuracy, the true skill statistic (TSS), which corrects for this dependence while still keeping all the advantages of kappa. We also compare the responses of kappa and TSS to prevalence using empirical data, by modelling distribution patterns of 128 species of woody plant in Israel. The theoretical analysis shows that kappa responds in a unimodal fashion to variation in prevalence and that the level of prevalence that maximizes kappa depends on the ratio between sensitivity (the proportion of correctly predicted presences) and specificity (the proportion of correctly predicted absences). In contrast, TSS is independent of prevalence. When the two measures of accuracy were compared using empirical data, kappa showed a unimodal response to prevalence, in agreement with the theoretical analysis. TSS showed a decreasing linear response to prevalence, a result we interpret as reflecting true ecological phenomena rather than a statistical artefact. This interpretation is supported by the fact that a similar pattern was found for the area under the ROC curve, a measure known to be independent of prevalence. Synthesis and applications. Our results provide theoretical and empirical evidence that kappa, one of the most widely used measures of model performance in ecology, has serious limitations that make it unsuitable for such applications. The alternative we suggest, TSS, compensates for the shortcomings of kappa while keeping all of its advantages. We therefore recommend the TSS as a simple and intuitive measure for the performance of species distribution models when predictions are expressed as presence–absence maps.

Effects of sample size on accuracy of species distribution models

Given increasing access to large amounts of biodiversity information, a powerful capability is that of modeling ecological niches and predicting geographic distributions. Because, sampling species’ distributions is costly, we explored sample size needs for accurate modeling for three predictive modeling methods via re-sampling of data for well-sampled species, and developed curves of model improvement with increasing sample size. In general, under a coarse surrogate model, and machine-learning methods, average success rate at predicting occurrence of a species at a location, or accuracy, was 90% of maximum within ten sample points, and was near maximal at 50 data points. However, a fine surrogate model and logistic regression model had significantly lower rates of increase in accuracy with increasing sample size, reaching similar maximum accuracy at 100 data points. The choice of environmental variables also produced unpredictable effects on accuracy over the range of sample sizes on the logistic regression method, while the machine-learning method had robust performance throughout. Examining correlates of model performance across species, extent of geographic distribution was the only significant ecological factor.