Different habitat suitability models yield different least-cost path distances for landscape genetic analysis

Abstract

Habitat suitability models (HSMs) are used to describe and predict species distributions based on multiple ecological variables and species occurrence data. HSMs may also provide a probabilistic identification of least-cost path (LCP) distances in landscape genetics. However, while several studies used HSMs for these purposes, the performance of different HSMs in landscape genetic analysis and, therefore, the consequences of model choice have not been carefully explored. In this study, we used a large dataset of wolf genotypes (Canis lupus; n=923) that were non-invasively sampled in the central and northern Italian Apennines and western Alps, aiming (i) to estimate LCP distances derived from ten different HSMs and (ii) to quantify the correlation between inter-individual genetic and LCP distances using three statistical procedures: partial Mantel tests, multiple regression on distance matrices (MRDM) and linear mixed effect models. All LCP distances based on HSMs explained genetic distances better than Euclidean distances, irrespective of the applied landscape genetic statistical test. However, LCP distances derived by different HSMs were significantly different (paired t-test, P≤0.0001), especially between “flexible discriminant analysis” (FDA) and “boosted regression trees” (BRT) models. LCP distances derived from “factorial decomposition of Mahalanobis distances” (MADIFA) in MRDM showed the highest regression coefficient (β) with genetic distances, indicating a strong correlation between LCPs and genetic distances. Results from our case study suggest that different HSMs should be compared and model-choice procedures applied to identify the best fitting HSM in landscape genetic analysis.