A statistical procedure to assess the significance level of barriers to gene flow

Abstract

Although several methods are available to study the extent of isolation by distance (IBD) among natural populations, comparatively few exist to detect the presence of sharp genetic breaks in genetic distance datasets. In recent years, Monmonier's maximum-difference algorithm has been increasingly used by population geneticists. However, this method does not provide means to measure the statistical significance of such barriers, nor to determine their relative contribution to population differentiation with respect to IBD. Here, we propose an approach to assess the significance of genetic boundaries. The method is based on the calculation of a multiple regression from distance matrices, where binary matrices represent putative genetic barriers to test, in addition to geographic and genetic distances. Simulation results suggest that this method reliably detects the presence of genetic barriers, even in situations where IBD is also significant. We also illustrate the methodology by analyzing previously published datasets. Conclusions about the importance of genetic barriers can be misleading if one does not take into consideration their relative contribution to the overall genetic structure of species.