Divergence Estimation in the Presence of Incomplete Lineage Sorting and Migration.

Abstract

This article focuses on the problem of estimating a species tree from multilocus data in the presence of incomplete lineage sorting and migration. I develop a mathematical model similar to IMa2 (Hey 2010) for the relevant evolutionary processes which allows both the population size parameters and the migration rates between pairs of species tree branches to be integrated out. I then describe a BEAST2 package DENIM (Divergence estimation notwithstanding ILS and migration) which is based on this model and which uses an approximation to sample from the posterior. The approximation is based on the assumption that migrations are rare, and it only samples from certain regions of the posterior which seem likely given this assumption. The method breaks down if there is a lot of migration. Using simulations, Leaché et al. (2014) showed that using the standard multispecies coalescent model to infer a species tree can result in poor accuracy if migration is present. I reanalyze this simulated data to explore DENIM's performance and demonstrate substantial improvements in accuracy over *BEAST. I also reanalyze an empirical data set.