A non exchangeable coalescent arising in phylogenetics

Abstract

A popular line of research in evolutionary biology is to use time-calibrated phylogenies in order to infer the underlying diversification process. This involves the use of stochastic models of ultrametric trees, i.e., trees whose tips lie at the same distance from the root. We recast some well-known models of ultrametric trees (infinite regular trees, exchangeable coalescents, coalescent point processes) in the framework of so-called comb metric spaces and give some applications of coalescent point processes to the phylogeny of bird species. However, these models of diversification assume that species are exchangeable particles, and this always leads to the same (Yule) tree shape in distribution. Here, we propose a non-exchangeable, individual-based, point mutation model of diversification, where interspecific pairwise competition is only felt from the part of individuals belonging to younger species. As the initial (meta)population size grows to infinity, the properly rescaled dynamics of species lineages converge to a one-parameter family of coalescent trees interpolating between the caterpillar tree and the Kingman coalescent. Keywords: ultrametric tree, inference, phylogenetic tree, phylogeny, birth-death process, population dynamics, evolution