The IICR and the non-stationary structured coalescent: towards demographic inference with arbitrary changes in population structure

Willy Rodríguez,Armando Arredondo,Lounès Chikhi,Simona Grusea,Josué M Corujo,Simon Boitard,Olivier Mazet

doi:10.1038/s41437-018-0148-0

Abstract

In the last years, a wide range of methods allowing to reconstruct past population size changes from genome-wide data have been developed. At the same time, there has been an increasing recognition that population structure can generate genetic data similar to those produced under models of population size change. Recently, Mazet et al. (Heredity 116:362–371, 2016) showed that, for any model of population structure, it is always possible to find a panmictic model with a particular function of population size changes, having exactly the same distribution of T2 (the coalescence time for a sample of size two) as that of the structured model. They called this function IICR (Inverse Instantaneous Coalescence Rate) and showed that it does not necessarily correspond to population size changes under non-panmictic models. Besides, most of the methods used to analyse data under models of population structure tend to arbitrarily fix that structure and to minimise or neglect population size changes. Here, we extend the seminal work of Herbots (PhD thesis, University of London, 1994) on the structured coalescent and propose a new framework, the Non-Stationary Structured Coalescent (NSSC) that incorporates demographic events (changes in gene flow and/or deme sizes) to models of nearly any complexity. We show how to compute the IICR under a wide family of stationary and non-stationary models. As an example we address the question of human and Neanderthal evolution and discuss how the NSSC framework allows to interpret genomic data under this new perspective.

Highlights

Electronic supplementary material The online version of this article contains supplementary material, which is available to authorized users.Reconstructing the demographic history of populations and species remains one of the great challenges of population genetics and statistical inference (Harpending and Rogers 2000; Beaumont et al 2002; Goldstein and Chikhi 2002; Hey and Machado 2003; Li and Durbin 2011; Liu and Fu 2015; Scerri et al 2018)
The theoretical framework presented in this study is closely related to Herbots’ works (Herbots 1994; Wilkinson-Herbots 1998), who introduced the use of transition rate matrices for studying structured models and computed the coefficients of the transition rate matrix for many stationary models
It includes a natural way of incorporating past demographic events into models of population structured

Summary

Introduction

Previous sections showed that to any given stationary structured population model corresponds a transition rate matrix, Q that can be constructed and used to predict the IICR for a given sample configuration. & etQ0 ; eTQ0 eðtÀTÞQ1 ; if t T otherwise: In particular, the distribution of Tkα, the first coalescence time of k genes sampled in configuration α under this structured model with a past demographic change event, can be computed by: P~ t ðnα ; the NSSC framework allows to compute the IICR for models considering a population split. The distribution of coalescence times (and the IICR) for structured models in which migration rates and demes sizes can arbitrarily change, can be obtained from the computation of matrix exponentials and matrix products. The simple fact of sampling diploids in different demes (humans or Neanderthals) generates the very different PSMC plots inferred for humans and Neanderthals

Discussion and perspectives

Findings

Compliance with ethical standards