Within a sample from a population, the distribution of the number of descendants of a subsample’s most recent common ancestor

John L Spouge

doi:10.1016/j.tpb.2013.11.004

Abstract

Sample n individuals uniformly at random from a population, and then sample m individuals uniformly at random from the sample. Consider the most recent common ancestor (MRCA) of the subsample of m individuals. Let the subsample MRCA have j descendants in the sample (m⩽j⩽n). Under a Moran or coalescent model (and therefore under many other models), the probability that j=n is known. In this case, the subsample MRCA is an ancestor of every sampled individual, and the subsample and sample MRCAs are identical. The probability that j=m is also known. In this case, the subsample MRCA is an ancestor of no sampled individual outside the subsample. This article derives the complete distribution of j, enabling inferences from the corresponding p-value. The text presents hypothetical statistical applications pertinent to taxonomy (the gene flow between Neanderthals and anatomically modern humans) and medicine (the association of genetic markers with disease).

Highlights

A small p-value might suggest among other possibilities, e.g., that gene flow between the subpopulations represented by the subsample and its complement within the sample is not free, or that the genetic characters have a causal influence on the morphological character
Consider the phylogenetic tree presented in (Krings et al 2000)
The tree was consistent with reciprocal monophyly of Neanderthals and modern humans, but contained too few Neanderthals to conclude reciprocal monophyly at p ≤0.05 from tree topology alone (e.g., (Rosenberg 2007))

Summary

Introduction

A small p-value might suggest among other possibilities, e.g., that gene flow between the subpopulations represented by the subsample and its complement within the sample is not free (i.e., that the mathematical assumptions underlying the coalescent are violated), or that the genetic characters have a causal influence on the morphological character. To determine the distribution corresponding to the p-value, consider Kingman’s coalescent (Kingman 1982a, Kingman 1982b), where n individuals are sampled uniformly at random at time t0 from a large population. For 1 ≤ m ≤ j ≤ n , let pn,m; j denote the probability that the subsample MRCA has j descendants within the sample.

Theory

Numerical Results

Discussion