A limit theorem for trees of alleles in branching processes with rare neutral mutations

Abstract

We are interested in the genealogical structure of alleles for a Bienaymé–Galton–Watson branching process with neutral mutations (infinite alleles model), in the situation where the initial population is large and the mutation rate small. We shall establish that for an appropriate regime, the process of the sizes of the allelic sub-families converges in distribution to a certain continuous state branching process (i.e. a Jiřina process) in discrete time. Itô’s excursion theory and the Lévy–Itô decomposition of subordinators provide fundamental insights for the results.