Bias in estimators of archaic admixture

Abstract

This article evaluates bias in one class of methods used to estimate archaic admixture in modern humans. These methods study the pattern of allele sharing among modern and archaic genomes. They are sensitive to “ghost” admixture, which occurs when a population receives archaic DNA from sources not acknowledged by the statistical model. The effect of ghost admixture depends on two factors: branch-length bias and population-size bias. Branch-length bias occurs because a given amount of admixture has a larger effect if the two populations have been separated for a long time. Population-size bias occurs because differences in population size distort branch lengths in the gene genealogy. In the absence of ghost admixture, these effects are small. They become important, however, in the presence of ghost admixture. Estimators differ in the pattern of response. Increasing a given parameter may inflate one estimator but deflate another. For this reason, comparisons among estimators are informative. Using such comparisons, this article supports previous findings that the archaic population was small and that Europeans received little gene flow from archaic populations other than Neanderthals. It also identifies an inconsistency in estimates of archaic admixture into Melanesia.