The Robustness of Hamilton's Rule with Inbreeding and Dominance: Kin Selection and Fixation Probabilities under Partial Sib Mating

Abstract

Assessing the validity of Hamilton’s rule when there is both inbreeding and dominance remains difficult. In this article, we provide a general method based on the direct fitness formalism to address this question. We then apply it to the question of the evolution of altruism among diploid full sibs and among haplodiploid sisters under inbreeding resulting from partial sib mating. In both cases, we find that the allele coding for altruism always increases in frequency if a condition of the form $$rb> c$$ holds, where r depends on the rate of sib mating α but not on the frequency of the allele, its phenotypic effects, or the dominance of these effects. In both examples, we derive expressions for the probability of fixation of an allele coding for altruism; comparing these expressions with simulation results allows us to test various approximations often made in kin selection models (weak selection, large population size, large fecundity). Increasing α increases the probability of fixation of recessive altruistic alleles ($$h< 1/ 2$$), while it can increase or decrease the probability of fixation of dominant altruistic alleles ($$h> 1/ 2$$).