Cytonuclear models of epistatic mating with backcrossing and selection

Abstract

We develop hybrid zone models that explore the combined effects of mating system and either backcrossing or viability selection on the disequilibria between nuclear and cytoplasmic genes. In the epistatic mating plus backcrossing model, we find patterns of permanent cytonuclear disequilibria like those found when epistatic mating is the only factor, as well as a novel combination of significant cytonuclear disequilibria sign patterns. The second group of models evaluates the potential of epistatic mating and postzygotic viability selection to maintain cytonuclear disequilibria. Simulations are used to evaluate nine patterns of selection, each of which represent differing forms of selection against hybrids, and show that while all disequilibria usually decay to zero, under certain circumstances a number of different patterns of significant cytonuclear disequilibria are possible at equilibrium. The results from these models are compared to the observed cytonuclear disequilibria previously found in a hybrid population of Hyla treefrogs.