The Effects of Violating Hardy–Weinberg Equilibrium Assumptions on a Cluster‐based Population Mixture Analysis of Steelhead Populations in Southeast Alaska

Abstract

AbstractClustering methods for population mixture analysis assign individuals probabilistically to populations based on their multilocus genotype data. An assumption of the methods is that loci satisfy Hardy–Weinberg equilibrium (HWE) conditions within populations. We observed that violating this assumption by including loci measured as deviating from HWE in baseline samples for the mixture analysis at times introduced extra structure into the mixture sample, leading to biased composition estimates and overestimation of the number of populations. Provided that samples from at least some contributing populations are available and that baseline samples can safely be assumed to come from single populations, then a conservative approach to mixture analysis would be to include only those characters that conform to HWE in the baseline samples, but this approach could result in a loss of resolving power. To address this problem, we outlined an ad hoc method of selecting loci for their use in mixture analysis based on individual analyses of baseline samples. A subset of loci is selected for mixture analysis based on whether the loci contribute additional structure in the baseline populations, assuming that (1) baseline samples come from single populations and (2) any subpopulation structure resulting from the inclusion of loci measured as deviating from HWE is artifactual. Given that the final set may include only a few loci, the method would be most useful for small problems. We demonstrated the method on two sets of microsatellite loci (10 and 17 loci) genotyped in steelhead Oncorhynchus mykiss sampled at Sashin Creek in southeast Alaska. The method for selecting loci for mixture analysis generally reduced bias in and improved the precision of composition estimates and reduced the overestimation of the number of populations.