Chromosome number is a fundamental genomic trait that is often the first recorded characteristic of a genome. Across large clades, a common pattern emerges: many or even most lineages exhibit relative stasis, while a handful of lineages or species exhibit striking variation. Despite recent developments in comparative methods, most of this heterogeneity is still poorly understood. It is essential to understand why some lineages have rapid rates of chromosome number evolution, as it can impact a variety of other traits. Previous research suggests that biased female meiotic drive may shape rates of karyotype evolution in some mammals. However, Carnivora exhibits variation that this female meiotic drive model cannot explain. We hypothesize that variation in effective population size may underlie rate variation in Carnivora. To test this hypothesis, we estimated rates of fusions and fissions while accounting for range size, which we use as a proxy for effective population size. We reason fusions and fissions are deleterious or underdominant and that only in lineages with small range sizes will these changes be able to fix due to genetic drift. In this study, we find that the rates of fusions and fissions are elevated in taxa with small range sizes relative to those with large range sizes. Based on these findings, we conclude that 1) naturally occurring structural mutations that change chromosome number are underdominant or mildly deleterious, and 2) when population sizes are small, structural rearrangements may play an important role in speciation and reduction in gene flow among populations.