The long-term balancing selection acting on mating types or sex-determining genes is expected to lead to the accumulation of deleterious mutations in the tightly linked chromosomal segments that are locally ‘sheltered’ from purifying selection. However, the factors determining the extent of this accumulation are poorly understood. Here, we took advantage of variations in the intensity of balancing selection along a dominance hierarchy formed by alleles at the sporophytic self-incompatibility system of the Brassicaceae to compare the pace at which linked deleterious mutations accumulate among them. We first experimentally measured the phenotypic manifestation of the linked load at three different levels of the dominance hierarchy. We then sequenced and phased polymorphisms in the chromosomal regions linked to 126 distinct copies of S-alleles in two populations of Arabidopsis halleri and three populations of Arabidopsis lyrata. We find that linkage to the S-locus locally distorts phylogenies over about 10–30 kb along the chromosome. The more intense balancing selection on dominant S-alleles results in greater fixation of linked deleterious mutations, while recessive S-alleles accumulate more linked deleterious mutations that are segregating. Hence, the structure rather than the overall magnitude of the linked genetic load differs between dominant and recessive S-alleles. Our results have consequences for the long-term evolution of new S-alleles, the evolution of dominance modifiers between them, and raise the question of why the non-recombining regions of some sex and mating type chromosomes expand over evolutionary times while others, such as the S-locus of the Brassicaceae, remain restricted to small chromosomal regions.