Wheat powdery mildew caused by Blumeria graminis f. sp. tritici (Bgt) is a devastating disease that threatens wheat production and yield worldwide. The powdery mildew resistance gene Pm21, originating from wheat wild relative Dasypyrum villosum, encodes a coiled-coil, nucleotide-binding site, leucine-rich repeat (CC-NBS-LRR) protein and confers broad-spectrum resistance to wheat powdery mildew. In the present study, we isolated 73 Pm21 alleles from different powdery mildew-resistant D. villosum accessions, among which, 38 alleles were non-redundant. Sequence analysis identified seven minor insertion-deletion (InDel) polymorphisms and 400 single nucleotide polymorphisms (SNPs) among the 38 non-redundant Pm21 alleles. The nucleotide diversity of the LRR domain was significantly higher than those of the CC and NB-ARC domains. Further evolutionary analysis indicated that the solvent-exposed LRR residues of Pm21 alleles had undergone diversifying selection (dN/dS = 3.19734). In addition, eight LRR motifs and four amino acid sites in the LRR domain were also experienced positive selection, indicating that these motifs and sites play critical roles in resistance specificity. The phylogenetic tree showed that 38 Pm21 alleles were divided into seven classes. Classes A (including original Pm21), B and C were the major classes, including 26 alleles (68.4%). We also identified three non-functional Pm21 alleles from four susceptible homozygous D. villosum lines (DvSus-1 to DvSus-4) and two susceptible wheat-D. villosum chromosome addition lines (DA6V#1 and DA6V#3). The genetic variations of non-functional Pm21 alleles involved point mutation, deletion and insertion, respectively. The results also showed that the non-functional Pm21 alleles in the two chromosome addition lines both came from the susceptible donors of D. villosum. This study gives a new insight into the evolutionary characteristics of Pm21 alleles and discusses how to sustainably utilize Pm21 in wheat production. This study also reveals the sequence variants and origins of non-functional Pm21 alleles in D. villosum populations.