Arbuscular mycorrhizal fungi (AMF) are key attributes for the grassland ecosystems and their occurrence varies with increasing atmospheric nitrogen (N) deposition. The increasing N-deposition inevitably reduces the species richness and changes the community composition in a variety of grassland systems throughout the world. Despite the significant impacts of N-deposition on grassland AMF communities, only limited studies have explicitly examined the N-deposition effects on AMF linked with dominant plant species of grassland. Therefore, the current study investigated the effect of N-addition (e.g., 0, 7.06, 14.14, and 21.19 mg·kg−1) on AMF diversity and community composition linked with two dominant plant species of Inner Mongolia, i.e., Leymus chinensis (L. chinensis), Cleistogene squarrosa (C. squarrosa). The results showed that 65 AMF phylotypes were identified with the most dominant belonging to Glomeraceae (54), and its relative abundance increased with increasing N-addition in both plant species. The increasing N-addition resulted in a significant increase in AMF colonization (maximum 81% at N3), but species richness and diversity were not affected in C. squarrosa. In contrast, L. chinensis showed significantly reduced AMF colonization (minimum 38% at N3) and diversity with increasing N-addition, while species richness was not affected. N-addition did not alter AMF community composition in C. squarrosa, while for L. chinensis, significant differences were recorded. These findings imply that the sensitivity of AMF to N-addition is closely dependent on their host plant species. These findings also provide critical bases for the N-evoked differential response of AMF communities associated with dominant species of temperate grassland system, e.g., L. chinensis and C. squarrosa.