Overgrazing and phosphorus (P) deficiency are two major factors limiting the sustainable development of grassland ecosystems. Exploring plant P utilization and acquisition strategies under grazing can provide a solid basis for determining a reasonable grazing intensity. Both foliar P allocation and root P acquisition are crucial mechanisms for plants to adapt to environmental P availability; however, their changing characteristics and correlation under grazing remain unknown. Here, we investigated foliar P fractions, root P-acquisition traits and gene expression, as well as rhizosphere and bulk soil properties of two dominant plant species, Leymus chinensis (a rhizomatous grass) and Stipa grandis (a bunchgrass), in a field grazing intensity gradient site in Inner Mongolia. Grazing induced different degrees of compensatory growth in the two dominant plant species, increased rhizosphere P availability, and alleviated plant P limitation. Under grazing, the foliar metabolite P of L. chinensis increased, whereas the nucleic acid P of S. grandis increased. Increased P fractions in L. chinensis were positively correlated with increased root exudates and rapid inorganic P absorption. For S. grandis, increased foliar P fractions were positively correlated with more fine roots, more root exudates, and up-regulated expression of genes involved in defense and P metabolism. Overall, efficient root P mobilization and uptake traits, as well as increases in leaf metabolic activity-related P fractions, supported plant compensatory growth under grazing, a process that differed between tiller types. The highest plant productivity and leaf metabolic activity-related P concentrations under medium grazing intensity clarify the underlying basis for sustainable livestock production.