Abstract Plant nutrient composition is a reliable tool to identify plant ecological strategies and niche differentiation. Here, we aimed to (i) reveal the potential physiological mechanisms governing distinct foliar nutrient compositions among coexisting grassland species and (ii) clarify the role of taxonomy and environmental conditions in these compositions. At the local (grassland) scale, we measured root morphological and physiological traits, photosynthetic parameters and foliar concentrations of six macronutrients (N, P, S, K, Ca and Mg) and four micronutrients (Fe, Mn, Zn and Cu) across 18 coexisting species. At the regional scale, foliar macro‐ and micronutrient concentrations of 76 site‐species combinations of grasses and forbs across temperate meadow, typical, desert and saline grasslands were measured in Inner Mongolia. At the national scale, we collected the data on foliar macronutrient concentrations of 513 site‐species combinations across grasslands in Xinjiang, Qinghai‐Tibet and Inner Mongolia of China. Compared with forbs, grasses had 3.9 times higher root branching intensity, but 58% and 48% lower root carboxylate exudation and leaf transpiration rate. These distinct nutrient acquisition and transport strategies led to lower foliar nutrient (N, P, Ca, Mg and Mn) concentrations in grasses than forbs at the local scale. Most foliar micronutrient concentrations did not differ between grasses and forbs at the regional scale, whereas macronutrient concentrations were higher in forbs than grasses regardless of the grassland type at both the regional and national scales. All macronutrient concentrations exhibited stronger phylogenetic conservatism and less environmental control than micronutrients. The biogeochemical niche segregation (Euclidean distance based on PCA score of foliar macronutrient concentrations) between sympatric grasses and forbs was lower in meadow grassland than in the typical, desert and saline grasslands. Our results suggest that diverse nutrient acquisition and transport strategies can underlie the distinct foliar nutrient compositions between grasses and forbs. The linkage between potential physiological mechanisms and biogeochemical niche differentiation of grasses and forbs will advance our understanding of plant species coexistence and adaptive strategy. Read the free Plain Language Summary for this article on the Journal blog.