Carbon budget estimation is crucial for scientifically assessing the ecosystem’s carbon sequestration capacity. Grasslands in the Qilian Mountains are widely recognized as vulnerable and sensitive to ongoing climate change. However, it is still unclear how the carbon budget of different grassland types responds to climate change in this area. Here, we evaluated the net primary productivity (NPP), soil heterotrophic respiration (RH), and net ecosystem productivity (NEP) of grassland ecosystems in the Qilian Mountains. Correlation analysis, linear regression, and standardized coefficient methodologies were employed to explore the response of NPP, RH, and NEP in various grassland types to changes in temperature, precipitation, and solar radiation over the past two decades. The results showed that the alpine meadow has the highest NPP and NEP while the temperate typical steppe has the highest RH. Grassland NPP, RH, and NEP increased significantly from 2001 to 2020, with a distinct spatial pattern of being low in the northwest and high in the southeast of the Qilian Mountains. NPP and NEP correlated with precipitation and solar radiation more closely than with temperature. Precipitation and solar radiation, respectively, explained 43.7% and 52.9% of grassland NEP alterations, and there was a threshold effect on the response of NEP and NPP to climate factors. Specifically, the NPP and NEP increased at first and then decreased as the precipitation, temperature, and solar radiation increased. The highest NEP and NPP occurred when precipitation ranged from 415 to 455 mm, along with temperatures between − 0.1–1.5 ℃ and solar radiation reaching 7685–7905MJ/m2. The warming and humidification of climate is conducive to the increase of NPP and NEP in alpine meadows and alpine steppes, but when the precipitation exceeds 415 mm and the temperature is greater than 1.5 ℃, the NPP and NEP begin to decrease. Solar radiation dominated changes in NPP and NEP of alpine meadows, alpine steppes, and temperate desert steppes. With the increase of solar radiation, NPP and NEP of these three grasslands first increased and then decreased. However, the changes in NPP and NEP of the temperate typical steppe were most affected by precipitation, and NPP and NEP first increase and then decrease with the increase of precipitation The higher the temperature, the higher the NPP, RH, and NEP of the temperate typical steppe. This study helps better understand how climate change impacts the carbon budget components of different grassland types in the Qilian Mountains and provides more theoretical thoughts on managing local grasslands sustainably.