In order to predict the effects of climate change on the global carbon cycle, it is crucial to understand the environmental factors that affect soil carbon storage in grasslands. In the present study, we attempted to explain the relationships between the distribution of soil carbon storage with climate, soil types, soil properties and topographical factors across different types of grasslands with different grazing regimes. We measured soil organic carbon in 92 locations at different soil depth increments, from 0 to 100 cm in southwestern China. Among soil types, brown earth soils (Luvisols) had the highest carbon storage with 19.5 ± 2.5 kg m−2, while chernozem soils had the lowest with 6.8 ± 1.2 kg m−2. Mean annual temperature and precipitation, exerted a significant, but, contrasting effects on soil carbon storage. Soil carbon storage increased as mean annual temperature decreased and as mean annual precipitation increased. Across different grassland types, the mean carbon storage for the top 100 cm varied from 7.6 ± 1.3 kg m−2 for temperate desert to 17.3 ± 2.9 kg m−2 for alpine meadow. Grazing/cutting regimes significantly affected soil carbon storage with lowest value (7.9 ± 1.5 kg m−2) recorded for cutting grass, while seasonal (11.4 ± 1.3 kg m−2) and year-long (12.2 ± 1.9 kg m−2) grazing increased carbon storage. The highest carbon storage was found in the completely ungrazed areas (16.7 ± 2.9 kg m−2). Climatic factors, along with soil types and topographical factors, controlled soil carbon density along a soil depth in grasslands. Environmental factors alone explained about 60% of the total variation in soil carbon storage. The actual depth-wise distribution of soil carbon contents was significantly influenced by the grazing intensity and topographical factors. Overall, policy-makers should focus on reducing the grazing intensity and land conversion for the sustainable management of grasslands and C sequestration.