Understanding changes in soil organic carbon and nutrient contents through converting grasslands to croplands and croplands to shrublands is critically important for grassland management. To investigate the impact of land-use changes on soil properties, we chose five sites within 60 km of the temperate grassland in Inner Mongolia as replicates. At each site, soil and vegetation characteristics of three land-use types, including natural grasslands, croplands, and shrublands (shrublands were converted from croplands), were separately investigated. We found the highest aboveground biomass in shrublands and the highest root biomass in grasslands among the three land-use types. We also found that soil bulk density increased, but soil organic carbon (OC), total nitrogen (TN) and total phosphorus (TP) contents decreased significantly in the cropland compared to the natural grassland. Compared with the long-term planted cropland, the 17-year-old shrub plantation did not significantly change soil organic carbon and nitrogen contents and storages, while it significantly reduced the soil phosphorus content and storage. Moreover, soil C:P ratio was the highest in the shrubland but the lowest in the cropland at 0–10 cm soil layer, while soil C:N and N:P ratios remained relatively stable in different land-use types within 0–50 cm soil profile. These findings demonstrate that the current shrub plantation and utilization mode are still not effective in restoring soil organic carbon and nutrients in the temperate grassland in Inner Mongolia. Further management improvements, such as decreasing mowing utilization and sowing herbage with developed roots in shrublands, are needed to accelerate the restoration.