Alt'itudinal gradients strongly affect plant biodiversity, but the effects on microbial patterns remain unclear, especially in the large scale. We therefore designed an altitudinal gradient experiment that covered three climate zones to monitor soil microbial community dynamics and to compare those with plant and soil characteristics. Illumina sequencing of the 16S rRNA gene and ITS gene was used to analyze soil microbial (bacterial and fungal) diversity and composition, and fumigation–extraction was used to determine microbial biomass; the plant community metrics (i.e., percent cover, Shannon-Wiener, grass biomass, and carbon/nitrogen in leaf and biomass) and soil properties (i.e., soil moisture, soil temperature, bulk density, organic carbon, total nitrogen, and available nitrogen) were determined. The results showed that carbon/nitrogen in microbial biomass was higher at medium altitude and was positively related to carbon and nitrogen in both soil and grass biomass along the altitudinal gradients. Soil bacterial alpha diversity was significantly higher at medium altitude but fungal alpha diversity did not affected by altitudinal gradients; the effect of altitudinal gradients on bacterial beta diversity was larger than that on fungal beta diversity, although both groups were significantly affected by altitudinal gradients. Moreover, Alpha-proteobacteria, Beta-proteobacteria, and Gemmatimonadetes were significantly more abundant in higher altitude than in lower altitude, both Acidobacteria and Actinobacteria significantly declined with increasing altitude; other bacterial taxa such as Chloroflexi, Nitrospirae, Gamma-proteobacteria, and Delta-proteobacteria were significantly higher at medium altitudes. For fungal taxa, Basidiomycota and Ascomycota were the dominant phyla and responded insignificantly to the altitudinal gradients. The responses of microbial alpha diversity were mostly associated with plant Shannon index, organic carbon, and total nitrogen, whereas microbial beta diversity and composition mainly depended on soil moisture and temperature. Overall, these results suggest that soil bacteria rather than fungi can reflect changes in plant and soil characteristics along altitudinal gradients.