To feed the growing human population, natural grasslands are being converted to agricultural use at a massive scale. This conversion may have negative consequences for soil biodiversity, but its impact on the community assembly of differentially microbial groups remains largely unknown. Here, we investigated the diversity and community compositions of bacteria, archaea, fungi and protists, using a paired sampling of grassland and cropland soils across the agro-pastoral ecotone of northern China. Land-use conversion decreased α diversity of bacteria, fungi and protists, and altered the structures of the entire soil microbial community (archaea, bacteria, fungi and protists). The community assembly of archaea and bacteria was dominated by stochastic processes, and that of protists dominated by deterministic processes in both land-use types. By contrast, the fungal community was governed more strongly by stochastic processes in grassland soil, than by deterministic processes in cropland soil. Our findings support the 'size-plasticity' hypothesis that smaller body-sized microorganisms (archaea and bacteria) are more structured by stochastic processes, and larger one (protist) is more influenced by deterministic processes. Our study demonstrates that distinct ecological processes govern microbial community assembly, and land-use change regulates the balance between determinism and stochasticity.