Soil microbial biomass is key to improving the prediction of soil organic carbon (SOC) dynamics by modeling. However, the driving mechanism of microbial biomass of different groups with soil depth is poorly understood across sites. Here, we compiled the biomass of different microbial groups (i.e., fungi, bacteria, gram-positive bacteria G+, and gram-negative bacteria G-) from the surface to a soil depth of 1 m from 71 soil profiles across three continents. We found that the biomass of microbial groups all decreased with soil depth but at different magnitudes, while the relative abundance of microbial groups, except G-, was relatively stable along soil profiles. Soil fungal biomass had a shallower vertical distribution than bacteria, especially G+, with 89% fungi and 76% G+ in the top 10 cm soils. In addition, a greater proportion of microbial biomass (71–89%) compared to SOC (64%) was in the top 10 cm soils, suggesting that microbes and SOC exhibited different vertical distributions. The vertical distributions of microbial biomass of different groups were significantly correlated with SOC and clay content but not with climate, and these distributions were different among land uses, highlighting the great influences of edaphic factors on vertical distributions of microbial biomass. The relationship between microbial biomass and soil depth provides a feasible way to estimate microbial biomass at different soil depths, which can serve as a benchmark to improve the prediction of SOC dynamics of entire soil profile at large scales.