Afforestation enhances the soil carbon (C) and nitrogen (N) sinks, critical for mitigating global warming and improving the ecological environment, processes that may be mediated by microbes. The rhizosphere soil normally hosts highly active microbes. However, the mechanism by which microbes in rhizosphere soil affect soil C and N after afforestation is still unclear. In this study, thirty years of poplar plantations were selected on the Lhasa River basin on the Tibetan Plateau. We found that afforestation improved the richness and evenness of soil bacteria, but not those of fungi. The alterations in rhizosphere soil microbial compounds were mainly driven by environmental changes induced by afforestation, while community differentiation was not. Increased bacterial richness and evenness were beneficial to soil C and N sinks. Interestingly, the differentiation in bacterial community composition had no obvious effect on soil C and N sinks, while the fungal community differentiation had a negative effect. Additionally, we found that Populus × beijingensis was the most suitable poplar species for enhancing soil C and N sinks on the Tibetan Plateau. In conclusion, this study first reports the mechanism of soil C and N sinks based on microbial diversity for afforestation and provides new insights into the selection of tree species on the Tibetan Plateau.