Rhizospheric microbes play important roles in plant growth and heavy metals (HMs) transformation, possessing great potential for the successful phytoremediation of environmental pollutants. In the present study, the rhizosphere of Elsholtzia haichowensis Sun was comprehensively studied to uncover the influence of environmental factors (EFs) on the whole microbial communities including bacteria, fungi and archaea, via quantitative polymerase chain reaction (qPCR) and high-throughput sequencing. By analyzing molecular ecological network and multivariate regression trees (MRT), we evaluated the distinct impacts of 37 EFs on soil microbial community. Of them, soil pH, HMs, soil texture and nitrogen were identified as the most influencing factors, and their roles varied across different domains. Soil pH was the main environmental variable on archaeal and bacterial community but not fungi, explaining 25.7%, 46.5% and 40.7% variation of bacterial taxonomic composition, archaeal taxonomic composition and a-diversity, respectively. HMs showed important roles in driving the whole microbial community and explained the major variation in different domains. Nitrogen (NH4-N, NO3-N, NO2-N and TN) explained 47.3% variation of microbial population composition and 15.9% of archaeal taxonomic composition, demonstrating its influence in structuring the rhizospheric microbiome, particularly archaeal and bacterial community. Soil texture accounted for 10.2% variation of population composition, 28.9% of fungal taxonomic composition, 19.2% of fungal a-diversity and 7.8% of archaeal a-diversity. Rhizosphere only showed strong impacts on fungi and bacteria, accounting for 14.7% and 4.9% variation of fungal taxonomic composition and bacterial a-diversity. Spatial distance had stronger influence on bacteria and archaea than fungi, but not as significant as other EFs. For the first time, our study provides a complete insight into key influential EFs on rhizospheric microbes and how their roles vary across microbial domains, giving a hand for understanding the construction of microbial communities in rhizosphere.