Bulk and rhizosphere soil microbiomes may be naturally linked and exhibit unique responses to different cover crop inputs during the early decomposition process. However, their relationship and contribution to soil carbon and nitrogen levels and plant growth during the decomposition process remain unclear. Herein, we performed a pot-based experiment to investigate the interactive effects of cover crops (rye, hairy vetch [HV], and a mixture of rye and HV [mix]) and soil microbial habitats (bulk and rhizosphere soils) on the microbial community. Additionally, we identified the hub taxa and evaluated their contributions to the microbial community, soil fertility, and plant growth. Lettuce was grown in soil exposed to control, rye, HV, or mixed cover crops. Bacterial 16S rRNA) and fungal (ITS) community structures were characterized in bulk and rhizosphere soils of lettuce plants using a DNA-based molecular approach. The results showed that cover crops and soil microbial habitats independently affected microbial community structures; the relative abundances of fungi and bacteria, known as decomposers in bulk soil and plant growth-promoting bacteria in rhizosphere soil, increased. The hub taxa under each treatment represented specific groups that connect bulk and rhizosphere microbiomes. Based on Mantel statistical analysis, hub taxa may maintain the bulk soil microbiome, within which the bacterial community in HV and mix treatments may contribute to the nitrogen supply from cover crop residues. Additionally, the bacterial community in rhizosphere soil in HV and mix treatments may contribute to nitrogen absorption by lettuce plants. These results add to the understanding of the early process of sustainable agriculture using cover crops.