Perturbations in soil microbial communities caused by climate warming are expected to have a strong impact on biodiversity and future climate–carbon (C) feedback, especially in vulnerable habitats that are highly sensitive to environmental change. Here, we investigate the impact of four-year experimental warming on soil microbes and C cycling in the Loess Hilly Region of China. The results showed that warming led to soil C loss, mainly from labile C, and this C loss is associated with microbial response. Warming significantly decreased soil bacterial diversity and altered its community structure, especially increasing the abundance of heat-tolerant microorganisms, but had no effect on fungi. Warming also significantly increased the relative importance of homogeneous selection and decreased “drift” of bacterial and fungal communities. Moreover, warming decreased bacterial network stability but increased fungal network stability. Notably, the magnitude of soil C loss was significantly and positively correlated with differences in bacterial community characteristics under ambient and warming conditions, including diversity, composition, network stability, and community assembly. This result suggests that microbial responses to warming may amplify soil C loss. Combined, these results provide insights into soil microbial responses and C feedback in vulnerable ecosystems under climate warming scenarios.