Revealing the diversity and composition of soil microbial communities (particularly bacterial communities) at aggregate scales exhibits a huge significance for maintaining the sustainable utilization of soil resources in the tea garden ecosystems. This study focused on the analysis of the bacterial community structure (based on the high-throughput sequencing measurement) in soil aggregates (at the 0–20 cm depth) collected from tea gardens at different ages (8, 17, 25, and 43 years) in southern Guangxi, China. Soil aggregates were classified into coarse (>2 mm), medium (2–1 mm), and fine (1–0.25 mm) macro-aggregates, as well as micro-aggregates (<0.25 mm) by the optimal moisture sieving procedure. In all tea gardens, soil coarse macro-aggregates exhibited the highest diversity of bacterial communities (as indicated by the Chao 1 and Shannon indices), thereby demonstrating that the mentioned aggregates exhibiting complicated bacterial communities could provide biological buffering and meantime avoid individual bacterial species to gain superiority via competition or predation. As the major parts of whole soil, coarse macro-aggregates were found to be more centralized in the 17-year tea gardens, indicating that the mentioned tea gardens provided a relatively conducive soil condition for bacterial proliferation and growth, hence making soil bacterial communities more diverse. Following 17 years of tea planting, the degradation of bacterial community structure in soil can be partially explained by the disintegration of coarse macro-aggregates. In addition, soil organic C availability (as indicated by the soil active C concentration: recalcitrant C concentration) and soil acidification also have significant effects on the variations in soil bacterial community diversity and composition during the tea planting. Therefore, following 17 years of tea planting, promoting the formation of soil coarse macro-aggregates is necessary to maintain soil health and bacterial community diversity in southern Guangxi, China.