Zinc (Zn) fertilizer application can certainly improve the production and nutritional quality of cereal crops. However, Zn accumulation in the soil may lead to some deleterious environmental impacts in agroecosystems. The effects of long-term Zn application on soil microbial properties remain unclear, but it is imperative to understand such effects. In this study, we collected soil samples from a nine-year field experiment in a wheat-maize system that continuously received Zn applied at various rates (0, 2.3, 5.7, 11.4, 22.7 and 34.1 kg ha−1) to evaluate the soil enzymes, microbial biomass and microbial community structure. The results showed that Zn application at the rate of 5.7 kg ha−1 significantly increased the activities of urease, invertase, alkaline phosphatase and catalase in the soil, while the rate of 34.1 kg ha−1 significantly decreased the evaluated enzyme activities. The microbial biomass carbon (C) and nitrogen (N) were not affected by Zn application rates, although an increase in the microbial biomass C was observed in the 11.4 kg ha−1 treatment. Moreover, the alpha diversity of the bacterial and fungal communities did not vary among the nil Zn, optimal Zn (5.7 kg ha−1) and excess Zn (34.1 kg ha−1) treatments. However, the bacterial communities in the soil receiving the optimal and excess Zn application rates were slightly changed. Compared to the nil Zn treatment, the other Zn application rates increased the relative abundances of the Rhodospirillales, Gaiellales and Frankiales orders and decreased the abundance of the Latescibacteria phylum. The redundancy analysis further indicated that the soil bacterial community composition significantly correlated with the concentrations of soil DTPA-Zn and total Zn. These results highlight the importance of optimal Zn application in achieving high production and high grain quality while concurrently promoting soil microbial activity, improving the bacterial community and further maintaining the sustainability of the agroecological environment.