Soil microbial communities perform critical functions in ecosystem processes. These functions can be used to assess the impact of agricultural practices on sustainable crop production. In this five-year study, the effect of various agricultural practices on soil microbial diversity and activity was investigated in a summer rainfall area under South African dryland conditions. Microbial diversity and activity were measured in the 0–15 cm layer of a field trial consisting of two fertilizer levels, three cropping systems, and two tillage systems. Using the Shannon–Weaver and Evenness diversity indices, soil microbial species richness and abundance were measured. Microbial enzymatic activities: β-glucosidase, phosphatase and urease, were used to evaluate ecosystem functioning. Cluster analysis revealed a shift in soil microbial community diversity and activity over time. Microbial diversity and activity were higher under no-till than conventional tillage. Fertilizer levels seemed to play a minor role in determining microbial diversity and activity, whereas the cropping systems played a more important role in determining the activity of soil microbial communities. Conservation agriculture yielded the highest soil microbial diversity and activity in diversified cropping systems under no-till.