Little is known about abundances of the major bacterial taxa in agricultural soils and how they are affected by fertilization or other agricultural practices. Our aim was to determine the abundance and relative distribution of several bacterial phyla and one class, as well as the archaeal and crenarchaeal communities, and how they were affected by different fertilization regimes to examine whether specific responses of microorganisms could be identified at these high taxonomic ranks. We used real-time PCR with taxa specific primers to quantify the abundance of the Actinobacteria, Acidobacteria, Bacteriodetes, Firmicutes, Gemmatimonadetes, Verrucomicrobia, Alphaproteobacteria and Crenarchaeota, as well as of the total bacteria and total archaea in soil sampled in 2002 and 2007 from a long term experimental field site subjected to six different fertilization regimes since 1956. The fertilization had affected soil pH, carbon and nitrogen, in addition to the C:N ratio. The total abundances of each taxon were affected in similar ways by the imposed treatments. By contrast, the relative abundance of certain taxa responded differently to the same fertilizer, with some taxa increasing while others decreased. Significant treatment effects were observed for the relative abundance of Alphaproteobacteria, Firmicutes, Verrucomicrobia and the Crenarchaeota and soil pH was the main driver for the observed differences. Differences between sampling years were also observed for the relative abundance of the Actinobacteria and the Bacteroidetes. Altogether, our results showed that agricultural practices can impact bacterial and archaeal phyla and classes in soil differently, which is consistent with recent work suggesting ecological coherence of members of bacterial clades at high taxonomic ranks.