It is still not clear which group of ammonia-oxidizing microorganisms plays the most important roles in nitrification in soils. Change in abundances and community compositions of ammonia-oxidizing bacteria (AOB) and ammonia-oxidizing archaea (AOA) under long-term different nitrogen (N) fertilization rates were investigated in an acidic luvisols soil using real-time polymerase chain reaction and denaturing gradient gel electrophoresis, respectively, based on the ammonia monooxygenase a-subunit gene. The experimental plan included the following treatments: control without N fertilization (NCK), low N fertilization rate, middle N fertilization rate, and high N fertilization rate as 0, 100, 150, and 250 kg urea-N ha−1, respectively. Long-term different N fertilization rates did not significantly alter the total C and N contents of soil while it significantly decreased soil pH, which ranged from 5.60 to 5.20. The AOB abundance was more abundant in the N fertilization treatments than the NCK treatment; the AOA abundance decreased by the increasing N fertilization rates, as did the ratios of AOA/AOB. The large differences in the potential nitrification rates among four treatments depended on the changes in AOA abundance but not to changes in AOB abundance. Phylogenetic analysis showed that the AOB communities were dominated by Nitrosospira clusters 1, 3, and 9 while all AOA sequences were grouped into soil/sediment cluster except for one sequence. Taken together, these results indicated that AOB and AOA preferred different soil N conditions and AOA were functionally more important in the nitrification than AOB in the acidic luvisols soil.