This study examines the effects of land use change on nitrate concentration and the abundances of ammonia-oxidizing archaea (AOA), ammonia-oxidizing bacteria (AOB), and narG-containing denitrifiers in shallow groundwater. The results reveal a general increase of nitrate in shallow groundwater following the change of land use from paddy fields to vegetable patches. Furthermore, a significant relationship between NO3−-N concentrations was observed both in groundwater and in soil at soil depths of 0–20, 20–40, 40–60, 60–80, and 80–100 cm. With regard to gene abundance in groundwater, the AOB amoA gene was most abundant and the AOA amoA gene copy numbers were lowest from the field with long-term paddy cultivation compared with the field under vegetable cultivation. The narG gene copy numbers were higher from the field under short-term vegetable cultivation compared with fields under long-term vegetable cultivation. The NO3−-N concentrations in groundwater correlated positively with AOA amoA gene copy numbers, negatively with the AOB amoA gene, but with no significant relationship with the narG gene. In conclusion, land use change from paddy fields to vegetable patches increases nitrate in groundwater, which is correlated significantly with nitrate in soil and the abundance of the amoA gene, but is not related to the narG gene in groundwater. This study also suggests that the removal of groundwater nitrate pollution is not feasible through biological denitrification without additional denitrifiers and that it might even become more aggravated because of the AOA.