Straw returning may stimulate soil microbial activity, thereby influencing microbial-mediated soil nitrification, which can lead to nitrate leaching and nitrogen (N) loss. However, its effects under long-term nitrogen fertilization remain unclear. At an experimental station with 34 years of fertilizer application (0, 135, and 270 kg ha−1 N), we investigated how nitrogen fertilization and straw returning affected the soil potential nitrification rate (PNR) and ammonia-oxidizing microorganisms (AOM). Our results suggest that N fertilization concurrently inhibits soil PNR, but this inhibition can be alleviated by straw returning, particularly with low nitrogen fertilization (p < 0.05). Long-term N fertilization significantly decreased the abundance of ammonia-oxidizing archaea (AOA), ammonia-oxidizing bacteria (AOB), and complete ammonia-oxidizing bacteria cladeB (CAOB-cladeB). Straw returning increased AOA abundance and diversity, especially with low or no fertilization (p < 0.05). Furthermore, the partial least squares path model demonstrated that AOA abundance affected soil PNR by altering the AOA community. According to random forest analysis, soil pH and AOA beta diversity were the primary factors affecting soil PNR (explaining 10.76% and 10.03% of the variation, respectively). Overall, our findings highlight the importance of straw returning and AOA in soil nitrification under long-term nitrogen fertilization, emphasizing the need to consider these interactions for sustainable agriculture.