With the development of agriculture and industry, the increase in nitrogen (N) deposition has caused widespread concern among scientists. Although emission reduction policies have slowed N releases in Europe and North America, the threat to biodiversity cannot be ignored. Arbuscular mycorrhizal (AM) fungi play an important role in the establishment and maintenance of plant communities in forest ecosystems, and both their distribution and diversity have vital ecological functions. Therefore, we analyzed the effects of long-term N addition on AM fungi and understory herbaceous plants in a Korean pine plantation in northern China. The soil properties, community structure, and diversity of AM fungi and understory herbaceous plants were detected at different concentrations of NH4NO3 (0, 20, 40, 80 kg N ha−1 year−1) after 7 years. The results showed that long-term N deposition decreased soil pH, increased soil ammonium content, and caused significant fluctuations in P elements. N deposition improved the stability of soil aggregates by increasing the content of glomalin-related soil protein (GRSP) and changed the AM fungal community composition. The Glomus genus was more adaptable to the acidic soil treated with the highest N concentration. The species of AM fungi, understory herbaceous plants, and the biomass of fine roots were decreased under long-term N deposition. The fine root biomass was reduced by 78.6% in the highest N concentration treatment. In summary, we concluded that long-term N deposition could alter soil pH, the distribution of N, P elements, and the soil aggregate fractions, and reduce AM fungal and understory herb diversity. The importance of AM fungi in maintaining forest ecosystem diversity was verified under long-term N deposition.