Host identity has been recognized as a key determinant of the structure of ectomycorrhizal (ECM) fungal communities, but the importance of neighboring ECM hosts is less well understood. To investigate the relative importance of host and neighborhood effects, we examined the ECM fungal communities associated with Alnus rhombifolia, a host of specific ECM fungi, and Betula occidentalis, a host of generalist ECM fungi. We hypothesized that the host-specific Alnus-associated ECM fungal community would not be susceptible to the influence of plant neighborhood, while the generalist Betula-associated community would. ECM fungal communities on both hosts were characterized using ITS sequences derived from conspecific and heterospecific host settings at a field site in western Idaho, USA, and from a growth chamber bioassay. In the field study, the Betula neighborhood added minor constituents to the Alnus ECM fungal community, while in the bioassay, late planting of Betula generated strong priority effects that allowed the established Alnus neighborhood to control the structure of the Betula community. Our results indicate that while host identity acts as a primary filter on the composition and diversity of ECM fungal communities, proximity to a closely related host can mediate significant changes in community structure.