Soil microbes are essential to maintain terrestrial ecosystem functionality. However, their diversity is threatened by land-use change, such as agricultural expansion and intensification. One important microbial group mediating the exchange of nutrients between plants and soil is arbuscular mycorrhizal (AM) fungi. The response of microorganism diversity to present and past habitat amount has been poorly studied. Here, we evaluate the potential role of current and historical natural habitat availability in explaining the diversity of AM fungi in arable fields. We conducted a spatially intensive sampling of three agricultural fields in Estonia. Soil AM fungal diversity was determined by soil DNA metabarcoding. We related AM fungal species richness, along with beta diversity components (turnover and nestedness), to abiotic conditions and natural habitat area availability at different spatial scales and time periods. Our findings showed a positive relationship between AM fungal richness and the amount of natural habitat area. Specifically, current AM fungal species richness was best explained by the amount of natural habitat from 130 years earlier, indicating a legacy effect of past land use on current soil biodiversity. The amount of past natural areas was negatively related to the beta diversity turnover component, indicating a replacement of AM fungal species in disturbed sites. While biodiversity-friendly farming is useful in promoting diverse soil biota, historical legacies can be persistent. Maintaining natural habitats around agricultural fields can further promote soil AM fungal diversity for future generations.