Compared to monocultures, multi‐species swards have demonstrated numerous positive diversity effects on aboveground plant performance, such as yield, N concentration, and even legacy effects on a following crop. Whether such diversity effects are seen in the soil microbiome is currently unclear. In a field experiment, we analyzed the effect that three plant species (a grass, forb, and legume), and mixtures of these, had on soil fungal and bacterial community structures, as well as their associated legacy effects under a following crop, the grass Lolium multiflorum. We utilized six sward types, three monocultures (Lolium perenne, Cichorium intybus and Trifolium pratense), two bi‐species mixtures, and a mixture of the three species. Soil samples were taken from these swards in March (at the end of a three year conditioning phase) and in June, August, and September after L. multiflorum was established, that is, the legacy samplings. When present, the differing monocultures had a significant effect on various aspects of the fungal community: structure, OTU richness, the relative abundance of the phylum Glomeromycota, and indicator OTUs. The effect on bacterial community structure was not as strong. In the multi‐species swards, a blending of individual plant species monoculture effects (identity effect) was seen in (a) fungal and bacterial community structure and (b) fungal OTU richness and the relative abundance of the Glomeromycota. This would indicate that plant species identity, rather than diversity effects (i.e., the interactions among the plant species), was the stronger determinant. During the legacy samplings, structural patterns in the fungal and bacterial communities associated with the previous swards were retained, but the effect faded with time. These results highlight that plant species identity can be a strong driver of soil microbial community structures. They also suggest that their legacy effect on the soil microbiome may play a crucial role in following crop performance.