Despite the detrimental impacts of invasive plants on native biodiversity, ecosystem function, and management cost, few studies have focused on the long-term persistence of invaders. Here, we use a unique, long-term dataset to examine the recovery of northern rough fescue prairie communities, 41 years after the removal of livestock from Riding Mountain National Park, Manitoba, Canada. Our 1973 data suggest that summer grazing of plains’ rough fescue (Festuca hallii (Vasey) Piper) leads to its displacement from the plant community and increased dominance by exotic invaders. After 41 years of recovery time, historic grazing intensity remained an excellent predictor of community structure and composition. Areas classified as heavily grazed in 1973 remained characterized by exotic grasses and had significantly lower richness and diversity. While some exotic invaders persisted despite 41 years of community recovery, others were ephemeral. For example, both Poa pratensis (L.) and Bromus inermis (Leyss.) persisted, increasing in abundance across all classes of grazing intensity, suggesting that their control requires active restoration of the invaded areas. In contrast, passive restoration may be possible for a subset of ephemeral exotic species such as Taraxacum officinale, which had virtually disappeared from invaded prairies by 2010. Our long-term data provide a rare perspective into the long-term dynamics of plant invasions. Based on our findings, conservation managers will need to consider the dichotomy between persistent and ephemeral invaders and their impact on the recovery of northern prairie communities as they focus their restoration efforts against the mounting impacts of exotic plant invaders.