Understanding the effects of herbivores on plant communities is needed for anticipating how variation in grazing regimes will impact natural and semi-natural ecosystems. Prominent ecological hypotheses predict that drastic reductions of herbivory, as have occurred in many European mountains, will trigger fast diversity loss and structural changes in grasslands, because grazing tolerant species are expected to be rapidly replaced by taller and more competitive species. The aims of this study were to test generalizations of herbivory effects (mostly from lowlands) in high elevation grasslands over ecological relevant times scales, and to unravel the contribution of climatic conditions to observed changes. Species richness, canopy height, plant community structure, and transition probabilities among ecological groups were monitored throughout 19 years in herbivore exclusions and control plots in the Central Pyrenees. We used ordination analyses to track the long-term community response to herbivore exclusion, and generalized additive models to assess the non-linear effects of herbivore exclusion and climatic conditions on measured variables. Contrary to expectations, herbivore exclusion did not significantly affect species richness, and although canopy height increased, it was not drastic enough to suppress shade intolerant species. The strongest shifts in plant community structure and transition probabilities between ecological groups occurred during a sequence of warm and dry growing seasons, whereas in control plots, these changes were smaller, and largely reversed after cooler and wetter climatic conditions returned. Our results suggest that long-term effects of grazing cessation in high elevation grasslands can be weaker and slower than predicted. However, these effects can act synergistically with dry and warm events. Therefore, the maintenance of past grazing activities can be key in the face of ongoing climatic warming.