Solar irradiance forecasting plays a crucial role in integrating large quantities of intermittent solar power. Forecasting systems are commonly evaluated using metrics like root‐mean‐ square error (RMSE) and skill scores. However, these metrics aggregated over larger data sets do not adequately assess the prediction of ramp events, which are critical for many applications. This article introduces a novel, simple, and adaptable ramp rate metric that analyzes ramp events between successive lead times within forecasts. A case study on ramp rate mitigation in PV systems benchmarks suitable ramp thresholds for various solar irradiance components. The capabilities and limitations of deterministic and probabilistic forecasts from two all‐sky imager‐based models are evaluated for ramp prediction. A state‐of‐the‐art data‐driven vision transformer End2End model excels in RMSE and skill scores but performs poorly in ramp prediction. Conversely, a novel generative forecasting model combined with a convolutional neural network‐based irradiance model shows superior ramp prediction, achieving an F1 score of ≥0.7 for critical ramp events. This study underscores the importance of suitable ramp rate metrics and highlights the potential of generative models for enhancing ramp forecasting.