Climate variability plays a crucial role in the annual fluctuations of crop yields, posing a substantial threat to food security. Maize, the main cereal in sub-Saharan Africa, has shown varied yield trends during increasingly warmer growing seasons. Here we explore how sub-seasonal dry-wet spell patterns contribute to this variability, considering the spatial heterogeneity of crop responses, to map weather-related risks at a regional level. Our results show that shifts in specific dry-wet spell patterns across growth stages influence maize yield fluctuations in sub-Saharan Africa, explaining up to 50-60% of the interannual variation, which doubles that explained by mean changes in precipitation and temperature (30-35%). Precipitation primarily drives the onset of dry spells, while the influence of temperature increases with event intensity and peaks at the start of the growing season. Our large-scale, data-limited analysis approach has the potential to inform climate-smart agriculture in developing regions.