Abstract With rising extreme weather events due to climate change, the impact on agricultural production has become increasingly severe. Yet, there has been a significant gap in research that assesses the influence of day-to-day temperature variability on agricultural productivity on a global scale. Our study addresses this gap by exploring the effects of day-to-day temperature variability and the change of rainfall patterns on agricultural productivity worldwide from 1961 to 2018. The results reveal that day-to-day temperature variability not only has a direct, negative impact on agricultural Total Factor Productivity (TFP), but also influences it by modulating the effects of monthly average temperatures and wet days. One unit increase of day-to-day temperature variability leads to a 2% decrease in TFP. Day-to-day temperature variability neutralizes the impact of monthly average temperature on TFP, while exacerbating the impact of wet days on TFP. Furthermore, extreme rainfall events result in a consistent negative marginal effect across all countries/seasons/rainfall intervals. This study also identifies differentiated impacts across countries with varying income levels. Low-income regions’ TFP demonstrates markedly significant sensitivities to both monthly average temperatures and daily temperature fluctuations, which means less resilient. Furthermore, the impacts of general and extreme rainfall are comparatively less pronounced in high-income countries, indicating higher resilience to climate variability in these regions and a relative vulnerability to extreme weather events in low-income regions. Our findings illuminate the intricate and multifaceted role that daily temperature variability plays in agricultural productivity, providing a theoretical basis for understanding the heterogeneous impacts of climate change on agriculture and contributing insights into the broader discourse on climate resilience and agricultural sustainability.