Interannual variation in grain yield of rice has been observed at both farm and regional scales, which is related to the climate variability. Previous studies focus on predicting the trend of climate change in the future and its potential effects on rice production using climate models; however, field studies are lacking to examine the climatic causes underlying the interannual yield variability for different rice cultivars. Here a 6-year field experiment from 2012 to 2017 was conducted using one hybrid (Yangliangyou6, YLY6) cultivar and one inbred (Huanghuazhan, HHZ) cultivar to determine the climate factors responsible for the interannual yield variation. A significant variation in grain yield was observed for both the inbred and hybrid cultivars across six planting years, and the coefficient of variation for grain yield was 7.3–10.5%. The night temperature (average daily minimum temperature, Tmin) contributed to the yield variability in both cultivars. However, the two cultivars showed different responses to the change in Tmin. The yield variation in HHZ was mainly explained by the effects of Tmin on grain filling percentage and grain weight, while the change in spikelets m−2 in response to Tmin accounted for the yield variability in YLY6. Further analysis found that spikelets m−2 of YLY6 significantly and negatively correlated with Tmin from transplanting to heading. For HHZ, the grain filling percentage and grain weight were significantly affected by Tmin of the week prior to heading and from heading to maturity, respectively. Overall, there were differences in the response mechanism between hybrid and inbred cultivars to high night temperature. These will facilitate the development of climate-resilient cultivars and appropriate management practices to achieve a stable grain yield.