To understand the characteristics of the diurnal variation in soil respiration and its response to temperature, we used root exclusion plots, and buried CO2 sensors in situ during the late growing season in northwestern China. Soil organic carbon (SOC) decomposition and root respiration dynamics were quantified. In our study, we found that the diurnal variations in root respiration and soil organic carbon (SOC) decomposition showed a contrasting diurnal pattern. SOC decomposition peaked in the afternoon and was in phase with an increase in soil temperature at 10 cm; whereas root respiration decreased from ~08:00–09:00 and was minimal at ~17:00–18:00 despite an increase in soil temperature. Furthermore, an exponential function explained the diurnal variation in total soil respiration and SOC decomposition (r2 > 0.6, n = 504), but not so for root respiration (r2 < 0.3, n = 504). The fitted Q10 value of 4.3 for SOC decomposition was significantly higher than the Q10 value of 3.1 for root respiration. This result suggested that the root respiration rate had a different temperature sensitivity to the microbial respiration rate. In addition, we observed a significant (p < 0.001) clockwise hysteretic effect for SOC decomposition with respect to soil temperature at 10 cm over a 24 h period, with higher rates when soil temperature was increasing and lower rates when soil temperature was decreasing. By contrast, the diurnal hysteresis in root respiration with soil temperature at 10 cm was always counter-clockwise, with lower rates when soil temperature was increasing than when soil temperature was decreasing. This study emphasizes that root respiration and SOC decomposition have different responses to changing soil temperature. Therefore, modeling the impact of global climate change on soil carbon efflux should consider simultaneously, but separately, the impact of the two components.