Summer temperature extremes can have large impacts on humans and the biosphere, and an increase in heat extremes is one of the most visible symptoms of climate change. Multiple mechanisms have been proposed that would predict faster warming of heat extremes than typical summer days, but it is unclear whether this is occurring. Here, we show that, in both observations and historical climate model simulations, the hottest summer days have warmed at the same pace as the median globally, in each hemisphere, and in the tropics from 1959 to 2023. In contrast, the coldest summer days have warmed more slowly than the median in the global average, a signal that is not simulated in any of 262 simulations across 28 CMIP6 models. The observed stretching of the cold tail indicates that observed summertime temperatures have become more variable despite the lack of hot day amplification. The interannual variability and trend in the warming of both hot and cold extremes compared to the median can be explained from a surface energy balance perspective based on changes in net surface radiation and evaporative fraction. Tropical hot day amplification is projected to emerge in the future (2024-2099, SSP3-7.0 scenario), while Northern Hemisphere heat extremes are expected to continue to follow the median.