The energy imbalance at the top of the atmosphere determines the temporal evolution of the global climate, and vice versa changes in the climate system can alter the planetary energy fluxes. This interplay is fundamental to our understanding of Earth's heat budget and the climate system. However, even today, the direct measurement of global radiative fluxes is difficult, such that most assessments are based on changes in the total energy content of the climate system. We apply the same approach to estimate the long-term evolution of Earth's radiative imbalance in the past. New measurements of noble gas-derived mean ocean temperature from the European Project for Ice Coring in Antarctica Dome C ice core covering the last 40,000 y, combined with recent results from the West Antarctic Ice Sheet Divide ice core and the sea-level record, allow us to quantitatively reconstruct the history of the climate system energy budget. The temporal derivative of this quantity must be equal to the planetary radiative imbalance. During the deglaciation, a positive imbalance of typically +0.2 W⋅m-2 is maintained for ∼10,000 y, however, with two distinct peaks that reach up to 0.4 W⋅m-2 during times of substantially reduced Atlantic Meridional Overturning Circulation. We conclude that these peaks are related to net changes in ocean heat uptake, likely due to rapid changes in North Atlantic deep-water formation and their impact on the global radiative balance, while changes in cloud coverage, albeit uncertain, may also factor into the picture.