Abstract We calculate the implied horizontal heat transport due to the spatial anomalies of radiative fluxes at the top of the atmosphere (TOA). The regional patterns of implied heat transport for different components of the TOA fluxes are calculated by solving the Poisson equation with the flux components as source terms. The shortwave (SW) part of the spectrum governs the spatial patterns of the total implied heat transport. Using the cloud radiative effect (CRE) as source term, we show that the direct effect of clouds is to reduce the poleward heat transport in the majority of the Northern Hemisphere and at high southern latitudes. Clouds flatten the gradients of the clear-sky energy flux potential and hence reduce the implied heat transport with respect to clear skies. Clouds reduce the implied cross-equatorial heat transport with respect to clear sky through changes in the SW part of the spectrum. It changes from 0.83 PW in clear sky to −0.01 PW in all sky, equivalent to the hemispheric albedo symmetry reported in previous studies. We investigate hemispheric symmetry by introducing a metric that measures the symmetry of implied meridional heat transports at all latitudes. The direct effect of clouds is to increase the symmetry in the implied heat transport, and this is achieved through an increase in symmetry in the SW part of the spectrum in the tropics. Whether this is trivial or the result of a fundamental control in the climate system is still an open question.