Inverse cascades of kinetic energy and thermal variance in the subset of vertically homogeneous modes in spectral space are found to cause a slow aggregation to a pair of convective supergranules that eventually fill the whole horizontally extended, three-dimensional, turbulent Rayleigh-B\'{e}nard convection layer when a heat flux is prescribed at the top and bottom. An additional weak rotation of the layer around the vertical axis stops this aggregation at a scale that is smaller than the lateral domain extension and ceases the inverse cascade for the thermal variance. The inverse cascade for the kinetic energy remains intact, even for times at which the root mean square values of temperature and velocity have reached the statistically steady state. This kinetic energy inverse cascade sustains the horizontally extended convection patterns which are best visible in the temperature field. The resulting characteristic length of the aggregated convection patterns depends on the thermal driving and linearly on the strength of rotation. Our study demonstrates the importance of inverse energy cascades beyond the two-dimensional turbulence case in a three-dimensional convection flow that is subject to a multi-scale energy injection by thermal plumes and driven by boundary heat fluxes as typically present in natural geo- and astrophysical systems, such as solar convection.