We present a phenomological model of the Cosmic Spectral Energy Distribution (CSED) and the integrated galactic light (IGL) over all cosmic time. This model, based on an earlier model by Driver et al. (2013), attributes the cosmic star formation history to two processes -- firstly, chaotic clump accretion and major mergers, resulting in the early-time formation of bulges and secondly, cold gas accretion, resulting in late-time disc formation. Under the assumption of a Universal Chabrier initial mass function, we combine the Bruzual & Charlot (2003) stellar libraries, the Charlot & Fall (2000) dust attenuation prescription and template spectra for emission by dust and active galactic nuclei to predict the CSED -- pre- and post-dust attenuation -- and the IGL throughout cosmic time. The phenomological model, as constructed, adopts a number of basic axioms and empirical results and has minimal free parameters. We compare the model output, as well as predictions from the semi-analytic model GALFORM to recent estimates of the CSED out to $z=1$. By construction, our empirical model reproduces the full energy output of the Universe from the ultraviolet to the far-infrared extremely well. We use the model to derive predictions of the stellar and dust mass densities, again finding good agreement. We find that GALFORM predicts the CSED for $z < 0.3$ in good agreement with the observations. This agreement becomes increasingly poor towards $z = 1$, when the model CSED is $\sim$50 per cent fainter. The latter is consistent with the model underpredicting the cosmic star formation history. As a consequence, GALFORM predicts a $\sim$30 per cent fainter IGL.