ABSTRACT The amount of power contained in the variations in galaxy star-formation histories (SFHs) across a range of time-scales encodes key information about the physical processes which modulate star formation. Modelling the SFHs of galaxies as stochastic processes allows the relative importance of different time-scales to be quantified via the power spectral density (PSD). In this paper, we build upon the PSD framework and develop a physically motivated, ‘stochastic’ prior for non-parametric SFHs in the spectral energy distribution (SED)-modelling code prospector. We test this prior in two different regimes: (1) massive, $z = 0.7$ galaxies with both photometry and spectra, analogous to those observed with the LEGA-C survey, and (2) $z = 8$ galaxies with photometry only, analogous to those observed with NIRCam on JWST. We find that it is able to recover key galaxy parameters (e.g. stellar mass, stellar metallicity) to the same level of fidelity as the commonly used continuity prior. Furthermore, the realistic variability information incorporated by the stochastic SFH model allows it to fit the SFHs of galaxies more accurately and precisely than traditional non-parametric models. In fact, the stochastic prior is $\gtrsim 2\times$ more accurate than the continuity prior in measuring the recent star-formation rates (log SFR$_{100}$ and log SFR$_{10}$) of both the $z = 0.7$ and $z = 8$ mock systems. While the PSD parameters of individual galaxies are difficult to constrain, the stochastic prior implementation presented in this work allows for the development of hierarchical models in the future, i.e. simultaneous SED-modelling of an ensemble of galaxies to measure their underlying PSD.