When regional (limited-area) models of the hydrodynamics of the atmosphere and ocean are run over an extended time, variability unrelated to external “drivers” emerges: this variability is colloquially named “hydrodynamical noise” or just “noise”. This article summarises what we have learned in the past few years about the properties of such noise and its implications for numerical experimentation and analysis. The presence of this noise can be identified easily in ensembles of numerical simulations, and it turns out that the intensity of the noise is closely linked to scale-dependent “memory”. The “memory” in the atmosphere and ocean describes the persistence of atmospheric and oceanic conditions, usually quantified by an autocorrelation function. At the system level, this “memory” term, as given by Hasselmann’s stochastic climate model, plays a key role. In the case of marginal seas, the process of baroclinic instability modulated by tides and the formation of seasonal thermoclines are significant aspects. Some more general aspects are discussed, such as the applicability of the stochastic climate model to systems outside of atmospheric and oceanic dynamics, for example, biogeochemical systems, the irreversibility of tipping points, the challenges of detecting changes beyond a noise level, and the attribution of causes of change.