A Paradox in the Theory of Prediction

Abstract

Given the set of past values, [Formula: see text], [Formula: see text], it is known that the conditional mean [Formula: see text] is the best predictor of [Formula: see text], [Formula: see text], where ‘best’ is defined in terms of minimization of mean square error. In this paper, we show that a prediction using the Riemann sum approximation to the spectral (Fourier) representation of a stationary time series produces a smaller mean square error. We attribute the resolution of this apparent paradox to the fact that the Riemann sum approach preserves more information of the spectral (frequency) content of the past time series than does the conditional mean — which effectively represents only the zeroth (constant value) frequency.