On the Theory of Prediction of Nonstationary Stochastic Processes

Abstract

We consider the following problem of prediction: During a finite time interval T the real valued function S(t)+N(t) is observed, in which S(t) is a signal and N(t) is a linearly superimposed noise disturbance. The problem is to predict the value of a given linear functional of S(t), the predictor formula having certain preassigned ``optimum properties'' among a certain class of predictors. In the case in which the mean value of S(t) is known, the random components of S(t) and N(t) are strictly stationary, and the time interval T is infinite, a complete solution to this problem has been given by N. Wiener. (In the case of discrete time series, the solution was given by A. Kolmogoroff.) This theory has been extended by L. Zadeh and J. Ragazzini [J. Appl. Phys. 21, 645 (1950] to the case in which T is a finite time interval and the mean value of S(t) is unknown but is restricted to be a polynomial in time. We extend the above theories to the case in which the random components of both S(t) and N(t) are nonstationary in time and merely possess finite continuous covariance and cross-covariance functions. The analytical tools of probability theory which we use are those developed independently by M. Loève and K. Karhunen in their studies of a class of stochastic processes usually termed processes of second order. Apparently these techniques are practically unknown outside of certain mathematical circles. In the opinion of the author these techniques are extremely powerful in the analysis of transient random phenomena in linear systems. Finally we give an exposition of a method of prediction by the theory of conditional probabilities. This method is applicable when the form of the joint probability distribution of signal and noise is known and hence is applicable to many practical problems, since this distribution is often Gaussian. In this particular case the predictor formula given by the theory of conditional probabilities is identical with the usual linear predictor formula given by the theory of least squares. In case the signal is a Markoff process, the method of conditional probabilities yields a much simpler prediction formula than the usual method of least squares.