From an a priori RNN to an a posteriori PRNN nonlinear predictor

Abstract

We provide an analysis of nonlinear time series prediction schemes, from a common recurrent neural network (RNN) to the pipelined recurrent neural network (PRNN), which consists of a number of nested small-scale RNNs. All these schemes are shown to be suitable for nonlinear autoregressive moving average (NARMA) prediction. The time management policy of such prediction schemes is addressed and classified in terms of a priori and a posteriori mode of operation. Moreover, it is shown that the basic a priori PRNN structure exhibits certain a posteriori features. In search for an optimal PRNN based predictor, some inherent features of the PRNN, such as nesting and the choice of cost function are addressed. It is shown that nesting in essence is an a posteriori technique which does not diverge. Simulations undertaken on a speech signal support the algorithms derived, and outperform linear least mean square and recursive least squared predictors.