A posteriori real-time recurrent learning schemes for a recurrent neural network based nonlinear predictor

Abstract

Recurrent neural networks (RNNs) are well established for the nonlinear and nonstationary signal prediction paradigm. Appropriate learning algorithms, such as the real-time recurrent learning (RTRL) algorithm, have been developed for that purpose. However, little is known about the RNN time-management policy. Here, insight is provided into the time management of the RNN, and an a posteriori approach to the RNN based nonlinear signal prediction paradigm is offered. Based upon the chosen time-management policy, algorithms are developed, from the a priori learning-a priori error strategy through to the a posteriori learning-a posteriori error strategy. Compared with the a priori algorithms, the a posteriori algorithms offered are shown to provide a better prediction performance with little further expense in terms of computational complexity. Simulations undertaken on speech using the newly introduced algorithms confirm the theoretical results.