An on-line learning algorithm for recurrent neural networks using variational methods

Abstract

In this paper, an on-line learning algorithm for recurrent neural networks (RNN) using optimal control and a variational method is proposed. First, we obtain optimal weights given by a two-point boundary-value problem using the variational methods. And then the local gradient descent algorithm is derived such that on-line training is possible. This method is intended to be used on learning complex dynamic mappings between time-varying input-output data. Therefore it is useful for nonlinear control, identification, and signal processing applications of RNN. Simulation results for nonlinear plant identification are illustrated.