Predicting a Chaotic Time Series Using a Dynamical Recurrent Neural Network

Abstract

Recurrent Neural Networks have appeared showing a better performance compared with traditional or feedforward networks. Recurrent Neural Networks are able to learn attractor dynamics, and they can store information for later use. This chapter presents two kinds of recurrent neural networks for time series forecasting. They both associate a time constant to each neuron. Moreover, dynamic recurrent neural networks (DRNN) are able to enhance static recurrent neural networks (SRNN) capabilities, specially handling with time dependent problems or temporal tasks. The main difference between DRNN and SRNN is that DRNN use an adaptive time constant associated to each neuron. These time constants act as a linear filter, and consider DRNN as a FIR network, but with recurrent connections. SRNN have limited storage capabilities and they may be inappropriate to deal with confusing time series. DRNN have more parameters than SRNN, hence to implement dynamical systems with chaotic behavior, one may train the network using a proper algorithm. Therefore, such network is trained using a modified version of the standard backpropagation algorithm called time dependent recurrent backpropagation.