A Comparison of Selected Training Algorithms for Recurrent Neural Networks

Abstract

Recurrent Neural Networks (RNNs) are one in which self-loops and backward weight connections between neurons are allowed. As a result of these network characteristics, recurrent networks can address temporal behaviors which not possible in feedforward neural networks, such as their behavior in the limit reaches a steady state (fixed point), an oscillation (limit cycle), and an a periodic instability (choas). Since RNNs have been increasingly applied to many dynamic system applications, there have been extensive efforts to develop a variety of architectures and training algorithms concerning on the enhancement of dynamic system characteristics. This work focuses on comparison of the selected and proposed training algorithms for RNNs. To evaluate the performance of the algorithms in the daily stock price forecasting in terms of efficiency index and computational time. A simple analysis on the complexity of RNNs was also carried out. It is noted that when comparing the speed of the algorithm, two components to be taken into account : the computation complexity and the space complexity.