Second Order Diagonal Recurrent Neural Network

Abstract

In this paper a new diagonal recurrent neural network that contains two recurrent weights in hidden layer is proposed. Since diagonal recurrent neural networks have simpler structure than the fully connected recurrent neural networks, they are easier to use in real-time applications. On the other hand, all diagonal recurrent neural networks in literature use one recurrent weight in hidden neurons, while the proposed network takes advantage of two recurrent weights. It will be shown, by simulations, that the proposed network can approximate nonlinear functions better than the existing diagonal recurrent neural networks. After deriving the training algorithm, the convergence stability and adaptive learning rate will be presented. The performance of the proposed network in model identification shows the accuracy of this network against the diagonal recurrent neural networks. Moreover, this network will be applied to realtime control of an image stabilization platform.