Learning Algorithm For Recurrent Networks Research Articles

Synchronized Interactions in Spiked Neuronal Networks

The study of artificial neural networks has originally been inspired by neurophysiology and cognitive science. It has resulted in a rich and diverse methodology and in numerous applications to machine intelligence, computer vision, pattern recognition and other applications. The random neural network (RNN) is a probabilistic model which was inspired by the spiking behaviour of neurons, and which has an elegant mathematical treatment that provides both its steady-state behaviour and offers efficient learning algorithms for recurrent networks. Second-order interactions, where more than one neuron jointly act upon other cells, have been observed in nature; they generalize the binary (excitatory–inhibitory) interaction between pairs of cells and give rise to synchronous firing (SF) by many cells. In this paper, we develop an extension of the RNN to the case of synchronous interactions, which are based on two cells that jointly excite a third cell; this local behaviour is in fact sufficient to create SF by large ensembles of cells. We describe the system state and derive its stationary solution as well as a O(N3) gradient descent learning algorithm for a recurrent network with N cells when both standard excitatory–inhibitory interactions, as well as SF, are present.

A New Learning Algorithm for Recurrent Networks

The most popular methods for modifying feedforward and recurrent neural networks’ weights, accordingly the backpropagation methods, actually are gradient methods. Due to die fact that the gradient is a local measure, the step towards the gradient’s direction, which step is given by the learning rate p, must be infinitesimal, which implies the choosing of a very small p. But this leads to a very slow convergence. Hence, still a large pis chosen. On the other hand, a too large pleads to strong oscillations of the aim function. Moreover, the values of pare relative to the problem to be solved.

Learning fixed point patterns by recurrent networks

Several learning algorithms have been derived for equilibrium points in recurrent neural networks. In this paper, we also consider learning the equilibrium points of such dynamical systems. We derive a structurally simple learning algorithm for recurrent networks which does not involve computing the trajectories of the system and we prove convergence and give examples. We also discuss solving for the connection weight matrix by iterative learning algorithms or direct solving.

Hopfield net generation, encoding and classification of temporal trajectories

Hopfield network transient dynamics have been exploited for resolving both path planning and temporal pattern classification. For these problems Lagrangian techniques and two well-known learning algorithms for recurrent networks have been used. For path planning, the Williams and Zisper's learning algorithm has been implemented and a set of temporal trajectories which join two points, pass through others, avoid obstacles and jointly form the shortest path possible are discovered and encoded in the weights of the net. The temporal pattern classification is based on an extension of the Pearlmutter's algorithm for the generation of temporal patterns which is obtained by means of variational methods. The algorithm is applied to a simple problem of recognizing five temporal trajectories with satisfactory robustness to distortions.