Learning high-degree sequences in a linear network

Abstract

An unsupervised learning algorithm for recurrent neural networks is proposed, that generalizes PCA to time series. A linear recurrent neural network using Oja's constrained Hebbian learning rule is presented. We demonstrate that this network extracts complex temporal information from a sequence of inputs. Temporal sequences stored in the network can be retrieved in the reverse order of presentation, providing a straight-forward implementation of a logical stack.