The capacity of time-delay recurrent neural network for storing spatio-temporal sequences

Abstract

We investigate the capacity of a type of discrete-time recurrent neural network, called time-delay recurrent neural network, for storing spatio-temporal sequences. By introducing the order of a spatio-temporal sequence, the match law between a time-delay recurrent neural network and a spatio-temporal sequence has been established. It has been proved that the full order time-delay recurrent neural network of l-step feedback is able to learn and memorize (or store) any bipolar (or binary) spatio-temporal sequence of the order k if k⩽l, and that the asymptotic memory capacity of the first-order time-delay recurrent neural network of one-step feedback is not less than C1(n)=n+1, where n is the number of processing neurons in the network. Moreover, we substantiate the theoretical results by simulation experiments.