Statistical analysis of the dynamics of a sparse associative memory

Abstract

A theoretical treatment of the dynamics of a recurrent autoassociative network is given. The network consists of randomly connected excitatory neurons, together with an inhibitory interneuron that sets their thresholds. Both the degree of connectivity between the neurons and the level of firing in the stored memories can be set arbitrarily. The memories are stored via a two-valued Hebbian, and evolution from an arbitrary initial state is by discrete, synchronous steps. The theory takes into account both spatial correlations between the learned connection strengths and temporal correlations between the state of the system and these connection strengths. Good qualitative and quantitative agreement with computer simulations is obtained for both the intermediate states and the final equilibrium state. Recall is studied as a function of initial state and of threshold parameters. The capacity of the network is investigated both numerically and analytically: there is a large increase in capacity as the level of firing decreases and also as the connectivity increases.