Willshaw model: Associative memory with sparse coding and low firing rates.

Abstract

The Willshaw model of associative memory, implemented in a fully connected network with stochastic asynchronous dynamics, is studied. In addition to Willshaw's learning rule, the network contains uniform synaptic inhibition, of relative strength K, and negative neural threshold -\ensuremath{\theta},\ensuremath{\theta}>0. The P stored memories are sparsely coded. The total number of on bits in each memory is Nf, where f is much smaller than 1 but much larger than lnN/N. Mean-field theory of the system is solved in the limit where C==exp(-${f}^{2}$P) is finite. Memory states are stable (at zero temperature), as long as C>${h}_{0}$==K-1+\ensuremath{\theta} and ${h}_{0}$>0. When C${h}_{0}$ or ${h}_{0}$0, P retrieval phases, highly correlated with the memory states, exist. These phases are only partially frozen at low temperature, so that the full memories can be retrieved from them by averaging over the dynamic fluctuations of the neural activity. In particular, when ${h}_{0}$0 the retrieval phases at low temperatures correspond to freezing of most of the population in a quiescent state while the rest are active with a time average that can be significantly smaller than the saturation level. These features resemble, to some extent, the observed patterns of neural activity in the cortex, in experiments of short-term memory tasks. The maximal value of P for which stable retrieval phases exist, scales as ${f}^{\mathrm{\ensuremath{-}}3}$/\ensuremath{\Vert}lnf\ensuremath{\Vert} for f\ensuremath{\gg}1/lnN, and as ${f}^{\mathrm{\ensuremath{-}}2}$ln(Nf/\ensuremath{\Vert}lnf\ensuremath{\Vert}) for f\ensuremath{\ll}1/lnN. Numerical simulations of the model with N=1000 and f=0.04 are presented. We also discuss the possible realization of the model in a biologically plausible architecture, where the inhibition is provided by special inhibitory neurons.