Sparse distributed memory using N-of- M codes

Abstract

An analysis is presented of a sparse distributed memory (SDM) inspired by that described by Kanerva [Kanerva, P. (1988). Sparse distributed memory. Cambridge, MA: MIT Press] but modified to facilitate an implementation based on spiking neurons. The memory presented here employs sparse binary N-of- M codes, unipolar binary synaptic weights and a simple Hebbian learning rule. It is a two-layer network, the first (fixed) layer being similar to the ‘address decoder’ in Jaeckel's [Jaeckel, L.A. (1989). A class of designs for a sparse distributed memory. RIACS Technical Report 89.30, NASA Ames Research Centre] ‘hyperplane’ variant of Kanerva's SDM and the second (writeable) ‘data store’ layer being a correlation matrix memory as first proposed by Willshaw et al. [Willshaw, D. J., Buneman, O.P., & Longuet-Higgins, H.C. (1969). Non-holographic associative memory. Nature, 222, 960–962]. The resulting network is shown to have good storage efficiency and is scalable. The analysis is supported by numerical simulations and gives results that enable the configuration of the memory to be optimised for a range of noiseless and noisy environments.