High-capacity Hebbian storage by sparse sampling

Abstract

The capacity of networks of ternary neurons, storing, by the so-called Hebbian rule, sparse vectors over {-1, 0, 1}(N), is shown to be at least of the order of N(2)/K log N, where K=Omega(log N) is the number of nonzero elements in each vector. The error correction capability of such networks is also analyzed. These results generalize previously known capacity bounds for binary networks storing vectors of equally probable {+/-1} bits and yield considerably higher capacities for small values of K.