Long-term attraction in higher order neural networks

Abstract

Recent results on the memory storage capacity of higher order neural networks indicate a significant improvement compared to the limited capacity of the Hopfield model. However, such results have so far been obtained under the restriction that only a single iteration is allowed to converge. This paper presents a indirect convergence (long-term attraction) analysis of higher order neural networks. Our main result is that for any kappa(d)<d!2(d-1)/(2d)!, and 0< or =rho<1/2, a Hebbian higher order neural network of order d with n neurons can store a random set of kappa(d)n(d)/log n fundamental memories such that almost all memories have an attraction radius of size rhon. If kappa(d)<d!2(d-1)/((2d)!(d+1)), then all memories possess this property simultaneously. It indicates that the lower bounds on the long-term attraction capacities are larger than the corresponding direct convergence capacities by a factor of 1/(1-2rho) (2d). In addition we upper bound the convergence rate (number of iterations required to converge). This bound is asymptotically independent of n. Similar results are obtained for zero diagonal higher order neural networks.