Retrieval dynamics of associative memory of the Hopfield type

Abstract

The authors have investigated the dynamics of retrieval processes of an associative memory of the Hopfield type (1982). For synchronous dynamics, they have generalized the theory of Amari and Maginu (1988), which enables them to treat the intermediate processes of memory retrieval in terms of a few simple macrovariables to the finite-temperature case. The resulting phase diagram in the equilibrium limit agrees qualitatively well with that from equilibrium statistical mechanics. They have carried out Monte Carlo simulations to clarify the limit of applicability of their theory. They have found that their basic assumption, an independent Gaussian distribution of the noise term with a time-dependent variance, is satisfied if the network succeeds in retrieval. When retrieval fails, the distribution of noise is non-Gaussian from very early stages of time development. For asynchronous dynamics, they propose a time-dependent Ginzburg-Landau approach, which simply expresses a downhill motion of the network in the free energy landscape. The resulting flow diagram in a phase space describes the behaviour of the network when it is close to equilibrium.