Optimization with the Hopfield network based on correlated noises: Experimental approach

Abstract

This paper presents two simple optimization techniques based on combining the Langevin Equation with the Hopfield Model. Proposed models – referred as stochastic model (SM) and pulsed noise model (PNM) – can be regarded as straightforward stochastic extensions of the Hopfield optimization network. Both models follow the idea of stochastic neural network (Levy and Adams, IEEE Conference on Neural Networks, vol. III, San Diego, USA, 1987, pp. 681–689) and diffusion machine (Wong, Algorithmica 6 (1991) 466–478). They differ form the referred approaches by the nature of noises and the way of their injection. Optimization with stochastic model, unlike in the previous works, in which δ-correlated Gaussian noises were considered, is based on Gaussian noises with positive autocorrelation times. This is a reasonable assumption from a hardware implementation point of view. In the other model – pulsed noise model, Gaussian noises are injected to the system only at certain time instances, as opposed to continuously maintained δ-correlated noises used in the previous related works. In both models (SM and PNM) intensities of injected noises are independent of neurons’ potentials. Moreover, instead of impractically long inverse logarithmic cooling schedules, the linear cooling is tested. With the above strong simplifications neither SM nor PNM is expected to rigorously maintain thermal equilibrium (TE). However, numerical tests based on the canonical Gibbs–Boltzmann distribution show, that differences between rigorous and estimated values of TE parameters are relatively low (within a few percent). In this sense both models are said to perform quasithermal equilibrium. Optimization performance and quasithermal equilibrium properties of both models are presented based on the travelling salesman problem (TSP).