Enhancement of Hopfield neural networks using stochastic noise processes

Abstract

Hopfield neural networks (HNN) are a class of densely connected single layer nonlinear networks of perceptrons. The network's energy function is defined through a learning procedure so that its minima coincide with states from a predefined set. However, because of the network's nonlinearity a number of undesirable local energy minima emerge from the learning procedure. This has shown to significantly effect the network's performance. In this work we present a stochastic process enhanced bipolar HNN. Presence of the stochastic process in the network enables us to describe its evolution using the Markov chains theory. When faced with a fixed network topology, the desired final distribution of states can be reached by modulating the network's stochastic process. Guided by the desired final distribution, we propose a general L/sup 2/ norm error density function optimization criterion for enhancement of the Hopfield neural network performance. This criterion can also be viewed in terms of stability intervals associated with the desired and non-desired stable states of the network. Because of the complexity of the general criterion we relax the optimization to the set of non-desired states. We further formulate a stochastic process design based on the stability intervals, which satisfies the optimization criterion and results in enhanced performance of the network. Our experimental simulations confirm the predicted improvement in performance.