Retrieval Properties Of Neural Networks Research Articles

Suppression of Retrieval States in Spin-Glass-Like Model of Neural Networks with Random Neuronal Threshold11The project supported by National Natural Science Foundation of China.

We propose a mechanism to consider the effects of the random neuronal threshold on the retrieval properties of neural networks. The retrieval properties of the fully asymmetric Sherrington-Kirkpatrick (SK) and Hopfield models are studied, respectively. We show that the ferromagnetic retrieval states are destroyed by fluctuations induced by random neuronal thresholds.

Retrieval phase diagrams in the asymmetric Sherrington-Kirkpatrick model and in the Little-Hopfield model.

The effects of random external fields on the retrieval properties of neural networks are examined. Phase diagrams of the fully asymmetric Sherrington-Kirkpatrick model and Little-Hopfield model are obtained. It is shown that the ferromagnetic retrieval states are destroyed by fluctuations induced by random external fields. The probability at zero field determines the size of the basins of attraction. The basin boundary occurring in systems with a discrete random-field distribution cannot be observed in systems with a continuous Gaussian distribution, and are due to the peculiarities of the discrete distribution.

Retrieval properties of neural networks with hierarchical clustering

By using the statistical mechanics formulation of Amit, Gutfreund and Sompolinsky (1985) the authors investigate the retrieval properties of a model for neural networks that exhibits the same organization into clusters as Dyson's hierarchical model for ferromagnetism, combined with Hebb's learning algorithm for a non-extensive number of stored patterns. They show that if the number of clusters l<or=4 the model is able to retrieve perfectly a family of 'descendants' together with a 'pure' embedded pattern, which appear as local minima of the free energy through a series of discontinuous transitions when the temperature (noise) is reduced. The highest critical temperature is for retrieval of the embedded patterns, that occur through a second order, continuous transition and remain as the global minima of the free energy. The 'descendants' differ from the 'ancestor' in the signs of the cluster overlaps. However, when the number of partitions in clusters increases there appear 'blurred' solutions, that consist in an arbitrary mixture of 'descendants' of a given pattern and may hinder the perfect retrieval. The number n(l) of mixed solutions increases exponentially with the cluster number, n(l) approximately=e0.45l, for large values of l.