Neural network with hierarchical clustering near saturation

Abstract

The authors perform a detailed investigation of the storage properties of a model for neural networks that exhibits the same organization into clusters as Dyson's hierarchical model (1969) for ferromagnetism, combined with Hebb's learning algorithm for an extensive number of stored patterns p= alpha N, where N is the size of the network. The authors first perform a signal-to-noise analysis, obtaining a succession of critical storage capacities for the 'ancestor' and its 'descendants' that are below the Hopfield value. Afterwards they apply the statistical mechanics formulation of Amit. Gutfreund and Sompolinsky (1987), to obtain also in this case a succession of critical storage capacities that are below the corresponding value for Hopfield's model. In both cases they consider the ratio of the critical storage capacity for the 'ancestor' to the same quantity as evaluated in Hopfield's model, and they prove rigorously that the signal-to-noise method provides a lower bound for this ratio, that is bounded from above by unity.