Optimal storage properties of neural network models

Abstract

The authors calculate the number, p= alpha N of random N-bit patterns that an optimal neural network can store allowing a given fraction f of bit errors and with the condition that each right bit is stabilised by a local field at least equal to a parameter K. For each value of alpha and K, there is a minimum fraction fmin of wrong bits. They find a critical line, alpha c(K) with alpha c(0)=2. The minimum fraction of wrong bits vanishes for alpha alpha c(K). The calculations are done using a saddle-point method and the order parameters at the saddle point are assumed to be replica symmetric. This solution is locally stable in a finite region of the K, alpha plane including the line, alpha c(K) but there is a line above which the solution becomes unstable and replica symmetry must be broken.