Weight space structure and analysis using a finite replica number in the Isingperceptron

Abstract

The weight space of the Ising perceptron, in which a set of random patterns isstored, is examined using the generating function of the partition functionϕ(n) = (1/N)log[Zn] as the dimension ofthe weight vector N tends to infinity, where Z is the partition function and represents the configurational average. We utilizeϕ(n) for two purposes, depending on the value of the ratioα = M/N, whereM is the number ofrandom patterns. For α<αs = 0.833..., we employ ϕ(n), in conjunction with Parisi’s one-step replica symmetry breaking scheme in the limit of , to evaluate the complexity that characterizes the number of disjoint clusters of weightsthat are compatible with a given set of random patterns, which indicates that, intypical cases, the weight space is equally dominated by a single large cluster ofexponentially many weights and exponentially many small clusters of a single weight. Forα>αs, on theother hand, ϕ(n) is used to assess the rate function of a small probability that a given set of random patterns is atypicallyseparable by the Ising perceptrons. We show that the analyticity of the rate function changes atα = αGD = 1.245..., which implies that the dominant configuration of the atypically separable patternsexhibits a phase transition at this critical ratio. Extensive numerical experiments areconducted to support the theoretical predictions.