Storage capacity of a constructive learning algorithm

Abstract

Upper and lower bounds for the typical storage capacity of a constructive algorithm, the tilinglike learning algorithm for the parity machine (Biehl M and Opper M 1991 Phys. Rev.A 446888), are determined in the asymptotic limit of large training set sizes. The properties of a perceptron with threshold, learning a training set of patterns having a biased distribution of targets, needed as an intermediate step in the capacity calculation, are determined analytically. The lower bound for the capacity, determined with a cavity method, is proportional to the number of hidden units. The upper bound, obtained with the hypothesis of replica symmetry, is close to the one predicted by Mitchinson and Durbin (1989 Biol. Cybern.60345).