Computational complexity, learning rules and storage capacities: A Monte Carlo study for the binary perceptron

Abstract

We examine the storage capacity for the binary perceptron using simulated annealing. In particular, we clarify the connection between the computational complexity of learning algorithms and the attained storage capacity. From finite-size studies we obtain a critical storage capacity,α c (κ)=0.8331±0.0016, in good agreement with the replica analysis of Krauth and Mezard. However, we demonstrate that a polynomial time cooling schedule yields a vanishing storage capacity in the thermodynamic limit as predicted by the dynamical theory of Horner. Nonetheless, we show these two results may be reconciled by explicitly verifying that the learning problem for the binary perceptron is NP-complete. This investigation has been made possible by the development of an accelerated annealing algorithm.