Maximal stability in unsupervised learning

Abstract

A perceptron can classify a set of random patterns into two groups. The maximal gap between the two groups is calculated using replica theory. The replica-symmetric solution is shown to be unstable and gives results which are qualitatively different from the one-step replica-symmetry breaking solution (RSB1). The results of a calculation without the replica method are given, yielding an exact upper bound for the gap which almost coincides with the RSB1 solution. Finding the classification of a set of patterns with a maximal gap is a difficult combinatorial optimization problem. An algorithm is developed which gives large gaps.