The Duality Between the Perceptron Algorithm and the von Neumann Algorithm

Abstract

The perceptron and the von Neumann algorithms were developed to solve linear feasibility problems. In this paper, we investigate and reveal the duality relationship between these two algorithms. The specific forms of linear feasibility problems solved by the perceptron and the von Neumann algorithms are a pair of alternative systems by the Farkas Lemma. A solution of one problem serves as an infeasibility certificate of its alternative system. Further, we adapt an Approximate Farkas Lemma to interpret the meaning of an approximate solution from its alternative perspective. The Approximate Farkas Lemma also enables us to derive bounds for the distance to feasibility or infeasibility from approximate solutions of the alternative systems. Based on these observations, we interpret variants of the perceptron algorithm as variants of the von Neumann algorithm and vice versa, as well as transit the complexity results from one family to the other.