Perceptrons of large weight

Abstract

A threshold gate is a linear combination of input variables with integer coefficients (weights). It outputs 1 if the sum is positive. The maximum absolute value of the coefficients of a threshold gate is called its weight. A degree-d perceptron is a Boolean circuit of depth 2 with a threshold gate at the top and any Boolean elements of fan-in at most d at the bottom level. The weight of a perceptron is the weight of its threshold gate. For any constant d ≥ 2 independent of the number of input variables n, we construct a degree-d perceptron that requires weights of at least $$ n^{\Omega (n^d )} $$; i.e., the weight of any degree-d perceptron that computes the same Boolean function must be at least $$ n^{\Omega (n^d )} $$. This bound is tight: any degree-d perceptron is equivalent to a degree-d perceptron of weight $$ n^{O(n^d )} $$. For the case of threshold gates (i.e., d = 1), the result was proved by Hastad in [2]; we use Hastad’s technique.