Nine switch-affine neurons suffice for Turing universality

Abstract

In a previous work Pollack showed that a particular type of heterogeneous processor network is Turing universal. Siegelmann and Sontag (1991) showed the universality of homogeneous networks of first-order neurons having piecewise-linear activation functions. Their result was generalized by Kilian and Siegelmann (1996) to include various sigmoidal activation functions. Here we focus on a type of high-order neurons called switch-affine neurons, with piecewise-linear activation functions, and prove that nine such neurons suffice for simulating universal Turing machines.