Solving nonlinear classification and regression problems by wave interference with a single-layer complex-valued perceptron

Abstract

The classical single-layer perceptron, while being able to solve linearly separable problems, fails when applied to nonlinear problems such as the XOR problem and thus is unable to represent a universal gate. In this paper, we show how a modification of the activation function of the perceptron can overcome this restriction and provide a geometric explanation of this finding. While these modifications have a negative effect on the convergence properties of real-valued networks, the introduction of complex weights and training only their phases enables wave-interference mechanisms to take place that can guarantee convergence for gradient-descent training, which is shown both analytically and by computer simulations. It is further demonstrated that the framework can be extended and applied to other nonlinear classification and regression problems. The relevance of the results with regard to artificial and biological neural networks is discussed.