Non-Lipschitzian Neural Dynamics

Abstract

The thrust of this paper is to present a new class of dynamical systems for neural computation. Our approach is motivated by an attempt to remove one of the most fundamental limitations of artificial neural networks — their rigid behavior compared with even the simplest biological systems. It exploits the concept of terminal attractors and repellers. We demonstrate that non-Lipschitzian dynamics, based upon the failure of the Lipschitz condition at repellers, exhibits a new qualitative effect — a multi-choice response to periodic external excitations. This property enables the construction of a substantially new class of dynamical systems — the unpredictable systems, represented in the form of coupled activation and learning dynamical equations, which exhibit two pathological characteristics. Firstly, such systems have zero Jacobian. As a result of that, they have an infinite number of equilibrium points which occupy curves, surfaces or hypersurfaces. Secondly, at all these equilibrium points the Lipschitz condition fails, so the equilibrium points become terminal attractors or repellers depending upon the sign of the periodic excitation. Both of these pathological characteristics result in multi-choice responses and lead to unpredictability. We show that the unpredictable systems can be controlled by sign strings, which uniquely define the systems behavior by specifying the direction of the motions at the critical points. By changing the combinations of signs in the code strings a system can reproduce any prescribed behavior to a prescribed accuracy. That is why such unpredictable systems driven by sign strings are extremely flexible and can be exploited as a powerful tool for complex pattern recognition.