Neural Networks as Dynamical Systems

Abstract

The activity levels of n neurons are represented by a point x in n-dimensional state space, that is, the space consisting of n-tuples x = (x1, x2,…, xn) of real numbers. Define a gain function gi = gi(xi) by gi = 0 if xi ≤ −εi; gi = 1 if xi ≥ εi; and gi is continuously differentiable and increasing for |x1| < εi. We call gi′(0) the gain of gi. For example, gi might be piecewise linear (see Fig. 0.1), that is, a ramp function, with gi(xi) = (xi+εi)/(2 εi) for |xi| < εi.