Continuous-time Neural Networks Research Articles

Equilibrium characterization of dynamical neural networks and a systematic synthesis procedure for associative memories

Several novel results concerning the characterization of the equilibrium conditions of a continuous-time dynamical neural network model and a systematic procedure for synthesizing associative memory networks with nonsymmetrical interconnection matrices are presented. The equilibrium characterization focuses on the exponential stability and instability properties of the network equilibria and on equilibrium confinement, viz., ensuring the uniqueness of an equilibrium in a specific region of the state space. While the equilibrium confinement result involves a simple test, the stability results given obtain explicit estimates of the degree of exponential stability and the regions of attraction of the stable equilibrium points. Using these results as valuable guidelines, a systematic synthesis procedure for constructing a dynamical neural network that stores a given set of vectors as the stable equilibrium points is developed.

A new feedback neural network with supervised learning

A model is introduced for continuous-time dynamic feedback neural networks with supervised learning ability. Modifications are introduced to conventional models to guarantee precisely that a given desired vector, and its negative, are indeed stored in the network as asymptotically stable equilibrium points. The modifications entail that the output signal of a neuron is multiplied by the square of its associated weight to supply the signal to an input of another neuron. A simulation of the complete dynamics is then presented for a prototype one neuron with self-feedback and supervised learning; the simulation illustrates the (supervised) learning capability of the network.

Dispersive propagation skew effects in iterative neural networks

During communication between neurons in a continuous-time analog neural network, propagation skew typically varies from neuron pair to neuron pair. For no dispersion, S. Oh et al. (Opt. Eng., vol.28, p.526-32, 1989) previously demonstrated that the steady-state performance of an iterative neural network is not affected if the combination of the neural network's weights and neural nonlinearity is contractive. This result is extended to the case of dispersive skew. It is shown that, under nearly the same conditions, the same steady-state result will occur in the neural network in the presence of dispersive skew.

Analysis and synthesis of a class of discrete-time neural networks described on hypercubes

A qualitative analysis is presented for a class of synchronous discrete-time neural networks defined on hypercubes in the state space. Analysis results are utilized to establish a design procedure for associative memories to be implemented on the present class of neural networks. To demonstrate the storage ability and flexibility of the synthesis procedure, several specific examples are considered. The design procedure has essentially the same desirable features as the results of J. Li et al. (1988, 1989) for continuous-time neural networks. For a given system dimension, networks designed by the present method may have the ability to store more patterns (as asymptotically stable equilibria) than corresponding discrete-time networks designed by other techniques. The design method guarantees the storage of all the desired patterns as asymptotically stable equilibrium points. The present method provides guidelines for reducing the number of spurious states and for estimating the extent of the patterns' domains of attraction. The present results provide a means of implementing neural networks by serial processors and special digital hardware.

Local minimum escape using thermodynamic properties of neural networks

The article presents a new thermodynamic model of continuous-time neural networks. The model is based on the thermodynamic fluctuations theory. The differential equations describing the behavior of the network in time are derived and analyzed. The analysis shows that the dynamics of the network is characterized by the time behavior of the Helmholtz free energy of the system. The possibility to control this behavior by the time-dependent temperature schedule is also discussed. The emphasis of the article is on the local energy minima escape in optimization problems. Two simulations of the Traveling Salesman Problem (TSP) have been used to demonstrate the discussed approach.

Chaos in neural systems

The occurence of chaos in continuous-time neural network models is demonstrated through two examples, (i) a single neuron with an unusual kind of periodic input and (ii) a randomly connected network of 26 neurons with 7 incoming synapses per neuron, half the neurons being subject to a steady external stimulus. The former case is susceptible to exhaustive analysis, while the latter, discovered by computer simulation, is more attuned to neurobiological reality.