Continuous-time Neural Networks Research Articles

Novel results for global robust stability of delayed neural networks

This paper investigates the global robust convergence properties of continuous-time neural networks with discrete time delays. By employing suitable Lyapunov functionals, some sufficient conditions for the existence, uniqueness and global robust asymptotic stability of the equilibrium point are derived. The conditions can be easily verified as they can be expressed in terms of the network parameters only. Some numerical examples are also given to compare our results with previous robust stability results derived in the literature.

Cooperative Recurrent Neural Networks for the Constrained $L_{1}$ Estimator

The constrained least absolute deviation (L1) estimator is an attractive alternative to both the unconstrained L1 estimator and the least-square estimator. This paper introduces a constrained L1 method and proposes two cooperative recurrent neural networks (CRNNs) for the constrained L1 estimator. Unlike existing cooperative neural networks, the proposed two CRNNs have a novel weighting cooperation scheme to integrate individual neural network information automatically. As a special case, the proposed continuous-time CRNN includes the existing continuous- time neural network for unconstrained L1 estimator. Compared with existing continuous-time neural networks for the constrained L1 estimator, the proposed continuous-time CRNN has a lower model complexity and the finite-time convergence to the exact optimal solution without any additional condition. Furthermore, compared with existing numerical algorithms for the constrained L1 estimator, in addition to a low computational complexity, the proposed two CRNNs are suitable for parallel implementation and can deal with the L1 estimation problem with degeneracy. The proposed two CRNNs are applied to parameter estimation problems under non-Gaussian noise environments. Simulation results demonstrate that the proposed CRNNs are indeed effective in dealing with the L1 estimation problem with nonunique solutions and in obtaining a better solution than relevant algorithms.

IMPULSIVE EFFECT OF CONTINUOUS-TIME NEURAL NETWORKS UNDER PURE STRUCTURAL VARIATIONS

By using the continuation theorem of coincidence degree theory and constructing suitable Lyapunov functions, we study the existence, uniqueness and global exponential stability of periodic solution for continuous-time neural networks under pure structural variations with impulsive perturbations: [Formula: see text] The results extend earlier ones where impulses are absent. Further, using numerical simulation method the influences of the impulsive perturbations on the inherent oscillation are investigated.

Neural networks with transient state dynamics

We investigate dynamical systems characterized by a time series of distinct semi-stable activity patterns, as they are observed in cortical neural activity patterns. We propose and discuss a general mechanism allowing for an adiabatic continuation between attractor networks and a specific adjoined transient-state network, which is strictly dissipative. Dynamical systems with transient states retain functionality when their working point is autoregulated—avoiding prolonged periods of stasis or drifting into a regime of rapid fluctuations. We show, within a continuous-time neural network model, that a single local updating rule for online learning allows simultaneously (i) for information storage via unsupervised Hebbian-type learning, (ii) for adaptive regulation of the working point and (iii) for the suppression of runaway synaptic growth. Simulation results are presented; the spontaneous breaking of time-reversal symmetry and link symmetry are discussed.

Applying the Hopfield Neural Network to the Solution of the Temperature Distribution Field in Heat Conduction Problems

This article employs the continuous-time analog Hopfield neural network (CHNN) to compute the temperature distribution in one- and two-dimensional transient heat conduction problems. The relationship between the CHNN synaptic connection weights and the governing equations of the problems is established and a corresponding network connectivity circuit design scheme proposed. The CHNN algorithm is initially applied to the solution of conventional problems and is then used to solve more complicated problems involving time-varying heat flux profiles. The results confirm that the CHNN scheme provides an accurate means of solving the transient temperature distributions of heat conduction problems on a real-time basis.

The existence and exponential attractivity of κ-almost periodic sequence solution of discrete time neural networks

In the present paper, several sufficient conditions are obtained for the existence and exponential attractivity of a unique κ-almost periodic sequence solution of discrete time neural network. Our results generalize the corresponding results about almost periodic sequence solution in common sense. It is shown that discretization step κ affects the dynamical characteristics of discrete-time analogues of continuous time neural networks and exponential convergence is dependent on small discretization step size. Our results on exponential attractivity of κ-almost periodic sequence solution can provide us with relevant estimates on how precise such networks can perform during real-time computations. Finally, computer simulations are performed in the end to show the feasibility of our results.

Existence and global exponential stability of periodic solution for discrete-time BAM neural networks

The discrete-time analogues of bidirectional associative memory neural networks with periodic coefficients and distributed delays are formulated and studied. And by using the continuation theorem of coincidence degree theory, we derive the existence of periodic solution for the discrete-time BAM neural networks. And by constructing a appropriate Lyapunov-type sequence, we prove the global exponential stability of the periodic solution for the model. It is shown that the discrete-time analogues inherit the existence and global exponential stability of periodic solution for the continuous-time BAM neural networks. An example is given to illustrate the effectiveness of the obtained results.

IDENTIFICATION OF NON LINEAR FRACTIONAL SYSTEMS USING CONTINUOUS TIME NEURAL NETWORKS

Fractional systems allow us to model the processes governed by a diffusive equation. The simulation of such processes can be realized by using a non integer integrator. The aim of this paper is to estimate, in the time domain, both the non integer derivative order and the physical law of a non linear fractional system. To achieve this goal, we use Continuous Time Neural Networks. Ctnn are dynamical neural structures that differ from the classical recurrent neural networks on the use of integrator blocks rather than delay blocks. This difference allows us to access the physical law of the system rather than only having a black box model. A non-integer Ctnn is composed of a neural network and a non integer integrator. The identification stage -which consists in finding the good parameters for the neural network and for the integrator block- will be performed by using an output error identification. At the end of the procedure, a model reduction stage can be performed in order to revert from the neural network to a more realistic expression of the physical law of the process. To illustrate the method we'll give some simulation results.

Global robust stability analysis of neural networks with multiple time delays

Global robust convergence properties of continuous-time neural networks with discrete delays are studied. By employing suitable Lyapunov functionals, we derive a set of delay-independent sufficient conditions for the existence, uniqueness, and global robust asymptotic stability of the equilibrium point. The conditions can be easily verified as they can be expressed in terms of the network parameters only. Some numerical examples are given to compare our results with previous robust stability results derived in the literature. One of our main results is shown to improve and generalize a previously published result. Other results proved to establish a new set of robust stability criteria for delayed neural networks.

Global robust stability analysis of neural networks with discrete time delays

Abstract Global robust convergence properties of continuous-time neural networks with discrete delays are studied. By using a Lyapunov functional, we derive a delay independent stability condition for the existence uniqueness and global robust asymptotic stability of the equilibrium point. The condition is in terms of the network parameters only and can be easily verified. It is also shown that the obtained result improves and generalizes a previously published result.

A new regression estimator with neural network realization

A new regression estimator viewed as the solution of a strictly convex quadratic programming problem is introduced in this paper. Two recurrent neural networks in continuous-time and discrete-time respectively are proposed to solve the quadratic programming problem in real time. The continuous-time neural network is shown to have a global stability, including the global asymptotic and exponential stability. The discrete-time neural network is shown to have a global convergence with a fixed step length. This fixed step length can be independent of the regression problem size by scaling a design parameter. Since the sizes of the proposed neural networks depend only on the constraints of the optimization problems, the proposed new regression estimator can obtained by two novel neural networks with lower implementation costs than the conventional methods. Our simulation results confirm that the proposed neural networks are effective in solving various kind of regression problems.

A Neural Network Algorithm for Computing Matrix Eigenvalues and Eigenvectors

While using continuous time neural network described by the E.Oja. learning rule (Oja-N) for computing real symmetrical matrix eigenvalues and eigenvectors, the initial vector must be on R n unit hyper-sphere surface, otherwise, the network may produce limit-time overflow. In order to get over this defect, a new neural network (lyNN) algorithm is proposed. By using the analytic solution of the differential equation of lyNN, the following results are received: If initial vector belongs to a space corresponding to certain eigenvector, the lyNN equilibrium vector will converge in this space; If initial vector does not fall into the space corresponding to any eigenvector, the equilibrium vector will belong to the space spanned by eigenvectors corresponding to the maximum eigenvalue. The initial vector maximum space for the lyNN equilibrium vector will fall into space spanned by eigenvectors corresponding to any eigenvalue received. If the initial vector is perpendicular to a known eigenvector, so is the equilibrium vector. The equilibrium vector is on the hyper-sphere surface decided by the initial vector. By using the above results, a method for computing real symmetric matrix eigenvalues and eigenvectors using lyNN is proposed, the validity of this algorithm is exhibited by two examples, indicating that this algorithm does not bring about limit-time overflow. But for Oja-N, if the initial vector is outside the unit hyper-sphere and the matrix is negatively determinant, the neural network will consequentially produce limit-time overflow. Compared with other algorithms based on optimization, lyNN can be realized directly and its computing weight is lighter.

Change of memory formation according to STDP in a continuous‐time neural network model

AbstractGerstner and colleagues have proposed a learning rule in which the incrementation of synaptic weight is adjusted according to the time difference between neuron firing and spike arrival. In this study, a continuous‐time associative memory model is constructed by using a learning rule based on that idea, and the functions of the learning rule are investigated. First, a continuous‐time associative memory model is constructed on the basis of the learning rule in continuous time, in which the neuron can store memory as the synchronous firing dynamics of the neuron. A result is presented in which multiple memory patterns can be recalled simultaneously under the proposed model. Then, using the proposed learning rule, an attempt is made to compose a nesting structure formed by arbitrary memory patterns. Based on the above series of results, it is shown that the learning rule has the function of modifying the memory storage structure according to changes in the environment. © 2004 Wiley Periodicals, Inc. Syst Comp Jpn, 35(12): 57–66, 2004; Published online in Wiley InterScience (www.interscience.wiley.com). DOI 10.1002/scj.10324

Exponential Periodicity of Continuous-time and Discrete-Time Neural Networks with Delays

Exponential periodicity of continuous-time neural networks with delays is investigated. Without assuming the boundedness and differentiability of the activation functions, some new sufficient conditions ensuring existence and uniqueness of periodic solution for a general class of neural systems are obtained. Discrete-time analogue of the continuous-time system with periodic input is formulated and we study their dynamical characteristics. The exponential periodicity of the continuous-time system is preserved by the discrete-time analogue without any restriction imposed on the uniform discretization step-size.

THE KINETICS OF CRYSTALLIZATION OF TRIPALMITIN IN OLIVE OIL: AN ARTIFICIAL NEURAL NETWORK APPROACH

ABSTRACTExperimental measurements of the variation in the solid fraction during crystallization of lipid mixtures are often correlated in terms of the so‐called Avrami model. In this paper, the above model was employed to describe measurements taken during the crystallization of blends of tripalmitin in olive oil at high concentrations. Although the blends appeared to behave ideally, the Avrami model failed to describe the experimental results over the entire range of tripalmitin concentration investigated. This discrepancy appears to be correlated with the interfacial free energy. As an alternative to the description of lipid crystallization experiments, the use of continuous‐time artificial neural network (ANN) approximators is proposed. For the system studied here, the ANN successfully reproduced the experimentally observed behavior for all temperatures and tripalmitin concentrations used.

Pattern storage and processing in attractor networks with short-time synaptic dynamics

Neurophysiological experiments show that the strength of synaptic connections can undergo substantial changes on a short time scale. These changes depend on the history of the presynaptic input. Using mean-field techniques, we study how short-time dynamics of synaptic connections influence the performance of attractor neural networks in terms of their memory capacity and capability to process external signals. For binary discrete-time as well as for firing rate continuous-time neural networks, the fixed points of the network dynamics are shown to be unaffected by synaptic dynamics. However, the stability of patterns changes considerably. Synaptic depression turns out to reduce the storage capacity. On the other hand, synaptic depression is shown to be advantageous for processing of pattern sequences. The analytical results on stability, size of the basins of attraction and on the switching between patterns are complemented by numerical simulations.

A novel continuous-time neural network for realizing associative memory

A novel neural network is proposed in this paper for realizing associative memory. The main advantage of the neural network is that each prototype pattern is stored if and only if as an asymptotically stable equilibrium point. Furthermore, the basin of attraction of each desired memory pattern is distributed reasonably (in the Hamming distance sense), and an equilibrium point that is not asymptotically stable is really the state that cannot be recognized. The proposed network also has a high storage as well as the capability of learning and forgetting, and all its components can be implemented. The network considered is a very simple linear system with a projection on a closed convex set spanned by the prototype patterns. The advanced performance of the proposed network is demonstrated by means of simulation of a numerical example.

On stability of nonlinear continuous-time neural networks with delays

We utilize the Lyapunov function method to analyze stability of continuous nonlinear neural networks with delays and obtain some new sufficient conditions ensuring the globally asymptotic stability independent of delays. Three main conditions imposed on the weighting matrices are established. (i). The spectral radius ρ( M −1(| W 0|+| W τ |) K)<1. (ii). The row norm ‖ M −1(| W 0|+| W τ |) K+ P −1((| W 0|+| W τ |) KM −1) T P‖ ∞<2. (iii). μ 2( W 0)+‖ W τ ‖ 2, F <( m/ k). These three conditions are independent to each other. The delayed Hopfield network, Bidirectional associative memory network and cellular neural network are special cases of the network model considered in this paper. So we improve some previous works of other researchers.

Results concerning the absolute stability of delayed neural networks

We report on results concerning the global asymptotic stability (GAS) and absolute stability (ABST) of delay models of continuous-time neural networks. These results present sufficient conditions for GAS and in case the network has instantaneous signalling as well as delay signalling (for example, a delayed cellular neural network (DCNN)), are milder than previously known criteria; they apply to neural networks with a broad range of activation functions assuming neither differentiability nor strict monotonicity. We are therefore able to interpret the results as guarantees of absolute stability of the network with respect to the wide class of admissible activation functions. Furthermore, these results do not assume symmetry of the connection matrices. We also present a sufficient condition for absolute stability in the presence of nonconstant delays.

Winner-take-all discrete recurrent neural networks

This paper proposes a discrete recurrent neural network has simple organizations and clear dynamic behaviors. The dynamic properties of the proposed winner-take-all networks are studied detail. Simulation results are given to show network performance. Since the network model is formulated as discrete time systems, it has advantages for computer simulations over digital simulations of a continuous time neural network model. Thus they can be easily implemented in digital hardware.