Delay Recurrent Neural Networks Research Articles

Complete Stability of Delayed Recurrent Neural Networks With New Wave-Type Activation Functions.

Activation functions have a significant effect on the dynamics of neural networks (NNs). This study proposes new nonmonotonic wave-type activation functions and examines the complete stability of delayed recurrent NNs (DRNNs) with these activation functions. Using the geometrical properties of the wave-type activation function and subsequent iteration scheme, sufficient conditions are provided to ensure that a DRNN with n neurons has exactly (2m + 3)n equilibria, where (m + 2)n equilibria are locally exponentially stable, the remainder (2m + 3)n - (m + 2)n equilibria are unstable, and a positive integer m is related to wave-type activation functions. Furthermore, the DRNN with the proposed activation function is completely stable. Compared with the previous literature, the total number of equilibria and the stable equilibria significantly increase, thereby enhancing the memory storage capacity of DRNN. Finally, several examples are presented to demonstrate our proposed results.

Time delay recurrent neural networks via PID control law to complex dynamic systems

This paper presents an extension to the theory on the analysis of stability of complex systems with time delay of fractional order; the previous study is based on the theory of functions of Lyapunov–Krasovskii and the fractional-order PID control law. The analytical results are illustrated by the simulation of fractional nonlinear systems interconnected with time delay which are forced to follow complex trajectories as chaotic systems.

Stability Analysis of Delayed Recurrent Neural Networks via a Quadratic Matrix Convex Combination Approach.

This brief addresses the stability analysis problem of a class of delayed recurrent neural networks (DRNNs). In previously published studies, the slope information of activation function (SIAF) is just reflected in three slope information matrices, i.e., the upper and lower boundary matrices and the maximum norm matrix. In practice, there are 2n possible combination cases on the slope information matrices. To exploit more information about SIAF, first, an activation function separation method is proposed to derive n slope-information-based uncertainties (SIBUs) containing SIAF; second, a quadratic matrix convex combination approach is proposed to dispose n SIBUs using 2n combination slope information matrices. Third, a stability criterion with less conservatism is established based on the proposed approach. Finally, two simulation examples are used to testify the validity of theoretical results.

Event-Triggered Regulation for Enhanced Synchronization of Recurrent Neural Networks using Time-Varying Delay

This research is primarily concerned having the synchronization issue of time-varying delay recurrent neural networks (RNNs) with event-triggered regulation (ETR). ETR may legitimately reduce the number of control activities performed and the amount of energy expended. Via the L-K functional, Schur complement lemma, linear matrix inequalities (LMIs), and other analytic techniques, a sufficient condition for the synchronization of the drive-response system is achieved. Consequently, a more sensible and effective controller is implemented. Using a Matlab-made example and graph, we can show that our inferred results are correct and we can ensure the rationality of the set controller.

Multistability analysis of delayed recurrent neural networks with a class of piecewise nonlinear activation functions

This paper studies the multistability of delayed recurrent neural networks (DRNNs) with a class of piecewise nonlinear activation functions. The coexistence as well as the stability of multiple equilibrium points (EPs) of DRNNs are proved. With the Brouwer’s fixed point theorem as well as the Lagrange mean value theorem, it is obtained that under some conditions, the n-neuron DRNNs with the proposed activation function can have at least 5n EPs and 3n of them are locally stable. Compared with the DRNNs with sigmoidal activation functions, DRNNs with this kind of activation function can have more total EPs and more locally stable EPs. It implies that when designing DRNNs with the proposed activation function to apply in associative memory, it can have an even larger storage capacity. Furthermore, it is obtained that there exists a relationship between the number of the total EPs/stable EPs and the frequency of the sinusoidal function in the proposed activation function. Last, the above obtained results are extended to a more general case. It is shown that, DRNNs with the extended activation function can have (2k+1)n EPs, (k+1)n of which are locally stable, therein k is closely related to the frequency of the sinusoidal function in the extended activation function. Two simulation examples are given to verify the correctness of the theoretical results.

SYNCHRONIZATION BETWEEN INTEGER AND FRACTIONAL ORDER DELAY NEURAL NETWORKS SYSTEM

The purpose is to present a method for synchronizing a recurrent neural networks system between integer and fractional-order order delay by active sliding mode control . The Active Sliding Mode Control (ASMC) scheme is used to solve the synchronization problem between the integer-order delayed recurrent neural networks system via active sliding mode control (IoDRNNASM) systems and the fractional-order delay recurrent neural networks system via active sliding mode control (FoDRNNASM) system based on the Lyapunov direct fractional method (LDFM). To explore the behavior of the IoDRNNASM systems and the FoDRNNASM systems, we performed the technique of numerical simulations using MATLAB software to prove the feasibility and strength of the archived outcomes. This concept can also be enhanced with the implementation of double encryption using RSA encryption to secure communication. Because we expected in the future that this enhanced concept will strengthen and increase the network security capabilities that will provide powerful protection in secure communications.

Time Delay Recurrent Neural Network for Speech Recognition

In Automatic Speech Recognition(ASR), Time Delay Neural Network (TDNN) has been proven to be an efficient network structure for its strong ability in context modeling. In addition, as a feed-forward neural architecture, it is faster to train TDNN, compared with recurrent neural networks such as Long Short-Term Memory (LSTM). However, different from recurrent neural networks, the context in TDNN is carefully designed and is limited. Although stacking Long Short-Term Memory (LSTM) together with TDNN in order to extend the context information have been proven to be useful, it is too complex and is hard to train. In this paper, we focus on directly extending the context modeling capability of TDNNs by adding recurrent connections. Several new network architectures were investigated. The results on the Switchboard show that the best model significantly outperforms the base line TDNN system and is comparable with TDNN-LSTM architecture. In addition, the training process is much simpler than that of TDNN-LSTM.

Further Improvement on Delay-range-dependent Stability Criteria for Delayed Recurrent Neural Networks with Interval Time-varying Delays

This paper considers the problem of delay-range-dependent stability analysis for a class of delayed recurrent neural networks (DRNNs) with a time-varying delay in a range. Based on the Lyapunov-Krasovskii functional and derive the time derivative of this with integral inequality approach (IIA), new delay-dependent stability criteria for the system are established in terms of linear matrix inequalities (LMIs), which can be solved easily by various efficient convex optimization algorithms. Information about the lower bound of the delay is fully used in the Lyapunov functional. Two examples are given to illustrate the effectiveness and the reduced conservatism of the proposed results.

New Criteria on Exponential Stability for Stochastic Delay Differential Systems Based on Vector Lyapunov Function

This paper established the vector Razumikhin-type theorem on exponential stability for stochastic functional differential systems by the stochastic analysis techniques and the property of ${M}$ -Matrix. Several novel stability criteria with vector $ {\mathscr {L}}$ -operator differential inequalities for stochastic delay differential systems, especially some of which includes the cross-item, were obtained by means of the vector Razumikhin theorem. By applying these new results, the robustness of global exponential stability of delay recurrent neural networks to random disturbed has been analyzed.

Th Moment Exponential Input-to-State Stability of Delayed Recurrent Neural Networks With Markovian Switching via Vector Lyapunov Function.

In this paper, the th moment input-to-state exponential stability for delayed recurrent neural networks (DRNNs) with Markovian switching is studied. By using stochastic analysis techniques and classical Razumikhin techniques, a generalized vector -operator differential inequality including cross item is obtained. Without additional restrictive conditions on the time-varying delay, the sufficient criteria on the th moment input-to-state exponential stability for DRNNs with Markovian switching are derived by means of the vector -operator differential inequality. When the input is zero, an improved criterion on exponential stability is obtained. Two numerical examples are provided to examine the correctness of the derived results.

Mean-square global exponential stability in Lagrange sense for delayed recurrent neural networks with Markovian switching

The paper discusses the mean-square global exponential stability in Lagrange sense for delayed recurrent neural networks (DRNNs) with Markovian switching. Two different types of activation functions are considered, which include both bounded and unbounded activation functions. By using the vector Lyapunov function and stochastic analysis technique, we establish two L-operator differential inequalities. By employing the two L-operator differential inequalities and vector Lyapunov functions methods, we provide easily verifiable criteria for Lagrange stability in mean-square sense of DRNNs with Markovian switching. Finally, two numerical examples are given to illustrate the efficiency of the derived results.

Delay-Dependent Global Exponential Stability for Delayed Recurrent Neural Networks.

This paper deals with the global exponential stability for delayed recurrent neural networks (DRNNs). By constructing an augmented Lyapunov-Krasovskii functional and adopting the reciprocally convex combination approach and Wirtinger-based integral inequality, delay-dependent global exponential stability criteria are derived in terms of linear matrix inequalities. Meanwhile, a general and effective method on global exponential stability analysis for DRNNs is given through a lemma, where the exponential convergence rate can be estimated. With this lemma, some global asymptotic stability criteria of DRNNs acquired in previous studies can be generalized to global exponential stability ones. Finally, a frequently utilized numerical example is carried out to illustrate the effectiveness and merits of the proposed theoretical results.

Stability in distribution of stochastic delay recurrent neural networks with Markovian switching

This paper investigates the stability in distribution of stochastic delay recurrent neural networks with Markovian switching. Using Lyapunov function and stochastic analysis techniques, sufficient conditions on the stability in distribution are given. For such recurrent neural networks, it reveals that the limit distribution of transition probability for segment process associated with solution process is indeed a unique ergodic invariant probability measure. Moreover, a numerical example is also provided to demonstrate the effectiveness and applicability of the theoretical results.

Robust Exponential Stabilization of Stochastic Delay Interval Recurrent Neural Networks with Distributed Parameters and Markovian Jumping by Using Periodically Intermittent Control

We consider a class of stochastic delay recurrent neural networks with distributed parameters and Markovian jumping. It is assumed that the coefficients in these neural networks belong to the interval matrices. Several sufficient conditions ensuring robust exponential stabilization are derived by using periodically intermittent control and Lyapunov functional. The obtained results are very easy to verify and implement, and improve the existing results. Finally, an example with numerical simulations is given to illustrate the presented criteria.

Robust Control, Optimization, and Applications to Markovian Jumping Systems

Robust Control, Optimization, and Applications to Markovian Jumping Systems

Robustness analysis for connection weight matrices of global exponential stable time varying delayed recurrent neural networks

This paper analyzes the robustness of global exponential stability of delayed recurrent neural networks (DRNNs) subject to parameter uncertainty in connection weight matrices. Given a globally exponentially stable DRNNs, the problem to be addressed herein is how much parameter uncertainty in the connection weight matrices that the neural network can remain to be globally exponentially stable. We characterize the upper bounds of the parameter uncertainty for the DRNNs to sustain global exponential stability. The upper bounds of parameter uncertainty intensity are characterized by using transcendental equations. Moreover, we prove theoretically that, for globally exponentially stable DRNNs, if additive parameter uncertainties in connection weight matrices are smaller than the derived supper bounds arrived at here, then the perturbed DRNNs are guaranteed to also be globally exponentially stable. A numerical example is provided to illustrate the theoretical results.

Exponential Stability and Numerical Methods of Stochastic Recurrent Neural Networks with Delays

Exponential stability in mean square of stochastic delay recurrent neural networks is investigated in detail. By using Itô’s formula and inequality techniques, the sufficient conditions to guarantee the exponential stability in mean square of an equilibrium are given. Under the conditions which guarantee the stability of the analytical solution, the Euler-Maruyama scheme and the split-step backward Euler scheme are proved to be mean-square stable. At last, an example is given to demonstrate our results.

Adaptive Neural Networks for Flood Routing in River Systems

A methodology based on adaptive ANN models is proposed for flood routing in river systems. The proposed methodology is capable of modeling both converging and diverging river networks. A Multilayer Perceptron Network (MLP), a Recurrent Neural Network (RNN), a Time Delay Neural Network (TDNN) and a Time Delay Recurrent Neural Network (TDRNN) are applied in this study. An Adaptive training procedure based on the Forgetting Factor (FF) approach is used to train ANNs models. The methodology provides a lead time equal to travel time for the flood estimation downstream of the river. The performances of the models are tested within the two distinctive parts of the Karoon River in south-west Iran. The first case study uses synthetic floods generated by the HEC-RAS hydraulic model; the second one uses observed floods. Besides, the Muskingum routing method is used in the second case study to be compared with the results of ANN models. Overall, the results demonstrated that the proposed methodology performs well considering goodness-of-fit criteria. Moreover, the dynamic neural networks outperform the static MLP and the Muskingum model.

Comparison of neural network methods for infilling missing daily weather records

Summary Accurate estimate of missing daily precipitation data remains a difficult task particularly for large watersheds with coarse rain gauge network. Reliable and representative precipitation time series are essential for any rainfall–runoff model calibration as well as for setting-up any downscaling model for hydrologic impact study of climate change. This study investigates six different types of artificial neural networks namely the multilayer perceptron (MLP) network and its variations (the time-lagged feedforward network (TLFN)), the generalized radial basis function (RBF) network, the recurrent neural network (RNN) and its variations (the time delay recurrent neural network (TDRNN)), and the counterpropagation fuzzy-neural network (CFNN) along with different optimization methods for infilling missing daily total precipitation records and daily extreme temperature series. Daily precipitation and temperature records from 15 weather stations located within the Gatineau watershed in northeastern Canada, are used to evaluate the accuracy of the different models for infilling data gaps of daily precipitation and daily extreme temperatures. The experiment results suggest that the MLP, the TLFN and the CFNN can provide the most accurate estimates of the missing precipitation values. However, overall, the MLP appears the most effective at infilling missing daily precipitation values. Furthermore, the MLP also appears the most suitable for infilling missing daily maximum and minimum temperature values. The CFNN is similar to the MLP at infilling missing daily maximum temperature, however, it is less effective at estimating minimum temperature. The experiment results show that the dynamically driven networks (RNN and TDRNN) are the less suitable for infilling both the daily precipitation and the extreme temperature records, whereas the RBF appears fairly suitable only for estimating maximum and minimum temperature.