Dynamic Recurrent Neural Network Research Articles

Modelling flexible robot dynamics using discrete-time dynamic recurrent neural networks

The identification of a general class of multi-input multi-output (MIMO) discrete-time nonlinear systems expressed in state space form is studied in this paper using dynamic recurrent neural networks (DRNNs). A discrete-time DRNN, which is represented by a set of parameterized nonlinear difference equations and has a universal approximation capability, is proposed for modelling unknown discrete-time nonlinear systems. A dynamic back propagation learning algorithm which carries out the modelling task using the input-output data is discussed. The proposed scheme is used for modelling the nonlinear dynamics of a single-link robot with a flexible joint.

Design and test of a multi‐input multi‐output adaptive active flutter suppression system based on neural network for a wing model

AbstractThis paper describes the application of a recurrent dynamic neural network to the problem of adaptive active flutter suppression. A significant feature of the neural controller is its application to an unstable and non‐minimum phase system. The development of the system and its experimental verification are carried out on a wing model with two control surfaces, one on the leading and one at the trailing edge, and two accelerometers. The controller combines a first network for the identification of the variables to be controlled, i.e. the two wing tip accelerations at the leading and trailing edge, with a second network, that commands the deflection required to the control surfaces. It is used for the control task using the output generated by the first network. Adaptivity is obtained by maintaining an on‐line training on both of the neural networks. Extensive preliminary numerical simulations have permitted the authors to define the values of the design parameters aimed at the achievement of an optimal compromise between computational requirements and system performances. The results of the application to the real wing model have shown the effectiveness of such a controller in adapting to different operational conditions, extending flutter free envelope of the wing model up to a speed 30% higher than the uncontrolled flutter onset.

Computational results on recurrent dynamic neural network for signal analysis

Computational results on recurrent dynamic neural network for signal analysis

Robust recurrent dynamic neural network for solving linear and nonlinear signal representation problems

This paper presents a new technique for signal processing and representation based on a continuous recurrent dynamic neural network (RDNN). The internal structure of the RDNN consists of output neurons feedback and feedforward arrays of integrators, linear gains, and bipolar sigmoid functions. The neural network parameters include synaptic weights calculated as mutual inner products of basis signals, and bias inputs calculated as inner products of these vectors with the measured signal. The RDNN was applied to representing motor vibration data using functions basis. Training the RDNN consisted of processing the first two cycles of a vibration record of eight cycles with the purpose of giving as outputs the appropriate expansion coefficients. The trained RDNN was then used to predict succeeding vibration cycles considered as testing data. Results were very satisfactory, especially in the presence of noise, where the RDNN showed little sensitivity and considerable robustness. The effect of varying the RDNN parameters was investigated, and it was found that for a certain set of optimal parameters, the RDNN performance was more satisfactory than the algebraic pseudo-inverse method. A scaling factor was introduced in the expression of the RDNN parameters and had a positive effect on reducing the approximation error between original and reconstructed waveforms, especially in the presence of noise. The RDNN was extended through the Newton-Raphson algorithm in order to handle dependent nonlinearities. This technique turned out useful especially in case of ill-posed signal problems. The various simulations presented in this paper have shown that the proposed recurrent dynamic neural network offers an alternative to traditional methods when dealing with noise corrupted data, and ill-conditioned and uncertain systems of linear and nonlinear equations.

神経回路網ダイナミクスの複雑さの制御法

神経回路網ダイナミクスの複雑さの制御法

Robust techniques for independent component analysis (ICA) with noisy data

In this contribution, we propose approaches to independent component analysis (ICA) when the measured signals are contaminated by additive noise. We extend existing adaptive algorithms with equivariant properties in order to considerably reduce the bias in the demixing matrix caused by measurement noise. Moreover, we describe a novel recurrent dynamic neural network for simultaneous estimation of the unknown mixing matrix, blind source separation, and reduction of noise in the extracted output signals. We discuss the optimal choice of nonlinear activation functions for various noise distributions assuming a generalized Gaussian-distributed noise model. Computer simulations of a selected approach are provided that confirm its usefulness and excellent performance.

Dynamic Recurrent Neural Networks for Stable Adaptive Control of Wing Rock Motion

Wing Rock or limit cycle oscillation (LCO) in an aircraft is often caused by aerodynamic nonlinearities or mechanical hysteresis. A dynamic recurrent radial basis function neural networ~s (DR-RBF) are proposed for modelling the nonlinear hysteresis. The concept based on Preisach hysteresis model is used in the design of neural networks. The structure and memory mechanism in the hysteresis model is analogous to parallel connectivity and memory formation in neural networks. Adaptive control law based on minimisation of the energy function to ensure stability in the Lyapunov sense is presented.

On the macroscopic description of recurrent neural network dynamics

It is expected that the dynamics of recurrent neural networks can be described by a set of macroscopic order parameters. For such descriptions to exist, macroscopic specification of microscopic states of neural networks during the dynamics has to be possible. We study this problem for the case of Coolen-Sherrington (CS) theory, which successfully describes qualitative aspects of the dynamics of the Hopfield-type recurrent neural network. We show that the macroscopic specification is incomplete in CS theory by providing direct evidence that the equipartitioning assumption employed by CS theory prevents the theory from describing the dynamics quantitatively.

A recurrent dynamic neural network for noisy signal representation

This paper presents a new recurrent dynamic neural network approach to solve noisy signal representation and processing problems. Essentially, the neural network solves, in a systematic way, for the sets of representation coefficients required to model a given signal in terms of basis elementary signals. The network converges by seeking the minimum energy states. The perceived advantages over traditional approaches are in robustness of computation and ability to handle time-varying noisy signals.

A dynamic recurrent neural-network-based adaptive observer for a class of nonlinear systems

An adaptive observer for a class of single-input single-output (SISO) nonlinear systems is proposed using a generalized dynamic recurrent neural network (DRNN). The neural-network (NN) weights are tuned on-line, with no off-line learning required. No exact knowledge of non-linearities in the observed system is required. Furthermore, no linearity with respect to unknown system parameters is assumed. The DRNN observer does not assume that nonlinearities in the system are restricted to the system output only. The overall adaptive observer scheme is shown to be uniformly ultimately bounded. Simulation results have verified the performance of the DRNN observer.

Penalty function approach to recurrent neural network dynamics

The Hopfield dynamics for recurrent neural networks minimizes a certain quadratic form on the unit hypercube. I show here how the dynamical system can be derived from a standard method of optimization theory. I use the method to give a precise meaning to nonsymmetric interactions, and I discuss the possibility of introducing other types of dynamics.

Emergence of clusters in the hidden layer of a dynamic recurrent neural network.

The neural integrator of the oculomotor system is a privileged field for artificial neural network simulation. In this paper, we were interested in an improvement of the biologically plausible features of the Arnold-Robinson network. This improvement was done by fixing the sign of the connection weights in the network (in order to respect the biological Dale's Law). We also introduced a notion of distance in the network in the form of transmission delays between its units. These modifications necessitated the introduction of a general supervisor in order to train the network to act as a leaky integrator. When examining the lateral connection weights of the hidden layer, the distribution of the weights values was found to exhibit a conspicuous structure: the high-value weights were grouped in what we call clusters. Other zones are quite flat and characterized by low-value weights. Clusters are defined as particular groups of adjoining neurons which have strong and privileged connections with another neighborhood of neurons. The clusters of the trained network are reminiscent of the small clusters or patches that have been found experimentally in the nucleus prepositus hypoglossi, where the neural integrator is located. A study was conducted to determine the conditions of emergence of these clusters in our network: they include the fixation of the weight sign, the introduction of a distance, and a convergence of the information from the hidden layer to the motoneurons. We conclude that this spontaneous emergence of clusters in artificial neural networks; performing a temporal integration, is due to computational constraints, with a restricted space of solutions. Thus, information processing could induce the emergence of iterated patterns in biological neural networks.

Improved Identification of Complex Temporal Systems with Dynamic Recurrent Neural Networks. Application to the Identification of Electromyography and Human Arm Trajectory Relationship

We propose a new approach based on dynamic recurrent neural networks (DRNN) to identify, in humans, the relationship between the muscle electromyographic (EMG) activities and the arm kinematics during the drawing of the figure eight using an extended arm. After learning, the DRNN simulations showed the efficiency of the model. We demonstrated its generalization ability to draw unlearned movements. We developed a test of its physiological plausibility by computing the error velocity vectors when small artificial lesions in the EMG signals were created. These lesion experiments demonstrated that the DRNN has identified the preferential direction of the physiological action of the studied muscles. The network also identified neural constraints such as the covariation between geometrical and kinematics parameters of the movement. This suggests that the information of raw EMG signals is largely representative of the kinematics stored in the central motor pattern. Moreover, the DRNN approach will allow one to dissociate the feedforward command (central motor pattern) and the feedback effects from muscles, skin and joints.

Dynamic recurrent neural networks: a dynamical analysis

In this paper, we explore the dynamical features of a neural network model which presents two types of adaptative parameters: the classical weights between the units and the time constants associated with each artificial neuron. The purpose of this study is to provide a strong theoretical basis for modeling and simulating dynamic recurrent neural networks. In order to achieve this, we study the effect of the statistical distribution of the weights and of the time constants on the network dynamics and we make a statistical analysis of the neural transformation. We examine the network power spectra (to draw some conclusions over the frequential behaviour of the network) and we compute the stability regions to explore the stability of the model. We show that the network is sensitive to the variations of the mean values of the weights and the time constants (because of the temporal aspects of the learned tasks). Nevertheless, our results highlight the improvements in the network dynamics due to the introduction of adaptative time constants and indicate that dynamic recurrent neural networks can bring new powerful features in the field of neural computing.

Input/output linearization using dynamic recurrent neural networks

Abstract This paper brings together two areas of research that have received considerable attention during the last years, namely feedback linearization and neural networks. A proposition that guarantees the Input/Output (I/O) linearization of nonlinear control affine systems with Dynamic Recurrent Neural Networks (DRNNs) is formulated and proved. The proposition and the linearization procedure are illustrated with the simulation of a single link manipulator.

Approximation of discrete-time state-space trajectories using dynamic recurrent neural networks

In this note, the approximation capability of a class of discrete-time dynamic recurrent neural networks (DRNN's) is studied. Analytical results presented show that some of the states of such a DRNN described by a set of difference equations may be used to approximate uniformly a state-space trajectory produced by either a discrete-time nonlinear system or a continuous function on a closed discrete-time interval. This approximation process, however, has to be carried out by an adaptive learning process. This capability provides the potential for applications such as identification and adaptive control. >

Dynamic recurrent neural network for system identification and control

A dynamic recurrent neural network (DRNN) that can be viewed as a generalisation of the Hopfield neural network is proposed to identify and control a class of control affine systems. In this approach, the identified network is used in the context of the differential geometric control to synthesise a state feedback that cancels the nonlinear terms of the plant yielding a linear plant which can then be controlled using a standard PID controller.

Dynamical recurrent neural networks--towards environmental time series prediction.

Dynamical Recurrent Neural Networks (DRNN) (Aussem 1995a) are a class of fully recurrent networks obtained by modeling synapses as autoregressive filters. By virtue of their internal dynamic, these networks approximate the underlying law governing the time series by a system of nonlinear difference equations of internal variables. They therefore provide history-sensitive forecasts without having to be explicitly fed with external memory. The model is trained by a local and recursive error propagation algorithm called temporal-recurrent-backpropagation. The efficiency of the procedure benefits from the exponential decay of the gradient terms backpropagated through the adjoint network. We assess the predictive ability of the DRNN model with meterological and astronomical time series recorded around the candidate observation sites for the future VLT telescope. The hope is that reliable environmental forecasts provided with the model will allow the modern telescopes to be preset, a few hours in advance, in the most suited instrumental mode. In this perspective, the model is first appraised on precipitation measurements with traditional nonlinear AR and ARMA techniques using feedforward networks. Then we tackle a complex problem, namely the prediction of astronomical seeing, known to be a very erratic time series. A fuzzy coding approach is used to reduce the complexity of the underlying laws governing the seeing. Then, a fuzzy correspondence analysis is carried out to explore the internal relationships in the data. Based on a carefully selected set of meteorological variables at the same time-point, a nonlinear multiple regression, termed nowcasting (Murtagh et al. 1993, 1995), is carried out on the fuzzily coded seeing records. The DRNN is shown to outperform the fuzzy k-nearest neighbors method.

Adaptative time constants improve the prediction capability of recurrent neural networks

Classical statistical techniques for prediction reach their limitations in applications with nonlinearities in the data set; nevertheless, neural models can counteract these limitations. In this paper, we present a recurrent neural model where we associate an adaptative time constant to each neuron-like unit and a learning algorithm to train these dynamic recurrent networks. We test the network by training it to predict the Mackey-Glass chaotic signal. To evaluate the quality of the prediction, we computed the power spectra of the two signals and computed the associated fractional error. Results show that the introduction of adaptative time constants associated to each neuron of a recurrent network improves the quality of the prediction and the dynamical features of a neural model. The performance of such dynamic recurrent neural networks outperform time-delay neural networks.

Gradient calculations for dynamic recurrent neural networks: a survey

Surveys learning algorithms for recurrent neural networks with hidden units and puts the various techniques into a common framework. The authors discuss fixed point learning algorithms, namely recurrent backpropagation and deterministic Boltzmann machines, and nonfixed point algorithms, namely backpropagation through time, Elman's history cutoff, and Jordan's output feedback architecture. Forward propagation, an on-line technique that uses adjoint equations, and variations thereof, are also discussed. In many cases, the unified presentation leads to generalizations of various sorts. The author discusses advantages and disadvantages of temporally continuous neural networks in contrast to clocked ones continues with some "tricks of the trade" for training, using, and simulating continuous time and recurrent neural networks. The author presents some simulations, and at the end, addresses issues of computational complexity and learning speed.