Dynamic Recurrent Neural Network Research Articles

Robustly Fitting and Forecasting Dynamical Data With Electromagnetically Coupled Artificial Neural Network: A Data Compression Method.

In this paper, a dynamical recurrent artificial neural network (ANN) is proposed and studied. Inspired from a recent research in neuroscience, we introduced nonsynaptic coupling to form a dynamical component of the network. We mathematically proved that, with adequate neurons provided, this dynamical ANN model is capable of approximating any continuous dynamic system with an arbitrarily small error in a limited time interval. Its extreme concise Jacobian matrix makes the local stability easy to control. We designed this ANN for fitting and forecasting dynamic data and obtained satisfied results in simulation. The fitting performance is also compared with those of both the classic dynamic ANN and the state-of-the-art models. Sufficient trials and the statistical results indicated that our model is superior to those have been compared. Moreover, we proposed a robust approximation problem, which asking the ANN to approximate a cluster of input-output data pairs in large ranges and to forecast the output of the system under previously unseen input. Our model and learning scheme proposed in this paper have successfully solved this problem, and through this, the approximation becomes much more robust and adaptive to noise, perturbation, and low-order harmonic wave. This approach is actually an efficient method for compressing massive external data of a dynamic system into the weight of the ANN.

Local community detection as pattern restoration by attractor dynamics of recurrent neural networks

Densely connected parts in networks are referred to as “communities”. Community structure is a hallmark of a variety of real-world networks. Individual communities in networks form functional modules of complex systems described by networks. Therefore, finding communities in networks is essential to approaching and understanding complex systems described by networks. In fact, network science has made a great deal of effort to develop effective and efficient methods for detecting communities in networks. Here we put forward a type of community detection, which has been little examined so far but will be practically useful. Suppose that we are given a set of source nodes that includes some (but not all) of “true” members of a particular community; suppose also that the set includes some nodes that are not the members of this community (i.e., “false” members of the community). We propose to detect the community from this “imperfect” and “inaccurate” set of source nodes using attractor dynamics of recurrent neural networks. Community detection by the proposed method can be viewed as restoration of the original pattern from a deteriorated pattern, which is analogous to cue-triggered recall of short-term memory in the brain. We demonstrate the effectiveness of the proposed method using synthetic networks and real social networks for which correct communities are known.

Fuzzy neural network-based robust adaptive control for dynamic positioning of underwater vehicles with input dead-zone

This paper proposes a design for a robust adaptive controller for the Dynamical Positioning (DP) of underwater vehicles with unknown hydrodynamic coefficients, unknown disturbances and input dead-zones. First, for convenience of controller design, the Multi-Input Multi-Output (MIMO) system is divided into several Single-Input Single-Output (SISO) systems. Next, a Dynamic Recurrent Fuzzy Neural Network (DRFNN) with feedback loops is employed to approximate the unknown portion of the controller, which can greatly reduce the number of neural network inputs. A fuzzy logic dead-zone compensator is designed to cope with the unknown dead-zone characteristics of actuators. The upper bounds of the approximation errors and disturbances of the network, which are often used in existing works, are not necessary in this paper due to the presentation of a special robust compensator. Stability analysis is conducted according to the Lyapunov theorem, and the tracking error is proved to converge to zero. Simulation results indicate that the proposed controller demonstrates good performance.

Echo State Network 모델의 은닉 뉴런 간 연결구조에 따른 성능과 동역학적 특성 분석

Recurrent Neural Network (RNN), a machine learning model which can handle time-series data, can possess more varied structures than a feed-forward neural network, since a RNN allows hidden-to-hidden connections. This research focuses on the network structure among hidden neurons, and discusses the information processing capability of RNN. Time-series learning potential and dynamics of RNNs are investigated upon several well-established network structure models. Hidden neuron network structure is found to have significant impact on the performance of a model, and the performance variations are generally correlated with the criticality of the network dynamics. Especially Preferential Attachment Network model showed an interesting behavior. These findings provide clues for performance improvement of the RNN.

Corrosion Fault Diagnosis of Rolling Element Bearing under Constant and Variable Load and Speed Conditions

Rolling element bearing defects is one of the main reasons for breakdown in electrical machines. Robust fault analysis (FA) including the diagnosis of faults and predicting their level of fault severity is thus necessary to optimise maintenance, improve reliability and to avoid more catastrophic failure consequences. The proposed diagnostic methods in this paper use the innovative discrete wavelet transform (DWT) for feature extraction and an orthogonal fuzzy neighbourhood discriminative analysis (OFNDA) approach for feature reduction. The dynamic recurrent neural network predicts the conditions of components and classifies faults under different operating conditions. The results obtained from the real time simulation demonstrate the effectiveness and reliability of the proposed methodology in classifying the different faults faster and accurately.

DYNAMIC RECURRENT NEURAL NETWORK BASED CLASSIFICATION SCHEME FOR MYOELECTRIC CONTROL OF UPPER LIMB REHABILITATION ROBOT

This paper proposes and evaluates the application of a modular dynamic recurrent neural network (DRNN) to classify upper limb motion using myoelectric signals. The DRNN algorithmic issues, including the structure selection, the segmentation of the data and various feature sets such as time-domain features and frequency features, were evaluated experimentally in order to actualize the optimization and configuration of this classification scheme. This was achieved by using a majority vote technique to post-process the output decision stream. The DRNN-based approach was then been compared with two commonly used classification methods: multilayer perceptron (MLP) neural network and linear discriminant analysis (LDA). The DRNN-based motion classification system demonstrated exceptional accuracy and a low computational load for the classification of robust limb motion. The DRNN may also display utility for online training and controlling rehabilitation robots.

Physiological modules for generating discrete and rhythmic movements: action identification by a dynamic recurrent neural network.

In this study we employed a dynamic recurrent neural network (DRNN) in a novel fashion to reveal characteristics of control modules underlying the generation of muscle activations when drawing figures with the outstretched arm. We asked healthy human subjects to perform four different figure-eight movements in each of two workspaces (frontal plane and sagittal plane). We then trained a DRNN to predict the movement of the wrist from information in the EMG signals from seven different muscles. We trained different instances of the same network on a single movement direction, on all four movement directions in a single movement plane, or on all eight possible movement patterns and looked at the ability of the DRNN to generalize and predict movements for trials that were not included in the training set. Within a single movement plane, a DRNN trained on one movement direction was not able to predict movements of the hand for trials in the other three directions, but a DRNN trained simultaneously on all four movement directions could generalize across movement directions within the same plane. Similarly, the DRNN was able to reproduce the kinematics of the hand for both movement planes, but only if it was trained on examples performed in each one. As we will discuss, these results indicate that there are important dynamical constraints on the mapping of EMG to hand movement that depend on both the time sequence of the movement and on the anatomical constraints of the musculoskeletal system. In a second step, we injected EMG signals constructed from different synergies derived by the PCA in order to identify the mechanical significance of each of these components. From these results, one can surmise that discrete-rhythmic movements may be constructed from three different fundamental modules, one regulating the co-activation of all muscles over the time span of the movement and two others elliciting patterns of reciprocal activation operating in orthogonal directions.

Intelligent Identification of Flow Regime Based on a Novel Neural Network

A noveol neural network of Elman is typically dynamic recurrent neural network. A novel method of flow regime identification based on Elman neural network and wavelet packet decomposition is proposed in this paper. Above all, the collected pressure-difference fluctuation signals are decomposed by the four-layer wavelet packet, and the decomposed signals in various frequency bands are obtained within the frequency domain. Then the wavelet packet energy eigenvectors of flow regimes are established. At last the wavelet packet energy eigenvectors are input into Elman neural network and flow regime intelligent identification can be performed.

Behavior control in the sensorimotor loop with short-term synaptic dynamics induced by self-regulating neurons.

The behavior and skills of living systems depend on the distributed control provided by specialized and highly recurrent neural networks. Learning and memory in these systems is mediated by a set of adaptation mechanisms, known collectively as neuronal plasticity. Translating principles of recurrent neural control and plasticity to artificial agents has seen major strides, but is usually hampered by the complex interactions between the agent's body and its environment. One of the important standing issues is for the agent to support multiple stable states of behavior, so that its behavioral repertoire matches the requirements imposed by these interactions. The agent also must have the capacity to switch between these states in time scales that are comparable to those by which sensory stimulation varies. Achieving this requires a mechanism of short-term memory that allows the neurocontroller to keep track of the recent history of its input, which finds its biological counterpart in short-term synaptic plasticity. This issue is approached here by deriving synaptic dynamics in recurrent neural networks. Neurons are introduced as self-regulating units with a rich repertoire of dynamics. They exhibit homeostatic properties for certain parameter domains, which result in a set of stable states and the required short-term memory. They can also operate as oscillators, which allow them to surpass the level of activity imposed by their homeostatic operation conditions. Neural systems endowed with the derived synaptic dynamics can be utilized for the neural behavior control of autonomous mobile agents. The resulting behavior depends also on the underlying network structure, which is either engineered or developed by evolutionary techniques. The effectiveness of these self-regulating units is demonstrated by controlling locomotion of a hexapod with 18 degrees of freedom, and obstacle-avoidance of a wheel-driven robot.

Fault detection of wind energy conversion systems using recurrent neural networks

Structure of wind energy conversion systems (WECSs) must be robust against faults. In order to accurately study WECSs during occurrence of faults and to explore the impact of faults on each component of the WECSs, a detailed model is required in which both mechanical and electrical parts of the WECSs are properly involved. In addition, a fault detection system (FDS) is required to diagnose the occurred faults at the appropriate time in order to ensure a safe system operation, avoid heavy economic losses, prevent damage to adjacent relevant systems and facilitate timely repair of failed components. This can be performed by subsequent actions through fast and accurate detection of faults. In this paper, by utilising a comprehensive dynamic model of the WECS, an FDS is presented using dynamic recurrent neural networks. In industrial processes, dynamic neural networks are known as a good mathematical tool for fault detection. The proposed FDS detects faults of the generator's angular velocity sensor, pitch angle sensors and pitch actuators. The presented FDS has high capability of fault detection in short time and it has much low false alarms rate. Simulation results verify validity and usefulness of the proposed fault detection scheme.

Robust fault detection of wind energy conversion systems based on dynamic neural networks.

Occurrence of faults in wind energy conversion systems (WECSs) is inevitable. In order to detect the occurred faults at the appropriate time, avoid heavy economic losses, ensure safe system operation, prevent damage to adjacent relevant systems, and facilitate timely repair of failed components; a fault detection system (FDS) is required. Recurrent neural networks (RNNs) have gained a noticeable position in FDSs and they have been widely used for modeling of complex dynamical systems. One method for designing an FDS is to prepare a dynamic neural model emulating the normal system behavior. By comparing the outputs of the real system and neural model, incidence of the faults can be identified. In this paper, by utilizing a comprehensive dynamic model which contains both mechanical and electrical components of the WECS, an FDS is suggested using dynamic RNNs. The presented FDS detects faults of the generator's angular velocity sensor, pitch angle sensors, and pitch actuators. Robustness of the FDS is achieved by employing an adaptive threshold. Simulation results show that the proposed scheme is capable to detect the faults shortly and it has very low false and missed alarms rate.

Weather prediction by recurrent neural network dynamics

Most of the weather forecasting approaches attempted to forecast only single weather attribute at a time. Moreover, they did not consider generalised weather dynamics hidden in meteorological data. This paper presents a multiple weather attributes forecasting method for one-day-ahead prediction using NARX NN in local scale. In this approach, a case is represented in such a way that it can capture dynamic and chaotic behaviour of weather, case base is segmented to capture seasonality of weather and then NARX NN is used to map nonlinear relationships of the meteorological data as well as the dynamic behaviour of the system over a period of time. Forecasting performance of CBR methods and NARX NN is compared. From the empirical results, superiority of NARX NN method is established in forecasting of multiple weather attributes. For model training, validating and testing, collected historical records of weather station from 1980 to 2009 are used.

A Model‐Free Stability‐Based Adaptive Control Method for Unknown Nonlinear Systems

AbstractThis contribution considers an adaptive control method based on a cognition‐based framework to stabilize unknown nonlinear systems online. This method requires only the system outputs, which are assumed as measurable. The structure of the framework consists of three parts. The first part is based on a dynamic recurrent neural network (DRNN) to be used for local identification, analysis and multi‐step‐ahead prediction of the system. In the second part, a set of given input values will be calculated numerically with a geometrical criterion based on a suitable definition of quadratic stability. In the third part, the most suitable control input value is chosen for the next predefined time interval according to a suitable cost function. The proposed controller is able to gain useful local knowledge and define autonomously suitable local control input according to the stability criterion. Numerical examples using inverted pendulum system and Lorenz system are shown to demonstrate the successful application and performance of the method. (© 2013 Wiley‐VCH Verlag GmbH & Co. KGaA, Weinheim)

Identification and modeling of the three rotational movements of a miniature coaxial helicopter

This paper presents the identification of a miniature coaxial helicopter system. First, the helicopter flying principles are described and the hardware setup of the developed platform is presented. Further, linear models are developed for the movements of the helicopter using prediction error identification methods. The results in this case are accurate and can be used for performant controller design in some operating points. But in order to model the complete dynamics of the helicopter, nonlinear models are developed using recurrent dynamic neural networks. In this case the models obtained present a higher accuracy compared with the linear case and also with the results published until now. In the end, the advantages of nonlinear modeling based on neural networks is emphasized and some conclusions are drawn.

Power System Short-Term Load Forecasting Based on Fuzzy Neural Network

Short-Term Load Forecasting (STLF) is an important operational function in both regulated power systems. This study is concerned with the problem of STLF. Considering the factors such as temperature, date type, weather status, etc, which influence the STLF, a model is set up by dynamic recurrent fuzzy neural network. The fuzzy inference function is realized easily by using a product operation in the network. Introducing local recurrent units to hidden layer, the proposed method can overcome the limit of the traditional BP algorithm. The actual simulation is given to demonstrate the effectiveness of the proposed methods.

PP1.3 – 1950 Learning walking coordination through dynamic recurrent neural network in children with bilateral cerebral palsy

PP1.3 – 1950 Learning walking coordination through dynamic recurrent neural network in children with bilateral cerebral palsy

Nonlinear Identification of a Magneto‐Rheological Damper Based on Dynamic Neural Networks

AbstractSemi‐active control of dynamic response of civil structures with magneto‐rheological (MR) fluid dampers has emerged as a novel revolutionary technology in recent years for designing “smart structures.” A small‐scale MR damper model with the valve mode mechanism has been examined in this research using dynamic recurrent neural network modeling approach to reproduce its hysteretic nonlinear behavior. Modified Bouc–Wen model based on nonlinear differential equations has not only been employed as the reference model to provide a comprehensive training data for the neural network but also for comparison purposes. A novel frequency and amplitude varying displacement input signal (modulated chirp signal) associated with a random supply voltage has been introduced for persistent excitation of the damper in such a way to cover almost all of its operating conditions. Finally a series of validation tests were conducted on the proposed model which proved the appropriate performance of the model in terms of accuracy and capability for realization.

Behavioural pattern identification and prediction in intelligent environments

In this paper, the application of soft computing techniques in prediction of an occupant's behaviour in an inhabited intelligent environment is addressed. In this research, daily activities of elderly people who live in their own homes suffering from dementia are studied. Occupancy sensors are used to extract the movement patterns of the occupant. The occupancy data is then converted into temporal sequences of activities which are eventually used to predict the occupant behaviour. To build the prediction model, different dynamic recurrent neural networks are investigated. Recurrent neural networks have shown a great ability in finding the temporal relationships of input patterns. The experimental results show that non-linear autoregressive network with exogenous inputs model correctly extracts the long term prediction patterns of the occupant and outperformed the Elman network. The results presented here are validated using data generated from a simulator and real environments.

Biological oscillations for learning walking coordination: dynamic recurrent neural network functionally models physiological central pattern generator

The existence of dedicated neuronal modules such as those organized in the cerebral cortex, thalamus, basal ganglia, cerebellum, or spinal cord raises the question of how these functional modules are coordinated for appropriate motor behavior. Study of human locomotion offers an interesting field for addressing this central question. The coordination of the elevation of the 3 leg segments under a planar covariation rule (Borghese et al., 1996) was recently modeled (Barliya et al., 2009) by phase-adjusted simple oscillators shedding new light on the understanding of the central pattern generator (CPG) processing relevant oscillation signals. We describe the use of a dynamic recurrent neural network (DRNN) mimicking the natural oscillatory behavior of human locomotion for reproducing the planar covariation rule in both legs at different walking speeds. Neural network learning was based on sinusoid signals integrating frequency and amplitude features of the first three harmonics of the sagittal elevation angles of the thigh, shank, and foot of each lower limb. We verified the biological plausibility of the neural networks. Best results were obtained with oscillations extracted from the first three harmonics in comparison to oscillations outside the harmonic frequency peaks. Physiological replication steadily increased with the number of neuronal units from 1 to 80, where similarity index reached 0.99. Analysis of synaptic weighting showed that the proportion of inhibitory connections consistently increased with the number of neuronal units in the DRNN. This emerging property in the artificial neural networks resonates with recent advances in neurophysiology of inhibitory neurons that are involved in central nervous system oscillatory activities. The main message of this study is that this type of DRNN may offer a useful model of physiological central pattern generator for gaining insights in basic research and developing clinical applications.

Robust Identification-Based State Derivative Estimation for Nonlinear Systems

A robust identification-based state derivative estimation method for uncertain nonlinear systems is developed. The identifier architecture consists of a recurrent multilayer dynamic neural network which approximates the system dynamics online, and a continuous robust feedback Robust Integral of the Sign of the Error (RISE) term which accounts for modeling errors and exogenous disturbances. Numerical simulations provide comparisons with existing robust derivative estimation methods including: a high gain observer, a 2-sliding mode robust exact differentiator, and numerical differentiation methods, such as backward difference and central difference.