Real-time Recurrent Learning Research Articles

Hand Recognition Obtained by Simulation of Hand Regard.

Eye-hand coordination of an infant is observed during the early months of their development. Hand regard, which is an example of this coordination, occurs at about 2 months. It is considered that after experiencing hand regard, an infant may recognize their own hands. However, it is unknown how an infant recognizes their hands through hand regard. Accordingly, the process by which an infant recognizes their hands and distinguishes between their hands and other objects was simulated. A simple neural network was trained with a modified real-time recurrent learning (RTRL) algorithm to deal with time-varying input and output during hand regard. The simulation results show that information about recognition of the modeled hands of an infant is stored in cell assemblies, which were self-organized. Cell assemblies appear during the phase of U-shaped developments of hand regard, and the configuration of the cell assemblies changes with each U-shaped development. Furthermore, movements like general movements (GMs) appear during the phase of U-shaped developments of hand regard.

Kurtosis-Based CRTRL Algorithms for Fully Connected Recurrent Neural Networks.

In this paper, kurtosis-based complex-valued real-time recurrent learning (KCRTRL) and kurtosis-based augmented CRTRL (KACRTRL) algorithms are proposed for training fully connected recurrent neural networks (FCRNNs) in the complex domain. These algorithms are designed by minimizing the cost functions based on the kurtosis of a complex-valued error signal. The KCRTRL algorithm exploits the circularity properties of the complex-valued signals, and this algorithm not only provides a faster convergence rate but also results in a lower steady-state error. However, the KCRTRL algorithm is suboptimal in the processing of noncircular (NC) complex-valued signals. On the other hand, the KACRTRL algorithm contains a complete second-order information due to the augmented statistics, thus considerably improves the performance of the FCRNN in the processing of NC complex-valued signals. Simulation results on the one-step-ahead prediction problems show that the proposed KCRTRL algorithm significantly enhances the performance for only circular complex-valued signals, whereas the proposed KACRTRL algorithm provides more superior performance than existing algorithms for NC complex-valued signals in terms of the convergence rate and the steady-state error.

Nonlinear Motion Tracking by Deep Learning Architecture

In the world of Artificial Intelligence, object motion tracking is one of the major problems. The extensive research is being carried out to track people in crowd. This paper presents a unique technique for nonlinear motion tracking in the absence of prior knowledge of nature of nonlinear path that the object being tracked may follow. We achieve this by first obtaining the centroid of the object and then using the centroid as the current example for a recurrent neural network trained using real-time recurrent learning. We have tweaked the standard algorithm slightly and have accumulated the gradient for few previous iterations instead of using just the current iteration as is the norm. We show that for a single object, such a recurrent neural network is highly capable of approximating the nonlinearity of its path.

Online natural gradient as a Kalman filter

We cast Amari’s natural gradient in statistical learning as a specific case of Kalman filtering. Namely, applying an extended Kalman filter to estimate a fixed unknown parameter of a probabilistic model from a series of observations, is rigorously equivalent to estimating this parameter via an online stochastic natural gradient descent on the log-likelihood of the observations. In the i.i.d. case, this relation is a consequence of the “information filter” phrasing of the extended Kalman filter. In the recurrent (state space, non-i.i.d.) case, we prove that the joint Kalman filter over states and parameters is a natural gradient on top of real-time recurrent learning (RTRL), a classical algorithm to train recurrent models. This exact algebraic correspondence provides relevant interpretations for natural gradient hyperparameters such as learning rates or initialization and regularization of the Fisher information matrix.

Study of Closed-Loop Model Reference Adaptive Control of Smart MicroGrid with QNU and Recurrent Learning

An adaptive quadratic polynomial neural unit (QNU) controller for optimization of a conventional Smart Microgrid control loop is studied and proposed. The parameters associated with the studied grid plants are considered to be known in this study, with the fact that the load is unknown and time-variant. A sample-by-sample real-time recurrent learning algorithm of an additional QNU controller is derived, with its performance tested and discussed as a result of this paper.

Trade-off analysis of discharge-desiltation-turbidity and ANN analysis on sedimentation of a combined reservoir–reach system under multi-phase and multi-layer conjunctive releasing operation

Multi-objective reservoir operation considering the trade-off of discharge-desiltation-turbidity during typhoons and sediment concentration (SC) simulation modeling are the vital components for sustainable reservoir management. The purposes of this study were (1) to analyze the multi-layer release trade-offs between reservoir desiltation and intake turbidity of downstream purification plants and thus propose a superior conjunctive operation strategy and (2) to develop ANFIS-based (adaptive network-based fuzzy inference system) and RTRLNN-based (real-time recurrent learning neural networks) substitute SC simulation models. To this end, this study proposed a methodology to develop (1) a series of multi-phase and multi-layer sediment-flood conjunctive release modes and (2) a specialized SC numerical model for a combined reservoir–reach system. The conjunctive release modes involve (1) an optimization model where the decision variables are multi-phase reduction/scaling ratios and the timings to generate a superior total release hydrograph for flood control (Phase I: phase prior to flood arrival, Phase II/III: phase prior to/subsequent to peak flow) and (2) a combination method with physical limitations regarding separation of the singular hydrograph into multi-layer release hydrographs for sediment control. This study employed the featured signals obtained from statistical quartiles/sediment duration curve in mesh segmentation, and an iterative optimization model with a sediment unit response matrix and corresponding geophysical-based acceleration factors, for efficient parameter calibration. This research applied the developed methodology to the Shihmen Reservoir basin in Taiwan. The trade-off analytical results using Typhoons Sinlaku and Jangmi as case examples revealed that owing to gravity current and re-suspension effects, Phase I+II can de-silt safely without violating the intake's turbidity limitation before reservoir discharge reaches 2238m3/s; however, Phase III can only de-silt after the release at spillway reaches 827m3/s, and before reservoir discharge reaches 1924m3/s, with corresponding maximum desiltation ratio being 0.221 and 0.323, respectively. Moreover, the model construction results demonstrated that the self-adaption/fuzzy inference of ANFIS can effectively simulate the SC hydrograph in an unsteady state for suspended load-dominated water bodies, and that the real-time recurrent deterministic routing of RTRLNN can accurately simulate that of a bedload-dominated flow regime.

Robust Recurrent Kernel Online Learning.

We propose a robust recurrent kernel online learning (RRKOL) algorithm based on the celebrated real-time recurrent learning approach that exploits the kernel trick in a recurrent online training manner. The novel RRKOL algorithm guarantees weight convergence with regularized risk management through the use of adaptive recurrent hyperparameters for superior generalization performance. Based on a new concept of the structure update error with a variable parameter length, we are the first one to propose the detailed structure update error, such that the weight convergence and robust stability proof can be integrated with a kernel sparsification scheme based on a solid theoretical ground. The RRKOL algorithm automatically weighs the regularized term in the recurrent loss function, such that we not only minimize the estimation error but also improve the generalization performance through sparsification with simulation support.

Adaptive Flutter Suppression for a Fighter Wing via Recurrent Neural Networks over a Wide Transonic Range

The paper presents a digital adaptive controller of recurrent neural networks for the active flutter suppression of a wing structure over a wide transonic range. The basic idea behind the controller is as follows. At first, the parameters of recurrent neural networks, such as the number of neurons and the learning rate, are initially determined so as to suppress the flutter under a specific flight condition in the transonic regime. Then, the controller automatically adjusts itself for a new flight condition by updating the synaptic weights of networks online via the real-time recurrent learning algorithm. Hence, the controller is able to suppress the aeroelastic instability of the wing structure over a range of flight conditions in the transonic regime. To demonstrate the effectiveness and robustness of the controller, the aeroservoelastic model of a typical fighter wing with a tip missile was established and a single-input/single-output controller was synthesized. Numerical simulations of the open/closed-loop aeroservoelastic simulations were made to demonstrate the efficacy of the adaptive controller with respect to the change of flight parameters in the transonic regime.

Coupled Heuristic Prediction of Long Lead-Time Accumulated Total Inflow of a Reservoir during Typhoons Using Deterministic Recurrent and Fuzzy Inference-Based Neural Network

This study applies Real-Time Recurrent Learning Neural Network (RTRLNN) and Adaptive Network-based Fuzzy Inference System (ANFIS) with novel heuristic techniques to develop an advanced prediction model of accumulated total inflow of a reservoir in order to solve the difficulties of future long lead-time highly varied uncertainty during typhoon attacks while using a real-time forecast. For promoting the temporal-spatial forecasted precision, the following original specialized heuristic inputs were coupled: observed-predicted inflow increase/decrease (OPIID) rate, total precipitation, and duration from current time to the time of maximum precipitation and direct runoff ending (DRE). This study also investigated the temporal-spatial forecasted error feature to assess the feasibility of the developed models, and analyzed the output sensitivity of both single and combined heuristic inputs to determine whether the heuristic model is susceptible to the impact of future forecasted uncertainty/errors. Validation results showed that the long lead-time–predicted accuracy and stability of the RTRLNN-based accumulated total inflow model are better than that of the ANFIS-based model because of the real-time recurrent deterministic routing mechanism of RTRLNN. Simulations show that the RTRLNN-based model with coupled heuristic inputs (RTRLNN-CHI, average error percentage (AEP)/average forecast lead-time (AFLT): 6.3%/49 h) can achieve better prediction than the model with non-heuristic inputs (AEP of RTRLNN-NHI and ANFIS-NHI: 15.2%/31.8%) because of the full consideration of real-time hydrological initial/boundary conditions. Besides, the RTRLNN-CHI model can promote the forecasted lead-time above 49 h with less than 10% of AEP which can overcome the previous forecasted limits of 6-h AFLT with above 20%–40% of AEP.

Neural network decoupling technique and its application to a powered wheelchair system.

This paper proposes a neural network decoupling technique for an uncertain multivariable system. Based on a linear diagonalization technique, a reference model is designed using nominal parameters to provide training signals for a neural network decoupler. A neural network model is designed to learn the dynamics of the uncertain multivariable system in order to avoid required calculations of the plant Jacobian. To avoid overfitting problem, both neural networks are trained by the Lavenberg-Marquardt with Bayesian regulation algorithm that uses a real-time recurrent learning algorithm to obtain gradient information. Three experimental results in the powered wheelchair control application confirm that the proposed technique effectively minimises the coupling effects caused by input-output interactions even under the condition of system uncertainties.

Real-Time Forecast of Reservoir Inflow Hydrographs Incorporating Terrain and Monsoon Effects during Typhoon Invasion by Novel Intelligent Numerical-Statistic Impulse Techniques

AbstractThis study develops an original methodology for forecasting real-time reservoir inflow hydrographs during typhoons, taking advantage of meteoro-hydrological methods such as analysis of typhoon hydrographs, numerical typhoon track forecasts, statistic typhoon central impulse-based quantitative precipitation forecasts model based on a real-time revised approach (TCI-RTQPF), real-time recurrent learning neural network (RTRLNN), and adaptive network-based fuzzy inference system (ANFIS). To derive the inflow hydrograph induced by interaction between typhoon rain bands, terrain, and monsoons, the inventive novel ensemble numerical-statistic impulse techniques are employed. The inflow during peak flow, inflection, and direct runoff ending (DRE) periods (impulse signal) are used for the deriving process. The hydrograph analysis is used to examine the mechanism between typhoon center location, wind field, precipitation, and the inflow hydrograph, and to establish the evaluation methods. Additionally, a nov...

Multi-phase intelligent decision model for reservoir real-time flood control during typhoons

This study applies an Adaptive Network-based Fuzzy Inference System (ANFIS) and a Real-Time Recurrent Learning Neural Network (RTRLNN) with an optimized reservoir release hydrograph using Mixed Integer Linear Programming (MILP) from historical typhoon events to develop a multi-phase intelligent real-time reservoir operation model for flood control. The flood control process is divided into three stages: (1) before flood (Stage I); (2) before peak flow (Stage II); and (3) after peak flow (Stage III). The models are then constructed with either three phase modules (ANFIS-3P and RTRLNN-3P) or two phase (Stage I + II and Stage III) modules (ANFIS-2P and RTRLNN-2P). The multi-phase modules are developed with consideration of the difference in operational decision mechanisms, decision information, release functions, and targets between each flood control stage to solve the problem of time-consuming computation and difficult system integration of MILP. In addition, the model inputs include the coupled short lead time and total reservoir inflow forecast information that are developed using radar- and satellite-based meteorological monitoring techniques, forecasted typhoon tracks, meteorological image similarity analysis, ANFIS and RTRLNN. This study uses the Tseng-Wen Reservoir basin as the study area, and the model results showed that RTRLNN outperformed ANFIS in the simulated outcomes from the optimized hydrographs. This study also applies the models to Typhoons Kalmaegi and Morakot to compare the simulations to historical operations. From the operation results, the RTRLNN-3P model is better than RTRLNN-2P and historical operations. Further, because the RTRLNN-3P model combines the innovative multi-phase module with monitored and forecasted decision information, the operation can simultaneously, effectively and automatically achieve the dual goals of flood detention at peak flow periods and water supply at the end of a typhoon event.

Identification of Nonlinear Dynamic System Using a Novel Recurrent Wavelet Neural Network Based on the Pipelined Architecture

This paper presents a novel modular recurrent neural network based on the pipelined architecture (PRWNN) to reduce the computational complexity and improve the performance of the recurrent wavelet neural network (RWNN). The PRWNN inherits the modular architectures of the pipelined recurrent neural network proposed by Haykin and Li and is made up of a number of RWNN modules that are interconnected in a chained form. Since those modules of the PRWNN can be simultaneously performed in a pipelined parallelism fashion, this would lead to a crucial improvement of computational efficiency. Furthermore, owing to the cascade interconnection of dynamic modules, the performance of the PRWNN can be further enhanced. An adaptive gradient algorithm based on the real-time recurrent learning is derived to suit for the modular PRWNN. Simulation examples are given to evaluate the effectiveness of the PRWNN model on the identification of nonlinear dynamic systems and analysis of sunspot number time series. According to simulation results, it is clearly shown that the PRWNN provides impressive better performance in comparison with the single RWNN model.

Learning long-term dependencies in segmented-memory recurrent neural networks with backpropagation of error

In general, recurrent neural networks have difficulties in learning long-term dependencies. The segmented-memory recurrent neural network (SMRNN) architecture together with the extended real-time recurrent learning (eRTRL) algorithm was proposed to circumvent this problem. Due to its computational complexity eRTRL becomes impractical with increasing network size. Therefore, we introduce the less complex extended backpropagation through time (eBPTT) for SMRNN together with a layer-local unsupervised pre-training procedure. A comparison on the information latching problem showed that eRTRL is better able to handle the latching of information over longer periods of time, even though eBPTT guaranteed a better generalisation when training was successful. Further, pre-training significantly improved the ability to learn long-term dependencies with eBPTT. Therefore, the proposed eBPTT algorithm is suited for tasks that require big networks where eRTRL is impractical. The pre-training procedure itself is independent of the supervised learning algorithm and can improve learning in SMRNN in general.

Improved Generalization in Recurrent Neural Networks Using the Tangent Plane Algorithm

The tangent plane algorithm for real time recurrent learning (TPA-RTRL) is an effective online training method for fully recurrent neural networks. TPA-RTRL uses the method of approaching tangent planes to accelerate the learning processes. Compared to the original gradient descent real time recurrent learning algorithm (GD-RTRL) it is very fast and avoids problems like local minima of the search space. However, the TPA-RTRL algorithm actively encourages the formation of large weight values that can be harmful to generalization. This paper presents a new TPA-RTRL variant that encourages small weight values to decay to zero by using a weight elimination procedure built into the geometry of the algorithm. Experimental results show that the new algorithm gives good generalization over a range of network sizes whilst retaining the fast convergence speed of the TPA-RTRL algorithm.

A robust recurrent simultaneous perturbation stochastic approximation training algorithm for recurrent neural networks

Training of recurrent neural networks (RNNs) introduces considerable computational complexities due to the need for gradient evaluations. How to get fast convergence speed and low computational complexity remains a challenging and open topic. Besides, the transient response of learning process of RNNs is a critical issue, especially for online applications. Conventional RNN training algorithms such as the backpropagation through time and real-time recurrent learning have not adequately satisfied these requirements because they often suffer from slow convergence speed. If a large learning rate is chosen to improve performance, the training process may become unstable in terms of weight divergence. In this paper, a novel training algorithm of RNN, named robust recurrent simultaneous perturbation stochastic approximation (RRSPSA), is developed with a specially designed recurrent hybrid adaptive parameter and adaptive learning rates. RRSPSA is a powerful novel twin-engine simultaneous perturbation stochastic approximation (SPSA) type of RNN training algorithm. It utilizes three specially designed adaptive parameters to maximize training speed for a recurrent training signal while exhibiting certain weight convergence properties with only two objective function measurements as the original SPSA algorithm. The RRSPSA is proved with guaranteed weight convergence and system stability in the sense of Lyapunov function. Computer simulations were carried out to demonstrate applicability of the theoretical results.

Reinforced recurrent neural networks for multi-step-ahead flood forecasts

Summary Considering true values cannot be available at every time step in an online learning algorithm for multi-step-ahead (MSA) forecasts, a MSA reinforced real-time recurrent learning algorithm for recurrent neural networks (R-RTRL NN) is proposed. The main merit of the proposed method is to repeatedly adjust model parameters with the current information including the latest observed values and model’s outputs to enhance the reliability and the forecast accuracy of the proposed method. The sequential formulation of the R-RTRL NN is derived. To demonstrate its reliability and effectiveness, the proposed R-RTRL NN is implemented to make 2-, 4- and 6-step-ahead forecasts in a famous benchmark chaotic time series and a reservoir flood inflow series in North Taiwan. For comparison purpose, three comparative neural networks (two dynamic and one static neural networks) were performed. Numerical and experimental results indicate that the R-RTRL NN not only achieves superior performance to comparative networks but significantly improves the precision of MSA forecasts for both chaotic time series and reservoir inflow case during typhoon events with effective mitigation in the time-lag problem.

Biological modeling of complex chemotaxis behaviors for C. elegans under speed regulation—a dynamic neural networks approach

In this paper, the modeling of several complex chemotaxis behaviors of C. elegans is explored, which include food attraction, toxin avoidance, and locomotion speed regulation. We first model the chemotaxis behaviors of food attraction and toxin avoidance separately. Then, an integrated chemotaxis behavioral model is proposed, which performs the two chemotaxis behaviors simultaneously. The novelty and the uniqueness of the proposed chemotaxis behavioral models are characterized by several attributes. First, all the chemotaxis behavioral model sare on biological basis, namely, the proposed chemotaxis behavior models are constructed by extracting the neural wire diagram from sensory neurons to motor neurons, where sensory neurons are specific for chemotaxis behaviors. Second, the chemotaxis behavioral models are able to perform turning and speed regulation. Third, chemotaxis behaviors are characterized by a set of switching logic functions that decide the orientation and speed. All models are implemented using dynamic neural networks (DNN) and trained using the real time recurrent learning (RTRL) algorithm. By incorporating a speed regulation mechanism, C. elegans can stop spontaneously when approaching food source or leaving away from toxin. The testing results and the comparison with experiment results verify that the proposed chemotaxis behavioral models can well mimic the chemotaxis behaviors of C. elegans in different environments.

Reinforced two-step-ahead weight adjustment technique for online training of recurrent neural networks.

A reliable forecast of future events possesses great value. The main purpose of this paper is to propose an innovative learning technique for reinforcing the accuracy of two-step-ahead (2SA) forecasts. The real-time recurrent learning (RTRL) algorithm for recurrent neural networks (RNNs) can effectively model the dynamics of complex processes and has been used successfully in one-step-ahead forecasts for various time series. A reinforced RTRL algorithm for 2SA forecasts using RNNs is proposed in this paper, and its performance is investigated by two famous benchmark time series and a streamflow during flood events in Taiwan. Results demonstrate that the proposed reinforced 2SA RTRL algorithm for RNNs can adequately forecast the benchmark (theoretical) time series, significantly improve the accuracy of flood forecasts, and effectively reduce time-lag effects.

Learning in fully recurrent neural networks by approaching tangent planes to constraint surfaces

In this paper we present a new variant of the online real time recurrent learning algorithm proposed by Williams and Zipser (1989). Whilst the original algorithm utilises gradient information to guide the search towards the minimum training error, it is very slow in most applications and often gets stuck in local minima of the search space. It is also sensitive to the choice of learning rate and requires careful tuning. The new variant adjusts weights by moving to the tangent planes to constraint surfaces. It is simple to implement and requires no parameters to be set manually. Experimental results show that this new algorithm gives significantly faster convergence whilst avoiding problems like local minima.