Recursive Levenberg-Marquardt Research Articles

Complex-valued neural network topology and learning applied for identification and control of nonlinear systems

In this paper we present a Complex-Valued Recurrent Neural Network (CVRNN), trained with a recursive Levenberg-Marquardt (LM) learning algorithm in the complex domain, applying it to the problem of dynamic system identification, and to an adaptive neural control scheme of a nonlinear oscillatory plant. This methodology is applied to two different CVRNN topologies with different kinds of activation functions. Furthermore, we applied the CVRNN identification and control for a particular case of a nonlinear, oscillatory mechanical plant to validate the performance of the adaptive neural controller using the LM algorithm developed throughout this work, compared to a complex-valued Backpropagation learning algorithm. The obtained comparative simulation results using a flexible robot arm gives a good performance of the derived control schemes. The results show some priority of the recursive LM learning over the BP learning, and the use of constructed activation functions in the neural network topology.

Recurrent Neural Adaptive Control of Nonlinear Oscillatory Systems Using a Complex-valued Levenberg-Marquardt Learning Algorithm

Abstract In this work, a Recursive Levenberg-Marquardt learning algorithm in the complex domain is developed and applied in the training of two adaptive control schemes composed by Complex-Valued Recurrent Neural Networks. Furthermore, we apply the identification and both control schemes for a particular case of nonlinear, oscillatory mechanical plant to validate the performance of the adaptive neural controller and the learning algorithm. The comparative simulation results show the better performance of the newly proposed Complex-Valued Recursive Levenberg-Marquardt learning algorithm over the gradient-based recursive Back-propagation one.

Design and Implementation of Recursive Model Predictive Control for Permanent Magnet Synchronous Motor Drives

In order to control the permanent-magnet synchronous motor system (PMSM) with different disturbances and nonlinearity, an improved current control algorithm for the PMSM systems using recursive model predictive control (RMPC) is developed in this paper. As the conventional MPC has to be computed online, its iterative computational procedure needs long computing time. To enhance computational speed, a recursive method based on recursive Levenberg-Marquardt algorithm (RLMA) and iterative learning control (ILC) is introduced to solve the learning issue in MPC. RMPC is able to significantly decrease the computation cost of traditional MPC in the PMSM system. The effectiveness of the proposed algorithm has been verified by simulation and experimental results.

Adaptive Learning and Control for MIMO System Based on Adaptive Dynamic Programming

Adaptive dynamic programming (ADP) is a promising research field for design of intelligent controllers, which can both learn on-the-fly and exhibit optimal behavior. Over the past decades, several generations of ADP design have been proposed in the literature, which have demonstrated many successful applications in various benchmarks and industrial applications. While many of the existing researches focus on multiple-inputs-single-output system with steepest descent search, in this paper we investigate a generalized multiple-input-multiple-output (GMIMO) ADP design for online learning and control, which is more applicable to a wide range of practical real-world applications. Furthermore, an improved weight-updating algorithm based on recursive Levenberg-Marquardt methods is presented and embodied in the GMIMO approach to improve its performance. Finally, we test the performance of this approach based on a practical complex system, namely, the learning and control of the tension and height of the looper system in a hot strip mill. Experimental results demonstrate that the proposed approach can achieve effective and robust performance.

An Adaptive Recurrent Neural Network for Remaining Useful Life Prediction of Lithium-ion Batteries


 
 
 Prognostics is an emerging science of predicting the health condition of a system (or its components) based upon current and previous system states. A reliable predictor is very useful to a wide array of industries to predict the future states of the system such that the maintenance service could be scheduled in advance when needed. In this paper, an adaptive recurrent neural network (ARNN) is proposed for system dynamic state forecasting. The developed ARNN is constructed based on the adaptive/recurrent neural network architecture and the network weights are adaptively optimized using the recursive Levenberg-Marquardt (RLM) method. The effectiveness of the proposed ARNN is demonstrated via an application in remaining useful life prediction of lithium-ion batteries.
 
 

Bit Rate Maximising Per-Tone Equalisation with Adaptive Implementation for DMT-Based Systems

We present a bit rate maximising per-tone equalisation (BM-PTEQ) cost function that is based on an exact subchannel SNR as a function of per-tone equaliser in discrete multitone (DMT) systems. We then introduce the proposed BM-PTEQ criterion whose derivation for solution is shown to inherit from the methodology of the existing bit rate maximising time-domain equalisation (BM-TEQ). By solving a nonlinear BM-PTEQ cost function, an adaptive BM-PTEQ approach based on a recursive Levenberg-Marquardt (RLM) algorithm is presented with the adaptive inverse square-root (iQR) algorithm for DMT-based systems. Simulation results confirm that the performance of the proposed adaptive iQR RLM-based BM-PTEQ converges close to the performance of the proposed BM-PTEQ. Moreover, the performance of both these proposed BM-PTEQ algorithms is improved as compared with the BM-TEQ.

An Evolving Fuzzy Predictor for Industrial Applications

A reliable and online predictor is very useful to a wide array of industries to forecast the behavior of time-varying dynamic systems. In this paper, an evolving fuzzy system (EFS) is developed for system state forecasting. An evolving clustering algorithm is proposed for cluster generation. Clusters are established and modified based on constraint criteria of mapping consistence and compatible measurement. A novel recursive Levenberg-Marquardt (R-LM) method is proposed for online training of nonlinear EFS parameters. The viability of the developed EFS predictor is evaluated based on both simulation from benchmark data and real-time tests corresponding to machinery condition monitoring and material property testing. Test results show that the developed EFS predictor is an effective and accurate forecasting tool. It can capture the system's dynamic behavior quickly and track the system's characteristics accurately. The proposed clustering algorithm is an effective structure identification method. The recursive training technique is computationally efficient, and can effectively improve reasoning convergence.

RECURRENT NEURAL CONTROL OF WASTEWATER TREATMENT BIOPROCESS VIA MARQUARDT LEARNING

The aim of this paper is to propose a new Recurrent Neural Network (RTNN) topology and a dynamic recursive Levenberg-Marquardt algorithm of its learning capable to estimate the states and parameters of a highly nonlinear wastewater treatment bioprocess. The proposed RTNN identifier is implemented in a direct adaptive control scheme incorporating feedback/feedforward recurrent neural controllers and a noise rejecting filter. The proposed control scheme is applied for continuous wastewater treatment bioprocess plant model, taken from the literature, where a good convergence, noise filtering and a low Mean Squared Error of reference tracking is achieved.

Adaptive bit rate maximizing time-domain equalizer design for DMT-based systems

The classical discrete multitone receiver as used in, e.g., digital subscriber line (DSL) modems, combines a channel shortening time-domain equalizer (TEQ) with one-tap frequency-domain equalizers (FEQs). In a previous paper, the authors proposed a nonlinear bit rate maximizing (BM) TEQ design criterion and they have shown that the resulting BM-TEQ and the closely related BM per-group equalizers (PGEQs) approach the performance of the so-called per-tone equalizer (PTEQ). The PTEQ is an attractive alternative that provides a separate complex-valued equalizer for each active tone. In this paper, the authors show that the BM-TEQ and BM-PGEQ, despite their nonlinear cost criterion, can be designed adaptively, based on a recursive Levenberg-Marquardt algorithm. This adaptive BM-TEQ/BM-PGEQ makes use of the same second-order statistics as the earlier presented recursive least-squares (RLS)-based adaptive PTEQ. A complete range of adaptive BM equalizers then opens up: the RLS-based adaptive PTEQ design is computationally efficient but involves a large number of equalizer taps; the adaptive BM-TEQ has a minimal number of equalizer taps at the expense of a larger design complexity; the adaptive BM-PGEQ has a similar design complexity as the BM-TEQ and an intermediate number of equalizer taps between the BM-TEQ and the PTEQ. These adaptive equalizers allow us to track variations of transmission channel and noise, which are typical of a DSL environment.

Second-Order Training of Adaptive Critics for Online Process Control

This paper deals with reinforcement learning for process modeling and control using a model-free, action- dependent adaptive critic (ADAC). A new modified recursive Levenberg Marquardt (RLM) training algorithm, called temporal difference RLM, is developed to improve the ADAC performance. Novel application results for a simulated continuously-stirred-tank-reactor process are included to show the superiority of the new algorithm to conventional temporal-difference stochastic backpropagation.

Decoupling PI controller design for a normal conducting RF cavity using a recursive LEVENBERG-MARQUARDT algorithm

This paper addresses the system identification and the decoupling PI controller design for a normal conducting RF cavity. Based on the open-loop measurement data of an SNS DTL cavity, the open-loop system's bandwidths and loop time delays are estimated by using batched least square. With the identified system, a PI controller is designed in such a way that it suppresses the time varying klystron droop and decouples the In-phase and Quadrature of the cavity field. The Levenberg-Marquardt algorithm is applied for nonlinear least squares to obtain the optimal PI controller parameters. The tuned PI controller gains are downloaded to the low-level RF system by using channel access. The experiment of the closed-loop system is performed and the performance is investigated. The proposed tuning method is running automatically in real time interface between a host computer with controller hardware through ActiveX Channel Access.

Efficient training of neural nets for nonlinear adaptive filtering using a recursive Levenberg-Marquardt algorithm

The Levenberg-Marquardt algorithm is often superior to other training algorithms in off-line applications. This motivates the proposal of using a recursive version of the algorithm for on-line training of neural nets for nonlinear adaptive filtering. The performance of the suggested algorithm is compared with other alternative recursive algorithms, such as the recursive version of the off-line steepest-descent and Gauss-Newton algorithms. The advantages and disadvantages of the different algorithms are pointed out. The algorithms are tested on some examples, and it is shown that generally the recursive Levenberg-Marquardt algorithm has better convergence properties than the other algorithms.