Persistent Excitation Condition Research Articles

Locally Weighted Learning Robot Control With Improved Parameter Convergence

An adaptive robot control approach that can achieve an accurate approximation of the complex robot dynamics without knowledge unlearning in the case of multiple sequential tasks is proposed in this article. A locally weighted learning (LWL) model with automatic structure growth according to the size of the learning domain is introduced to approximate unknown robot dynamics, and a composite learning technique with regressor extension is applied to improve parameter convergence. The LWL ensures that learning in one area of the learning domain does not cause unlearning in another area, and the composite learning theoretically guarantees the identification performance of the LWL model. By the exploitation of both stored data in memory and instantaneous data, parameter convergence in the LWL model is achieved under a more achievable interval-excitation condition than the stringent persistent-excitation condition. This further enhances the trajectory tracking performance for multiple sequential tasks, which is generally not achievable by existing approximation-based adaptive robot control approaches. Experimental studies have verified the superiority of the proposed approach over prevalent approaches.

Online adaptive optimal control algorithm based on synchronous integral reinforcement learning with explorations

In this study, we present a novel algorithm, based on synchronous policy iteration, to solve the continuous-time infinite-horizon optimal control problem of input affine system dynamics. The integral reinforcement is measured as an excitation signal to estimate the solution to the Hamilton–Jacobi–Bellman equation. In addition, the proposed method is completely model-free, that is, no a priori knowledge of the system is required. Using the adaptive tuning law, the actor and critic neural networks can simultaneously approximate the optimal value function and policy. The persistence of excitation condition is required to guarantee the convergence of the two networks. Unlike in traditional policy iteration algorithms, the restriction of the initial admissible policy was eliminated using this method. The effectiveness of the proposed algorithm is verified through numerical simulations.

Dynamic event-triggered-based single-network ADP optimal tracking control for the unknown nonlinear system with constrained input

This paper presents an event-triggered-based optimal tracking control scheme for the unknown nonlinear system with unmeasurable state, constrained input, and limited communication. Firstly, an event-triggered Hamilton–Jacobi-Bellman (HJB) equation is derived based on an augmented system and a discounted cost function. Next, a novel online event-triggered-based neural networks (NNs) observer (NNsO) is constructed to obtain the unmeasurable state and the unknown system dynamics. Then, a single critic neural network (NN) based on the adaptive dynamic programming (ADP) framework is established to solve the event-triggered HJB equation, and the designed critic NN weight updating law can relax the restrictions of initial admissible control and persistence of excitation condition. Furthermore, a static triggering rule with an exponential term is designed to reduce communication and computing burden, and based on this, a dynamic triggering rule is established by introducing an auxiliary dynamic variable to obtain efficient communication and computing. It is proved that the tracking error and all the NN weight estimation errors are uniformly ultimately bounded (UUB), and the Zeno behavior is excluded. Finally, two simulation examples are presented to verify the effectiveness and superiority of the proposed control scheme.

COMPOSITE NEURAL NETWORK LEARNING FROM FRACTIONAL BACKSTEPPING

This paper focuses on parameter convergence and precise modeling for fractional-order nonlinear systems with functional uncertainties via using adaptive backstepping neural network control (ABNNC) and composite learning adaptive backstepping neural network control (CLABNNC). In the ABNNC design, a command filter is proposed, and the neural network approximation system is considered to deal with the unknown function, where an adaptation law is designed to ensure tracking errors converge to an arbitrarily small region near the origin under a strict persistent excitation condition that is too strict for the convergence of adaptive parameters. In order to relax this condition, a composite learning adaptation law is established by taking advantage of the tracking error and the prediction error to update the free parameter of the neural network system. The proposed CLABNNC method can not only ensure the convergence of tracking errors, but also achieve the accurate approximation of functional uncertainties under a weaker interval excitation condition. Finally, a numerical simulation example is put forward to demonstrate the effectiveness of our method.

Composite fuzzy learning finite-time prescribed performance control of uncertain nonlinear systems with dead-zone inputs

This paper presents a composite fuzzy learning finite-time prescribed performance control (PPC) approach for uncertain nonlinear systems with dead-zone inputs. First, a finite-time performance function is constructed by a quartic polynomial. Subsequently, with the help of an error transformation function, the restriction problem of the tracking error performance is transformed into a stability problem of an equivalent transformation system. In order to ensure that all signals of the closed-loop system are bounded, a finite-time PPC method combined with fuzzy logic systems (FLSs) is proposed. Although the tracking error can be guaranteed to be limited within a predefined range, the proposed finite-time PPC method only uses instantaneous data, which cannot guarantee the accurate estimation of unknown functions under the influence of dead-zone inputs. Therefore, based on the persistent excitation (PE) condition, a predictive error is defined by using online recorded data and instantaneous data, and a corresponding composite learning finite-time PPC method with parameter updating the law, which can not only achieve the control aim of the former method but also improve the control effect, is designed. The simulation results verified that the composite learning finite-time PPC method is more effective than the finite-time PPC method without learning.

Adaptive observer design for uncertain nonlinear ODE‐PDE cascades

SummaryWe are revisiting the problem of adaptive observer design for systems that are constituted of an Ordinary Differential Equation (ODE), containing a globally Lipschitz function of the state, and a linear Partial Differential Equation (PDE) of a diffusion–reaction heat type. The ODE and PDE are connected in series and both are subject to parametric uncertainties. In addition to nonlinearity and uncertainty, the system complexity also lies in the fact that no sensor can be implemented at the junction point between the ODE and the PDE. In the absence of parameter uncertainty, nonadaptive state observers are available featuring exponential convergence. However, convergence is guaranteed only under the condition that either the Lipschitz coefficient is sufficiently small or the PDE domain length is sufficiently small. To get around this limitation, and also to account for parameter uncertainty, we develop a design that involves two concatenated adaptive observers, covering the two subintervals of the PDE domain. The proposed design employs one extra sensor, providing the measurement of the PDE state at an inner position close to the ODE‐PDE junction point. Both observers are shown to be exponentially convergent, under ad‐hoc persistent excitation (PE) conditions, with no limitation on the Lipschitz coefficient and domain length.

Finite-time parameter estimation for an online monitoring of transformer: A system identification perspective

The effective online monitoring of thermal performance is the principal criterion that determines the Loss-of-Life (LoL), ageing rate, and loading capability of the transformer. To assess transformer thermal performance and life expectancy, Top-oil Temperature (TOT) and Hot spot Temperature (HST) should be accurately estimated. In general, the first-principle approach with a thermal-electrical analogy is used for the first-order dynamical model based on the Resistance-Capacitance (RC) circuit structure to approximate the evolution of thermal performance. Traditionally, the TOT model parameters are identified by looking for the input-output data that minimize the error between estimated and actual values. The Gradient Estimator (GE) based on the least-square minimization principle ensures the parameter convergence to their actual value and the parametric estimation error to zero only when the regressor (information vector) signals fulfill the stringent Persistence of Excitation (PE) condition. As the choice of input-output data plays crucial role in parameter estimation, the Design of Experiment (DoE) is generally conducted in the laboratory to satisfy the PE condition. Therefore, the TOT model parameter estimation problem is reformulated from a system identification perspective by exploring the finite-time estimators (FTEs) that accurately identify and estimate the TOT model parameters for the streaming data from real-time working transformers without any laboratory experiments and DoE. The experimental analysis on the sufficiency of data for parameter convergence is carried out on MATLAB to demonstrate the identified PE problem, the effect of DoE, and the performance of finite time estimators for real-time scenario based non-PE data in thermal modeling of the transformer.

Instrumental variable‐based multi‐innovation gradient estimation for nonlinear systems with scarce measurements

SummaryThis article considers the identification problems of nonlinear systems with scarce measurements by using the instrumental variable technique. When the product of the instrumental matrix and the information matrix is a nonsingular matrix and the weak persistent excitation condition about the instrumental vector is true, the obtained parameter estimates can be unbiased consistent estimates. The key is how to choose the instrumental variables. Difficulty arises in that the system outputs are unavailable. By applying the negative gradient search, a recursive instrumental variable‐based gradient algorithm is derived to estimate the parameters of the nonlinear systems with missing observed data. Moreover, the multi‐innovation identification theory is introduced to further improve the parameter estimation accuracy. The simulation results illustrate that the proposed methods are effective.

Optimal tracking control of mechatronic servo system using integral reinforcement learning

Recent years, there has been an increasing interest in the application of reinforcement learning (RL) to nonlinear system control. And many published studies have shown that reusing historical data can improve the learning efficiency of intelligent agents. However, reusing historical data leads to a large increase in the data to be processed. This paper proposes a novel adaptive law for the critic network in the RL framework to address this problem. The innovation of the proposed method is the introduction of an auxiliary matrix containing information about the historical estimation errors instead of the historical data itself. Moreover, the addition of the auxiliary matrix allows the persistent excitation (PE) conditions to be checked online. As a validation, the proposed approach is applied to a motor servo system containing unknown dynamics. Simulation and experimental results demonstrate the effectiveness of the proposed controller in the end.

Fault Identification for a Class of Nonlinear Systems of Canonical Form via Deterministic Learning.

In this article, through a combination of the deterministic learning (DL) method and the adaptive high gain observer (AHGO) technology, a fault identification approach for a class of nonlinear systems in canonical form is proposed. By using the DL method, the partial persistent excitation condition of the identification system is satisfied, and then, the AHGO technology is exploited to estimate the states and the neural network weights simultaneously. To analyze the convergence of the proposed method, we first analyze the uniformed completely observability (UCO) property of the linear part of the nonlinear identification system. Then, by using the Lipschitz property of the nonlinear item and the Bellman-Gronwall lemma, we show that the UCO property of the nonlinear identification system is depended on the UCO property of the linear part when the observer gain is chosen large. Therefore, by using the UCO property of the nonlinear identification system and the Lyapunov stability theorem, the convergence of the proposed learning observer is proven. The attraction of this article is based on the analysis of the UCO property of the identification system, and the convergence of the proposed learning observer can be directly proven. The simulation example is given to demonstrate the effectiveness of the proposed method.

Optimized Leader-Follower Consensus Control Using Reinforcement Learning for a Class of Second-Order Nonlinear Multiagent Systems

In this article, an optimized leader-follower consensus control is proposed for a class of second-order unknown nonlinear dynamical multiagent system. Different with the first-order multiagent consensus, the second-order case needs to achieve the agreement not only on position but also on velocity, therefore this optimized control is more challenging and interesting. To derive the control, reinforcement learning (RL) can be a natural consideration because it can overcome the difficulty of solving the Hamilton–Jacobi–Bellman (HJB) equation. To implement RL, it needs to iterate both adaptive critic and actor networks each other. However, if this optimized control learns RL from most existing optimal methods that derives the critic and actor adaptive laws from the negative gradient of square of the approximating function of the HJB equation, this control algorithm will be very intricate, because the HJB equation correlated to a second-order nonlinear multiagent system will become very complex due to strong state coupling and nonlinearity. In this work, since the two RL adaptive laws are derived via implementing the gradient descent method to a simple positive function, which is obtained on the basis of a partial derivative of the HJB equation, this optimized control is significantly simple. Meanwhile, it not merely can avoid the requirement of known dynamic acknowledge, but also can release the condition of persistent excitation, which is demanded in most RL optimization methods for training the adaptive parameter more sufficiently. Finally, the proposed control is demonstrated by both theory and computer simulation.

Finite-time composite learning control for trajectory tracking of dynamic positioning vessels

This paper studies the trajectory tracking control method for dynamic positioning vessels with state constraints and parameter uncertainties. A composite learning method is introduced to identify unknown parameters online. Regressor filtering together with dynamic regressor extension and mixing procedure are combined not only to relax the dependence of parameter estimation convergence process on the persistent excitation condition, but also to ensure the independence and flexibility of each parameter. Subsequently, a finite-time composite learning controller is designed based on asymmetric integral barrier Lyapunov functions, which guarantees the asymmetric constraints on vessel states. Furthermore, Lyapunov stability shows that the parameter identification and tracking errors can converge to zero in finite-time. Finally, tracking control task for dynamic positioning systems is carried out to illustrate the merits of the proposed method.

Dynamic learning from neural network‐based control for sampled‐data strict‐feedback nonlinear systems

AbstractThis article focuses on the dynamic learning and control problem for a class of nonlinear sampled‐data systems in strict‐feedback form. To achieve learning of unknown system dynamics in closed‐loop control, two obstacles need to be overcome: the inherent noncausal problem in the backstepping design of discrete‐time nonlinear systems, and the exponential stability of the sampling closed‐loop error system. First, a novel command filtered adaptive neural control is developed to achieve closed‐loop stability and the convergence of output tracking error based on Lyapunov theory, the noncausal problem is overcome by utilizing the command filter technique, and the ‐step delay is avoided in the recursive process. Second, by combining a state transformation method, the closed‐loop error system is divided into perturbed linear time‐varying (LTV) subsystems. Based on the convergence of the tracking errors, the partial persistent excitation (PE) condition of the radial basis function network is verified and established recursively. By extending the exponential stability result of a class of discrete‐time LTV systems, the convergence of the estimated weights to their optimal values is guaranteed with the PE condition. Subsequently, locally accurate approximation/learning of the unknown closed‐loop sampled dynamics can be obtained and stored in the form of constant neural networks. Finally, by reutilizing the acquired knowledge, a learning‐based control scheme for the sampled‐data system is further proposed to pursue improved control performance. Simulation studies are performed to illustrate the validity of the proposed scheme.

Reinforcement learning‐based tracking control for a quadrotor unmanned aerial vehicle under external disturbances

AbstractThis article addresses the high‐accuracy intelligent trajectory tracking control problem of a quadrotor unmanned aerial vehicle (UAV) subject to external disturbances. The tracking error systems are first reestablished by utilizing the feedforward control technique to compensate for the raw error dynamics of the quadrotor UAV. Then, two novel appointed‐fixed‐time observers are designed for the processed error systems to reconstruct the disturbance forces and torques, respectively. And the observation errors can converge to origin within the appointed time defined by users or designers. Subsequently, two novel control policies are developed utilizing reinforcement learning methodology, which can balance the control cost and control performance. Meanwhile, two critic neural networks are used to replace the traditional actor‐critic networks for approximating the solutions of Hamilton–Jacobi–Bellman equations. More specifically, two novel weight update laws are developed. They can not only update the weights of the critic neural networks online, but also avoid utilizing the persistent excitation condition innovatively. And that the ultimately uniformly bounded stability of the whole control system is proved according to Lyapunov method by utilizing the proposed reinforcement learning‐based control polices. Finally, simulation results are presented to illustrate the effectiveness and superior performances of the developed control scheme.

Adaptive-Critic Design for Decentralized Event-Triggered Control of Constrained Nonlinear Interconnected Systems Within an Identifier-Critic Framework.

This article studies the decentralized event-triggered control problem for a class of constrained nonlinear interconnected systems. By assigning a specific cost function for each constrained auxiliary subsystem, the original control problem is equivalently transformed into finding a series of optimal control policies updating in an aperiodic manner, and these optimal event-triggered control laws together constitute the desired decentralized controller. It is strictly proven that the system under consideration is stable in the sense of uniformly ultimate boundedness provided by the solutions of event-triggered Hamilton-Jacobi-Bellman equations. Different from the traditional adaptive critic design methods, we present an identifier-critic network architecture to relax the restrictions posed on the system dynamics, and the actor network commonly used to approximate the optimal control law is circumvented. The weights in the critic network are tuned on the basis of the gradient descent approach as well as the historical data, such that the persistence of excitation condition is no longer needed. The validity of our control scheme is demonstrated through a simulation example.

Dynamic Learning From Adaptive Neural Control for Discrete-Time Strict-Feedback Systems.

This article first investigates the issue on dynamic learning from adaptive neural network (NN) control of discrete-time strict-feedback nonlinear systems. To verify the exponential convergence of estimated NN weights, an extended stability result is presented for a class of discrete-time linear time-varying systems with time delays. Subsequently, by combining the n -step-ahead predictor technology and backstepping, an adaptive NN controller is constructed, which integrates the novel weight updating laws with time delays and without the σ modification. After ensuring the convergence of system output to a recurrent reference signal, the radial basis function (RBF) NN is verified to satisfy the partial persistent excitation condition. By the combination of the extended stability result, the estimated NN weights can be verified to exponentially converge to their ideal values. The convergent weight sequences are comprehensively represented and stored by constructing some elegant learning rules with some novel sequences and the mod function. The stored knowledge is used again to develop a neural learning control scheme. Compared with the traditional adaptive NN control, the proposed scheme can not only accomplish the same or similar tracking tasks but also greatly improve the transient control performance and alleviate the online computation. Finally, the validity of the presented scheme is illustrated by numerical and practical examples.

Adaptive Tracking Control of Hydraulic Systems With Improved Parameter Convergence

Most recent studies on adaptive hydraulic tracking control focus on the trajectory tracking performance while the parameter convergence property is often unsatisfying. This article proposes a composite learning adaptive position tracking controller with improved parameter convergence for electro-hydraulic servo systems. In the composite learning, a prediction error is formulated to exploit input–output memory data, and parameter estimates are driven simultaneously by tracking and prediction errors. Practical exponential stability of the closed-loop system, which implies the convergence of both the tracking and parameter estimation error, is established by a more realizable interval-excitation condition than the stringent persistent–excitation condition. Therefore, superior trajectory tracking is obtained compared with the classical adaptive hydraulic control. Besides, the initial fluid control volumes of hydraulic systems are assumed to be unknown <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">a priori</i> , which enhances the generality of the proposed control approach. The abovementioned two properties are generally not achievable in prevalent approaches to adaptive hydraulic control. Moreover, noisy acceleration signals and the time derivatives of pressure signals are not needed in the proposed approach, which improves its robustness against measurement noise. Extensive experimental results verify its superiority over currently available ones.

Adaptive Boundary Observer Design for Coupled Parabolic PDEs With Different Diffusions and Parameter Uncertainty

This paper focuses on establishing an adaptive boundary observer for fully coupled reaction-diffusions partial differential equations (PDEs) containing unknown parameters. Firstly, the state error system is transformed into an intermediate system by finite dimensional backstepping-like transformation, and then we convert the intermediate system to a target system by infinite dimensional backstepping transformation. Secondly, a least-squares type parameter adaptive law is given. Thirdly, exploiting an ad hoc persistent excitation condition, the exponentially convergent of the observer will be established by applying Lyapunov functional. Finally, we use simulation analysis for Chemical Tubular Reactor model to demonstrate the validity of the results.

Adaptive tracking control for stochastic nonlinear systems with time-varying delays using multi-dimensional Taylor network

For stochastic nonlinear systems (SNSs) perturbed by compound uncertainties, the conventional model-based control approaches assume that the evolution behavior of uncertain variables is known. Unfortunately, such approaches are often conservative for most practical scenarios with the slow convergence speed and unsatisfactory anti-interference performance. For this sake, an adaptive control scheme based on deep deterministic policy gradient (DDPG) and multi-dimensional Taylor network (MTN) is proposed here to address the tracking problem for a category of SNSs subject to fast time-varying uncertainties, stochastic disturbance and unknown time-varying delays. The effect of time delay is embedded in the reproducing kernel Hilbert space through the error coordinate transformation. In the framework of DDPG, the MTN-based surrogate is utilized to construct the online network and target network via the temporal-difference method, which promises more desirable real-time performance due to its concise structure than conventional NN-based surrogates. In order to enhance the robustness of the system under fast time-varying uncertainties, a novel persistent excitation (PE) mechanism is designed to ensure that the control policy is appropriately rewarded or punished. Based on the PE condition, weights of MTNs converge exponentially and animate the system to evolve towards the target persistently. The tracking error and closed-loop state signals are proved theoretically to be uniformly ultimately bounded (UUB) via Lyapunov–Krasovskii functional. A numerical simulation from the process industry verifies the effectiveness of the proposed method.

Adaptive State Observer for Linear Time-Varying System with Partially Unknown State Matrix and Input Matrix Parameters

In this paper the problem of adaptive state observer synthesis for linear time-varying SISO (single-input-single-output) dynamical system with partially unknown parameters was considered. It is assumed that the input signal and output variable of the system are measurable. It is also assumed that the state matrix of the plant contains known variables and unknown constants when the input matrix (vector) is unknown. Observer synthesis is based on GPEBO (generalized parameter estimation based observer) method proposed in [1]. Observer synthesis provides preliminary parametrization of the initial system and its conversion to a linear regression model with further unknown parameters identification. For identification of the unknown constant parameters classical estimation algorithm — least squares method with forgetting factor — was used. This approach works well in cases, when the known regressor is " frequency poor" (i.e. the regressor spectrum contains r/2 harmonics, where r is a value of the unknown parameters) or does not meet PE (persistent excitation) condition. To illustrate performance of the proposed method, an example is provided in this paper. A time-varying second-order plant with four unknown parameters was considered. Parametrization of the initial dynamical model was made. A linear static regression with six unknown parameters (including unknown state initial conditions vector) was obtained. An adaptive observer was synthesized and the simulation results were provided to illustrate the purpose reached. The main difference with the results, that were published earlier in [2], is the new assumption that not only does the state matrix of the linear time-varying system contain unknown parameters, but input matrix (vector) contains unknown constant coefficients.