Persistent Excitation Research Articles

Adaptive tracking control for stochastic nonlinear systems with time-varying delays using multi-dimensional Taylor network

For stochastic nonlinear systems (SNSs) perturbed by compound uncertainties, the conventional model-based control approaches assume that the evolution behavior of uncertain variables is known. Unfortunately, such approaches are often conservative for most practical scenarios with the slow convergence speed and unsatisfactory anti-interference performance. For this sake, an adaptive control scheme based on deep deterministic policy gradient (DDPG) and multi-dimensional Taylor network (MTN) is proposed here to address the tracking problem for a category of SNSs subject to fast time-varying uncertainties, stochastic disturbance and unknown time-varying delays. The effect of time delay is embedded in the reproducing kernel Hilbert space through the error coordinate transformation. In the framework of DDPG, the MTN-based surrogate is utilized to construct the online network and target network via the temporal-difference method, which promises more desirable real-time performance due to its concise structure than conventional NN-based surrogates. In order to enhance the robustness of the system under fast time-varying uncertainties, a novel persistent excitation (PE) mechanism is designed to ensure that the control policy is appropriately rewarded or punished. Based on the PE condition, weights of MTNs converge exponentially and animate the system to evolve towards the target persistently. The tracking error and closed-loop state signals are proved theoretically to be uniformly ultimately bounded (UUB) via Lyapunov–Krasovskii functional. A numerical simulation from the process industry verifies the effectiveness of the proposed method.

Adaptive State Observer for Linear Time-Varying System with Partially Unknown State Matrix and Input Matrix Parameters

In this paper the problem of adaptive state observer synthesis for linear time-varying SISO (single-input-single-output) dynamical system with partially unknown parameters was considered. It is assumed that the input signal and output variable of the system are measurable. It is also assumed that the state matrix of the plant contains known variables and unknown constants when the input matrix (vector) is unknown. Observer synthesis is based on GPEBO (generalized parameter estimation based observer) method proposed in [1]. Observer synthesis provides preliminary parametrization of the initial system and its conversion to a linear regression model with further unknown parameters identification. For identification of the unknown constant parameters classical estimation algorithm — least squares method with forgetting factor — was used. This approach works well in cases, when the known regressor is " frequency poor" (i.e. the regressor spectrum contains r/2 harmonics, where r is a value of the unknown parameters) or does not meet PE (persistent excitation) condition. To illustrate performance of the proposed method, an example is provided in this paper. A time-varying second-order plant with four unknown parameters was considered. Parametrization of the initial dynamical model was made. A linear static regression with six unknown parameters (including unknown state initial conditions vector) was obtained. An adaptive observer was synthesized and the simulation results were provided to illustrate the purpose reached. The main difference with the results, that were published earlier in [2], is the new assumption that not only does the state matrix of the linear time-varying system contain unknown parameters, but input matrix (vector) contains unknown constant coefficients.

Adaptive Control With Guaranteed Transient Behavior and Zero Steady-State Error for Systems With Time-Varying Parameters

It is nontrivial to achieve global zero-error regulation for uncertain nonlinear systems. The underlying problem becomes even more challenging if mismatched uncertainties and unknown time-varying control gain are involved, yet certain performance specifications are also pursued. In this work, we present an adaptive control method, which, without the persistent excitation (PE) condition, is able to ensure global zero-error regulation with guaranteed output performance for parametric strict-feedback systems involving fast time-varying parameters in the feedback path and input path. The development of our control scheme benefits from generalized <tex xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">$t$</tex> -dependent and <tex xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">$x$</tex> -dependent functions, a novel coordinate transformation and “congelation of variables” method. Both theoretical analysis and numerical simulation verify the effectiveness and benefits of the proposed method.

Performance analysis of the compressed distributed least squares algorithm

In this paper, we consider the distributed estimation problem of unknown high-dimensional sparse signals for a random dynamic system. We propose a compressed distributed algorithm by using the compressive sensing theory and the distributed least squares (LS) algorithm. Under a compressed cooperative persistent excitation condition, the upper bound of the estimation error is established which is positively related to the restricted isometry constant. Our results are obtained without relying on some stringent conditions such as independency or stationarity of the regression vectors. Finally, we provide a simulation example to show that the compressed distributed least squares algorithm has better performance than the regularized distributed LS algorithm with l 1 penalty for the estimation of high-dimensional sparse signals. • High-dimensional sparse signal. • Compressed distributed least squares. • System identification. • Random dynamic system

Off-policy reinforcement learning-based novel model-free minmax fault-tolerant tracking control for industrial processes

For industrial processes with external disturbance and actuator failure, off-policy reinforcement learning-based novel model-free minmax fault-tolerant control is proposed in this paper to solve H∞ fault-tolerant tracking control problem. An augmented model equivalent to the original system is constructed, and the state of the new augmented model is composed of state increment and tracking error of the original system. The original H∞ fault-tolerant tracking problem was transformed into the linear quadratic zero-sum game problem by establishing performance index function, and the Game Algebraic Riccati Equation (GARE) was established. Then Q function was introduced and the Off-policy reinforcement learning algorithm was designed. Different from the traditional model-based fault-tolerant control method, the proposed algorithm does not need the knowledge of system dynamics, and it can learn from the measured data of the system trajectory to solve the GARE. In addition, it is proved that the probing noise added to satisfy the persistent excitation condition does not cause bias. A simulation example of injection molding process is used to verify the effectiveness of the proposed algorithm.

Optimized tracking control based on reinforcement learning for a class of high-order unknown nonlinear dynamic systems

The article is to develop an optimized tracking control using fuzzy logic system (FLS)-based reinforcement learning (RL) for a class of unknown nonlinear dynamic single-input–single-output (SISO) system under canonical form. From mathematical viewpoint, optimal nonlinear control depends on the solution of Hamilton–Jacobi-Bellman (HJB) equation, but finding the equation’s analytic solution is almost impossible because of the strong nonlinearity. For addressing the problem in this work, the RL approximation strategy is employed by on-line iterating both critic and actor FLSs, where critic FLS aims for evaluating control performance and making feedback to actor, and actor FLS aims for executing the modified control behavior. Different with the previous RL-based optimization approach, the proposed approach derives RL training laws from negative gradient of a simple positive function rather than the square of HJB equation’s approximation. As a result, the control algorithm can be significantly simplified. Furthermore, it can avoid two general conditions, persistence excitation and known dynamic, required in most RL optimal control methods. Finally, the proposed optimized control is demonstrated from both aspects of theory proof and computation simulation.

Novel Adaptive Extended State Observer for Dynamic Parameter Identification with Asymptotic Convergence

In this paper, a novel method of parameter identification of linear in parameter dynamic systems is presented. The proposed scheme employs an Extended State Observer to online estimate a state of the plant and momentary value of total disturbance present in the system. A notion is made that for properly redefined dynamics of the system, this estimate can be interpreted as a measure of modeling error caused by the parameter uncertainty. Under this notion, a disturbance estimate is used as a basis for classic gradient identification. A global convergence of both state and parameter estimates to their true values is proved using the Lyapunov approach under an assumption of a persistent excitation. Finally, results of simulation and experiments are presented to support the theoretical analysis. The experiments were conducted using a compliant manipulator joint and obtained results show the usefulness of the proposed method in drive control systems and robotics.

Reinforcement learning-based optimised control for a class of second-order nonlinear dynamic systems

This paper presents an optimised tracking control scheme based on reinforcement learning (RL) for a class of second-order nonlinear systems with unknown dynamics. Different from the first-order dynamic system control, the second-order case is required to synchronously steer two variables of position and velocity, hence it makes this optimised control more challenging to accomplish. To achieve the optimised control, first, neural network (NN) is employed to approximate the solution of Hamilton–Jacobi–Bellman (HJB), and then an RL is performed by constructing both critic and actor based on the NN approximation. Since the RL training laws are derived from the negative gradient of a simple positive function generated in accordance with the partial derivative of HJB equation, it can make the control algorithm significantly simple to compare with the existing optimal control methods. Meanwhile, it can also release the condition of persistence excitation and compensate for the nonlinear uncertainty. Finally, the proposed adaptive control method can guarantee the desired results that are demonstrated by theorem, proof and simulation.

Composite learning sliding mode control of uncertain nonlinear systems with prescribed performance

This paper explores the prescribed performance tracking control problem of nonlinear systems with triangular structure. To obtain the desired transient performance and precise estimations of uncertain terms, the techniques of neural network control, sliding mode control and composite learning control are incorporated into the proposed control method. The presented control strategy can ensure the tracking error converges to a prescribed small residual set. Compared with the persistent excitation condition required in the conventional adaptive control, the interval excitation condition needed in the proposed control approach is weak, which guarantees that the radial basis function neural networks approximate the unknown nonlinear terms more accurately. Finally, two simulation examples are exploited to manifest the effectiveness of the proposed approach.

Finite-time adaptive optimal consensus control for multi-agent systems subject to time-varying output constraints

In this paper, a finite-time optimal consensus control strategy is presented for unknown multi-agent systems (MASs) with the time-varying asymmetric output constraint. Different from existing results, the output constraint problem investigated here eliminates the requirements that constraint boundary functions must be strictly non-zero and have different signs, which is successfully handled by introducing special barrier functions. Moreover, to deal with disturbances well, a reinforcement learning (RL) with the critic-actor-disturbance structure is introduced. Meanwhile, the weights of neural networks are adjusted online by applying the gradient descent method to positive functions newly constructed, which not only significantly simplifies the algorithm but also eliminates the persistent excitation condition. For obtaining a fast convergence rate, the finite-time control technique is embedded into the RL algorithm, and an effective finite-time optimal control scheme is proposed to achieve the consistency of multi-agent system in a finite time. Finally, the effectiveness of the proposed protocol is demonstrated by two simulation examples.

Relaxed bearing rigidity and bearing formation control under persistence of excitation

This paper addresses the problem of time-varying bearing formation control in d(d≥2)-dimensional Euclidean space by exploring Persistence of Excitation (PE) of the desired bearing reference. A general concept of Bearing Persistently Exciting (BPE) formation defined in d-dimensional space is here fully developed. By providing a desired formation that is BPE, distributed control laws for multi-agent systems under both single- and double-integrator dynamics are proposed using bearing measurements (along with velocity measurements when the agents are described by double-integrator dynamics), which guarantee uniform exponential stabilization of the desired formation up to a translation. A key contribution of this work is to show that the classical bearing rigidity condition on the graph topology, required for achieving the stabilization of a formation up to a translation and a scale factor, is relaxed and extended in a natural manner by exploring PE conditions imposed either on a specific set of desired bearing vectors or on the whole desired formation. Simulation results are provided to illustrate the performance of the proposed control method.

Synchronization analysis of linear time-varying matrix-weighted coupled systems and its applications

In this paper, we study the synchronization problem of continuous-time linear systems under time-varying matrix-weighted coupling. With the help of the notion of persistent excitation and an adapted stability result in adaptive control, we establish two sufficient exponential synchronization conditions using the connectivity of the persistent graph and the joint connectivity of the instantaneous time-varying graphs. The conditions recover those for the single-integrator consensus model with scalar-valued coupling. The results are further applied to solve the synchronization problem of continuous-time linear systems with time-varying and nonidentical coupling matrices, extending the existing conclusions in the literature. Finally, numerical examples are given to verify the derived results.

Stable robot manipulator parameter identification: A closed-loop input error approach

This paper presents an on-line parametric estimation method for robot manipulators. The identification algorithm estimates the parameters by using the input error between the robot and a parallel estimated model. Both, the robot and the estimated model are controlled by two Proportional–Derivative (PD) controller tuned with the same gain values, and a persistent excitation (PE) signal for ensuring parameters convergence is included. The exact model matching and the estimation error cases are analysed. Noisy state measurements and filters are avoided in the model parameterization by using only the states of the estimated model. A second parameter identification algorithm, which is based on a composite update law, is also proposed. It improves parameters convergence and robustness of the update rule in presence of estimation errors. The stability of the closed-loop dynamics related to the estimated model is assessed via Lyapunov stability theory. Simulations are carried out to validate the proposed approaches.

Quantized‐output‐based least squares of ARX systems

SummaryThis paper considers the estimates of parameters of stochastic autoregressive exogenous input (ARX) systems with quantized data. The quantized error caused by quantized data brings difficulties to the parameter estimation of ARX systems. Therefore, I start with the boundedness of quantized error and design the size of quantized error thanks to good properties of the least squares. Then the influences of the noise of the system itself and the bounded noise caused by quantized error on the parameter estimation error are analyzed, respectively. The inputs of the system are designed to make the system satisfy the persistent excitation condition, so as to ensure the boundedness of the parameter estimation error. A numerical example is given to demonstrate the theorem.

Adaptive size-independent control of uncertain leader following systems with only relative displacement information

The adaptive robust size-independent consensus problem of leader following multi-agent systems (MASs) with unknown dynamics based on only onboard relative displacement information is studied. It is assumed that all followers only can measure the relative displacement regarding their adjacent agents. Due to lack of communication tools, each follower must estimate its uncertain dynamics and the leader acceleration simultaneously. A decentralized adaptive robust consensus law is presented to achieve the global state consensus for uncertain second-order multi-agent systems by employing only relative displacement information. According to the Lyapunov theorem, it is proved that without any knowledge about the number of followers and communication graph, the global state consensus is attained. The control and adaptive gains are tuned independently of the number of followers and communication topology. Therefore, the controller is robust against size-changing and can be applied to large-scale uncertain MASs. Afterward, it is verified that if the basis vectors are persistently exciting (PE), the estimation errors of uncertain vectors tend to zero. Several numerical scenarios are prepared to confirm the efficiency of this method.

Optimal detumbling strategy for a non-cooperative target with unknown inertial parameters using a space manipulator

Capturing tumbling target is considered as one of the greatest challenges in on-orbit service missions due to tumbling motion and parameter uncertainty of the space non-cooperative target. The tumbling motion of the target should be attenuated as quickly as possible before grasping to avoid the unexpected collisions and possible damages to the space manipulator or the target, which means the shortest-time detumbling trajectory is necessary. In many studies about the optimal detumbling trajectory planning the inertial parameters of the target have been assumed to be accurately known, which is usually difficult for a non-cooperative target so that parameter identification is needed before planning a optimal detumbling trajectory. In this paper, a novel strategy for parameter identification and detumbling of a tumbling target by a space manipulator is proposed. The strategy includes two phases. First, unknown inertial parameters of the target are identified in finite-time by a new estimation method with the assumption of interval excitation (IE), which relaxes the strict persistent excitation (PE) condition required in most prior studies. Second, the shortest-time detumbling trajectory of the tumbling target is generated by formulating a time-optimal control problem (OCP) in which the target attitude bounds and the limitations of the contact force as well as the relative velocity between the space manipulator and the target are represented as extended dynamical sub-systems and saturation functions. Then, the Calculus of Variations method is used to solve the OCP to obtain a high accuracy solution. During two phases, the contact force between the space manipulator and the target is utilized to generate external moment for identifying the inertia parameters of the target and detumbling the target. Meanwhile, a unified hybrid impedance control framework is applied to the space robot to track the desired contact force and motion trajectory. The whole strategy is verified by a space manipulator capturing a tumbling target, and numerical simulations demonstrate the effectiveness of the proposed methods.

On consistency of least squares algorithm for ARMAX time-delay models

Jointly identification of the sample delay and the parameters of time-delay discrete-time ARMAX models are addressed in this article. Like any identified problem, the proposed approach is based on the solution of an optimization problem. It consists then of two main steps: the first is the problem formulation and the second is the problem resolution. The problem formulation defines the criterion to be minimized according to an unknown parameter vector consisting of the sample delay and the parameters with their corresponding observation vector. Since the sample delay is of integer type, the obtained optimization problem cannot be efficiently solved using a real optimization algorithm. To overcome this difficulty, the rounding properties are used to transform the obtained problem into a real optimization problem. The solution is then done using the least square approach. Consistency of the estimates with their convergence rates is established under the persistent excitation condition. Finally, some numerical simulation and experimental results are presented to illustrate the performance of the proposed method.

Time-optimal robotic manipulation on a predefined path of loosely placed objects: Modeling and experiment

A consistent method is presented for solving the so-called general waiter problem, which is to move a tray (mounted at the end-effector of a robot) with a number of loosely placed objects (e.g. cups) as fast as possible such that the objects do not slide at any time. The geometric path of the tray motion is prescribed while the attitude of the tray must vary in order to avoid sliding or tip-over of the objects. The time optimal robot trajectory is determined with a multiple shooting method. For the considered wrist-partitioned robot, the motion is described by the path parameter and the three joint coordinates of the wrist. This 4-parametric description is beneficial for solving the optimal control problem with improved convergence. The optimization takes into account the technical limitations of the robot as well as the limiting friction between objects and tray. Experimental results for a time-optimal motion of a Stäubli RX130L manipulating a tray with 4 cups are presented. A crucial aspect is the use of a model-based control strategy along with the identification of dynamic parameters. Moreover, the basis for any optimization and real-time control is a reliable dynamic model of the robot. Therefore a parameter identification is performed using optimized persistent excitation trajectories.

Generalized recursive least squares: Stability, robustness, and excitation

We study a class of recursive least-squares estimators in an errors-in-variables setting where disturbances affect both the regressor and the regressand variables. We prove the existence and stability of an optimal steady state and robustness with respect to the disturbances in form of input-to-state and input–output stability relative to the unperturbed steady-state trajectories. Depending on the choice of some design parameters, different specific estimators can be realized within the considered class, each of which is associated with a different underlying optimization problem and with different excitation requirements for the unperturbed regressor. As expected, we find that persistence of excitation is associated with uniform, in fact exponential, convergence. In addition, we also show that choices of the design parameters are possible for which convergence and robustness hold without persistence of excitation and with the same asymptotic gain, the only difference being a loss of uniformity in the convergence rate.

Data-Driven Immersion and Invariance Adaptive Attitude Control for Rigid Bodies With Double-Level State Constraints

This article investigates the attitude control problem of a rigid body subject to attitude and angular rate constraints (double-level state constraints), and inertia uncertainties. A data-driven immersion and invariance (I&I) adaptive control scheme is proposed to tackle this technically challenging problem. As a stepping stone, a novel dynamically scaled I&I adaptive controller is designed to bypass the realizability condition that may not hold in the Lyapunov sense when considering angular rate constraints. Lyapunov stability analysis shows that this controller can enable the attitude errors and angular rates to converge asymptotically to zero for most initial conditions in the accessible space, while strictly obeying double-level state constraints with the help of two judiciously constructed potential functions. After that, to further relax the dependence of parameter convergence on the persistent excitation (PE) condition, the I&I adaptive law is extended to a data-driven counterpart through adding a learning term that is acquired by adopting the regressor filtering in conjunction with the dynamic regressor extension and mixing (DREM) procedure. The extended adaptive controller can not only preserve all the results obtained by the earlier proposed I&I adaptive controller, but also ensure asymptotic parameter convergence under a finite excitation condition much weaker than PE. In addition, benefiting from the DREM method and some special designs, the parameter convergence rates across all the parameter vector components can be flexibly tuned in an explicit way, and moreover, they are independent of the excitation level. Finally, simulation results are given to show the effectiveness of the proposed method.