Persistent Excitation Research Articles

Adaptive critic learning for event‐triggered safe control of nonlinear safety‐critical systems

AbstractIn this paper, an event‐triggered safe control method based on adaptive critic learning (ACL) is proposed for a class of nonlinear safety‐critical systems. First, a safe cost function is constructed by adding a control barrier function (CBF) to the traditional quadratic cost function; the optimization problem with safety constraints that is difficult to deal with by classical ACL methods is solved. Subsequently, the event‐triggered scheme is introduced to reduce the amount of computation. Further, combining the properties of CBF with the ACL‐based event‐triggering mechanism, the event‐triggered safe Hamilton–Jacobi–Bellman (HJB) equation is derived, and a single critic neural network (NN) framework is constructed to approximate the solution of the event‐triggered safe HJB equation. In addition, the concurrent learning method is applied to the NN learning process, so that the persistence of excitation (PE) condition is not required. The weight approximation error of the NN and the states of the system are proven to be uniformly ultimately bounded (UUB) in the safe set with the Lyapunov theory. Finally, the availability of the presented method can be validated through the simulation.

Filtering‐based concurrent learning adaptive attitude tracking control of rigid spacecraft with inertia parameter identification

SummaryThis paper investigates the attitude tracking control problem of rigid spacecraft with inertia parameter identification. Based on the relative attitude and angular velocity error dynamics, a basic adaptive backstepping based attitude tracking control scheme is firstly designed such that asymptotic attitude tracking can be achieved. However, the parameter identification error cannot decay to zero if the persistent excitation (PE) condition is not satisfied. To solve this issue, a filtering‐based concurrent learning adaptive backstepping control scheme is then proposed, by incorporating torque filtering technique with concurrent learning technique. A more mild rank condition, which consists of some collectable historical data, is provided to guarantee the convergence of parameter identification error. In addition, a valid data collection algorithm is given. It should be mentioned that a distinctive feature of the proposed filtering‐based concurrent learning adaptive control scheme is that the convergence rate of attitude tracking errors can be improved from asymptotical to exponential. Finally, simulation results are provided to illustrate the effectiveness of the proposed control schemes.

Distributed adaptive estimation without persistence of excitation: An online optimization perspective

SummaryThis work proposes a distributed adaptive estimation algorithm for multi‐agent system (MAS) architecture using online measurements in a continuous‐time setting. Classical adaptive parameter estimation algorithms demand to satisfy persistence of excitation (PE) or its distributed variant cooperative persistent of excitation (C‐PE) for parameter convergence. PE, C‐PE, are restrictive in nature since it requires excitation (richness of information content) over the entire time span of the signal/data, making it unrealistic in most real‐world applications, especially in robotics and cyber‐physical systems. Unlike past literature, the proposed work ensures parameter convergence under a slackened condition, coined as cooperative initial excitation (C‐IE), which demands information richness only in the initial time‐window (transient period) suitable for practical scenarios. A distributed differential parameter estimator algorithm is designed, which with the help of stable closed‐loop filter dynamics and strategic switching guarantees global exponential stability (GES) of the parameter estimation error dynamics in the sense of Lyapunov. The formulation is further augmented by providing an online optimization perspective. A novel cost function is constructed in such a way that the proposed distributed parameter estimator acts as a distributed continuous‐time gradient‐descent algorithm based on the cost function and the true uncertain parameter vector is the global minima of the cost function. Simulation results validate the efficacy of the proposed algorithm in contrast to the PE/C‐PE based methods.

Adaptive anti‐disturbance sampling control of autonomous surface vehicles based on discrete‐time concurrent learning extended state observers

SummaryThis paper investigates the surge speed tracking control of an autonomous surface vehicle (ASV) subject to fully unknown internal dynamic, external disturbance, and unknown control input gain. An adaptive anti‐disturbance sampling control method is proposed for an ASV without using any model parameters. Specifically, by utilizing real‐time and historical data, discrete‐time reduced‐order and full‐order concurrent learning extended state observers (CLESOs) are designed to estimate the unknown ASV model parameters and ensure estimation convergence without requiring persistent excitation. Then, a surge speed tracking control law is designed based on the discrete‐time full‐order CLESO. Through Lyapunov stability analysis, the closed‐loop system is proven to be stable, and the estimation and tracking errors are bounded. Simulation and hardware‐in‐loop experimental results validate the effectiveness of the proposed discrete‐time CLESOs for the surge speed tracking of an ASV with fully unknown dynamic model.

Sufficient Conditions for Parameter Convergence Over Embedded Manifolds Using Kernel Techniques

The persistence of excitation (PE) condition is sufficient to ensure parameter convergence in adaptive estimation problems. Recent results on adaptive estimation in reproducing kernel Hilbert spaces (RKHS) introduce PE conditions for RKHS. This article presents sufficient conditions for PE for the particular class of uniformly embedded RKHS defined over smooth Riemannian manifolds. This article also studies the implications of the sufficient condition in the case when the RKHS is finite or infinite-dimensional. When the RKHS is finite-dimensional, the sufficient condition implies parameter convergence as in the conventional analysis. On the other hand, when the RKHS is infinite-dimensional, the same condition implies that the function estimate error is ultimately bounded by a constant that depends on the approximation error in the infinite-dimensional RKHS. We illustrate the effectiveness of the sufficient condition in a practical example.

Adaptive Optimal Motion Control of Uncertain Underactuated Mechatronic Systems With Actuator Constraints

Underactuated mechatronic systems are widely used in industrial production, where the control efforts and operation accuracy are both important aspects of performance evaluations. Hence, how to realize effective motion control, while reducing control efforts as much as possible, becomes an open problem for underactuated systems. Although some open loop approaches (e.g., trajectory planning) take energy optimization into account, they need linearization/approximation manipulations and exhibit weak robustness, which is prone to degrading practical control performance. To this end, this article proposes an adaptive tracking controller for uncertain multi-input-multi-output (MIMO) underactuated mechatronic systems, to fulfill accurate positioning/tracking control with saturated inputs and reduce control efforts as well. Particularly, by elaborately developing an auxiliary compensation term and a robust term, the proposed controller ensures <i>asymptotic convergence</i> of both <i>actuated</i> and <i>unactuated</i> variables. Meanwhile, the modified performance index function is approximated online and introduced into the Lyapunov function candidate to make the stability analysis process <i>more concise</i>. To the best of our knowledge, without the need of offline computation and the persistence of excitation (PE) condition, this article presents the <i>first</i> adaptive optimal controller to simultaneously achieve error elimination, control effort optimization, and actuator constraints for a class of underactuated systems. Finally, strict theoretical analysis and experimental validations show the effectiveness and robustness of the suggested controller.

On Preserving-Excitation Properties of Kreisselmeier’s Regressor Extension Scheme

In this article, we consider the excitation preservation problem of Kreisselmeier’s regressor extension scheme. We analyze this problem within the context of the dynamic regressor extension and mixing procedure. The well-known qualitative result is that such a scheme preserves excitation. We perform a quantitative analysis and derive lower bounds on the resulting regressor signal considering both persistent and interval excitation cases. We also show that the resulting signal is excited if and only if the original regressor is. Studying the dynamics of the novel regressor, we provide a lower bound on its derivative. Illustrative simulations support our theoretical results.

Composite Observer-Based Optimal Attitude-Tracking Control With Reinforcement Learning for Hypersonic Vehicles.

This article proposes an observer-based reinforcement learning (RL) control approach to address the optimal attitude-tracking problem and application for hypersonic vehicles in the reentry phase. Due to the unknown uncertainty and nonlinearity caused by parameter perturbation and external disturbance, accurate model information of hypersonic vehicles in the reentry phase is generally unavailable. For this reason, a novel synchronous estimation is proposed to construct a composite observer for hypersonic vehicles, which consists of a neural-network (NN)-based Luenberger-type observer and a synchronous disturbance observer. This solves the identification problem of nonlinear dynamics in the reference control and realizes the estimation of the system state when unknown nonlinear dynamics and unknown disturbance exist at the same time. By synthesizing the information from the composite observer, an RL tracking controller is developed to solve the optimal attitude-tracking control problem. To improve the convergence performance of critic network weights, concurrent learning is employed to replace the traditional persistent excitation condition with a historical experience replay manner. In addition, this article proves that the weight estimation error is bounded when the learning rate satisfies the given sufficient condition. Finally, the numerical simulation demonstrates the effectiveness and superiority of the proposed approaches to attitude-tracking control systems for hypersonic vehicles.

Composite Learning Finite-Time Control of Robotic Systems With Output Constraints

In this article, for robotic systems with uncertain dynamics and time-varying asymmetric output constraints, we present a composite learning finite-time control scheme with salient features benefited from two design steps. In the first step, unlike existing composite adaptive/learning control algorithms, which result in either exponential convergence or finite-time convergence but exhibit a potential singularity issue, a modified nonsingular terminal sliding mode-based composite learning controller is adopted such that both tracking error and parameter estimation error converge to zero in finite time without singularity. The unknown parameter learning law is constructed by using online historical data and regressor extension, which gives a benefit of relaxing the typically required stringent persistent excitation with a much weaker excitation condition termed interval excitation. In the second step, a universal time-varying asymmetric barrier function (UTABF) is adopted to directly constrain the system output rather than the most commonly used barrier Lyapunov function, which indirectly converts the output constraints into the conservative tracking error constraints. Moreover, the UTABF can handle both constrained and unconstrained cases uniformly without the need for changing the control structure. Both theoretical analysis and experiments results on an industrial manipulator confirm the benefits and effectiveness of the proposed control scheme.

Distributed adaptive formation control of multi-agent systems with measurement noises

In this paper, we investigate the distributed adaptive formation control problem of leader-following multi-agent systems where the measurements are subject to noises. A distributed LMS-like algorithm is proposed to estimate the relative position between the followers and the leader. By employing the “certainty equivalence” principle, an adaptive motion controller is designed to achieve the desired formation under dynamically changing interaction topologies. The incompatibility in simultaneously satisfying the persistent excitation condition required by localization estimators and accomplishing the control tasks makes the theoretical analysis challenging. In order to deal with this issue, an excited noise signal is embedded into the control law. We give the upper bounds of both the estimation errors and formation error of the closed-loop multi-agent system under the distributed adaptive control law. Finally, the numerical simulations are provided to demonstrate the effectiveness of our theoretical results.

Event-driven adaptive intermittent control applied to a rotational pendulum

Intermittent control combines open-loop trajectories with feedback at discrete time instances determined by events. Among other applications, it has recently been used to model quiet standing in humans where the system was assumed to be time-invariant. This article expands this work to the time-variant case by introducing an adaptive intermittent controller that exploits the well-known self-tuning architecture of adaptive control with a Kalman filter to perform online state and parameter estimation. Simulation and experimental results using a rotational inverted pendulum show advantages of the intermittent controllers compared to continuous feedback control since the former can provide persistent excitation due to their internal triggering mechanism, even when no external reference changes or disturbances are applied. Moreover, the results show that the event thresholds of intermittent control can be used to adjust the degree of responsiveness of the adaptation in the system, becoming a tool to balance the trade-off between steady-state performance and flexibility against parametric changes, addressing the stability–plasticity dilemma of adaptation and learning in control.

Deterministic learning-based neural network control with adaptive phase compensation

Under the persistent excitation (PE) condition, the real dynamics of the nonlinear system can be obtained through the deterministic learning-based radial basis function neural network (RBFNN) control. However, in this scheme, the learning speed and accuracy are limited by the tradeoff between the PE levels and the approximation capabilities of the neural network (NN). Inspired by the frequency domain phase compensation of linear time-invariant (LTI) systems, this paper presents an adaptive phase compensator employing the pure time delay to improve the performance of the deterministic learning-based adaptive feedforward control with the reference input known a priori. When the adaptive phase compensation is applied to the hidden layer of the RBFNN, the nonlinear approximation capability of the RBFNN is effectively improved such that both the learning performance (learning speed and accuracy) and the control performance of the deterministic learning-based control scheme are improved. Theoretical analysis is conducted to prove the stability of the proposed learning control scheme for a class of systems which are affine in the control. Simulation studies demonstrate the effectiveness of the proposed phase compensation method.

Excitation for Adaptive Optimal Control of Nonlinear Systems in Differential Games

This article focuses on the fulfillment of the persistent excitation (PE) condition for signals which result from transformations by means of polynomials. This is essential, e.g., for the convergence of adaptive dynamic programming algorithms due to commonly used polynomial function approximators. As theoretical statements are scarce regarding the nonlinear transformation of PE signals, we propose conditions on the system state such that its transformation by polynomials is PE. To validate our theoretical statements, we develop an exemplary excitation procedure based on our conditions using a feed-forward control approach and demonstrate the effectiveness of our method in a nonzero-sum differential game. In this setting, our approach outperforms commonly used probing noise in terms of convergence time and the degree of PE, shown by a numerical example.

On the Design of Persistently Exciting Inputs for Data-Driven Control of Linear and Nonlinear Systems

In the context of data-driven control, persistence of excitation (PE) of an input sequence is defined in terms of a rank condition on the Hankel matrix of the input data. For nonlinear systems, recent results employed rank conditions involving collected input and state/output data, for which no guidelines are available on how to satisfy them a priori. In this paper, we first show that a set of discrete impulses is guaranteed to be persistently exciting for any controllable LTI system. Based on this result, for certain classes of nonlinear systems, we guarantee persistence of excitation of sequences of basis functions a priori, by design of the physical input only.

Two-Tier Filter-based Experience Replay for Neuro-Adaptive Control with Inherent Robustness

Neural network (NN) - based adaptive control is popularly used for controlling nonlinear dynamical systems with unstructured uncertainty. NN structures are embedded in the feedback loop with a real-time tuning mechanism to ensure desired tracking performance. Due to the presence of approximation error, neuro-adaptive controllers are affected by the phenomenon of parameter drift in the absence of persistence of excitation (PE) condition, and thereby require robust modifications like projection, σ-modification etc. to guarantee boundedness of the estimation error. The PE condition is stringent in nature since it lacks practical feasibility in most real world applications. Unlike conventional neuro-adaptive controllers, the proposed work relaxes the need for PE condition, while imposing a milder condition, called initial excitation (IE) for ensuring robust tracking. The algorithm relies on a dual-layer filter architecture to facilitate the NN weight tuning mechanism, while inserting an experience replay block in the update law based on captured information from IE condition. The proposed algorithm has a feature of inherent robustness, i.e., it does not require any robust modification like the traditional neuro-adaptive controllers. The IE-based experience replay suffices to prevent parameter drift by ensuring uniform ultimate boundedness (UUB) of the closed-loop concatenated error (tracking error+ NN weight tuning error) dynamics. A Barrier Lyapunov Function (BLF) is introduced in the adaptive law, which facilitates compact set pre-conditioning for the applicability of Universal Approximation property of the NN structure. Moreover, the ultimate-bound can be arbitrarily reduced by selecting the design parameters appropriately. Simulation results on wing-rock dynamics validate the efficacy of the proposed method as compared to status quo.

Adaptive Observer Design for Heat PDEs with Discrete and Distributed Delays and Parameter Uncertainties

The problem of observer design is addressed for heat partial differential equations (PDEs) that are subject to parameter uncertainties and time delays. In addition to parameter uncertainty, the novelty also lies in the presence of time-delay in the domain. Both discrete and distributed delays are accounted for and dealt with using a unified integral representation. We design an adaptive observer that provides online estimates of the states and parameters. The observer is designed using the backstepping design method, involving a Volterra transformation, and a state-parameter decoupling transformation. The observer stability analysis is dealt with using a Lyapunov-Krasovskii functional. It is shown, under a persistent excitation (PE) assumption, that the resulting estimation error system is exponentially stable for small delays.

Composite Learning Adaptive Interaction Control for High-DoF Collaborative Robots*

Variable impedance control is essential for improving safety and naturalness during physical human-robot interaction. However, the robot target impedance is usually difficult to be realized for existing variable impedance control methods. This paper proposes a composite learning adaptive interaction control strategy for robots to achieve target impedance under parametric uncertainty. A multi-mode adaptive control scheme is defined based on weighting functions to ensure smooth mode transitions. A composite learning technique is applied for exact robot modeling online such that target impedance can be achieved under interval excitation that is much weaker than persistent excitation. The proposed method can achieve variable stiffness and damping with guaranteed stable and safe interaction. Experiments on a collaborative robot with 7 degrees of freedom named Franka Emika Panda have validated the effectiveness and superiority of the proposed method.

Distributed Parameter Estimation under Gaussian Observation Noises

In this paper, we consider the problem of distributed parameter estimation in sensor networks. Each sensor makes successive observations of an unknown d-dimensional parameter, which might be subject to Gaussian random noises. They aim to infer true value of the unknown parameter by cooperating with each other. To this end, we first generalize the so-called dynamic regressor extension and mixing (DREM) algorithm to stochastic systems, with which the problem of estimating a d-dimensional vector parameter is transformed to that of d scalar ones: one for each of the unknown parameters. For each of the scalar problem, an estimation scheme is given, where each sensor fuses the regressors and measurements in its in-neighborhood and updates its local estimate by using least-mean squares. Particularly, a counter is also introduced for each sensor, which prevents any (noisy) measurement from being repeatedly used such that the estimation performance will not be greatly affected by certain extreme values. A novel excitation condition termed as local persistent excitation (Local-PE) condition is also proposed, which relaxes the traditional persistent excitation (PE) condition and only requires that the collective signals in each sensor's in-neighborhood are sufficiently excited. With the Local-PE condition and proper step sizes, we show that the proposed estimator guarantee that each sensor infers the true parameter in mean square, even if any individual of them cannot. Numerical examples are finally provided to illustrate the established results.

Boundary adaptive observer design for heat PDEs with spatially varying parameters

We are considering the problem of observer design for parabolic partial differential equations (PDEs) that are subject to parameter uncertainty. The novelty lies in the fact that the unknown parameters are allowed to be spatially varying and the associated regressor is subject to a mild assumption. Then, the initial observer design problem, involving a scalar PDE with spatially-varying unknown parameters, is shown to be transformable into a new problem involving coupled PDEs with spatially-invariant parameters and a bounded modelling error. Invoking the high-gain principle and the so-called decoupling transformation technique, an adaptive observer is designed that provides online estimates of the state and parameter vectors. The original spatially varying parameters are recovered using least-squares estimators. All state and parameter estimation errors are shown, under appropriate persistent-excitation (PE) conditions, to be exponentially convergent to balls cantered on the origin with radiuses depending on the adaptive observer gain: the higher the observer gain the better the accuracy of estimates.

An I & I Adaptive Redesign Approach for Asymptotic Stability without PE

This paper proposes an adaptation redesign approach of the immersion and invariance (I&I) adaptive control to achieve asymptotic stability of the controlled plant and the parameter estimator at the desired equilibrium. The key idea is to employ the technique of generalized parameter estimation-based observer on the parameter estimation error dynamics by applying the indirect I & I adaptive control scheme, yielding a linear regression equation, from which the adaptive law can be redesigned. As a result, it is shown that globally exponential parameter convergence can be guaranteed under an interval excitation (IE) condition, which is much weaker than the conventionally required persistent excitation. Under a stabilizability assumption, an adaptation-redesigned feedback control law can be designed to achieve global asymptotic stability at the desired equilibrium point under the IE condition. The proposed adaptive control approach is applied to a class of parameteric strict-feedback systems without overparameterization, which is needed by the standard I & I adaptive control.