Persistent Excitation Research Articles

Self-organizing model reference adaptive control for aircraft with enhanced persistent excitation

Model Reference Adaptive Control (MRAC) methods are commonly applied in aerospace control systems, yet they face challenges in terms of persistent excitation, adaptation speed, and network structure design. To tackle these issues, this study introduces a novel MRAC method that employs practical techniques, including fast adaptation, experience replay, and network self-organization. Specifically, this study emphasizes three key contributions. First, it introduces a modified MRAC architecture that incorporates a fast adaptation term and a network approximation term to compensate for control errors more effectively. The fast adaptation term, which exhibits higher adaptive speed, enables quick response to dynamic variations and is stored in an experience replay buffer. Additionally, the network term is trained using batches of past samples extracted from the experience replay buffer, rather than relying solely on the current sample. This approach successfully overcomes challenges related to achieving persistent excitation and setting adaptation speed. Secondly, this paper introduces an online method for network self-organization to approximate a global model. This approach autonomously selects basis functions to enhance the organization of the network, leading to improved matching and overall performance. Finally, theoretical proofs and numerical simulations are presented to validate the effectiveness of the proposed techniques and demonstrate the advantages of the method in terms of parameter convergence, learning speed, and control stability.

Robust optimal control of the multi‐input systems with unknown disturbance based on adaptive integral reinforcement learning Q‐function

AbstractConsidering overshoot and chatter caused by the unknown interference, this article studies the adaptive robust optimal controls of continuous‐time (CT) multi‐input systems with an approximate dynamic programming (ADP) based Q‐function scheme. An adaptive integral reinforcement learning (IRL) scheme is proposed to study the optimal solutions of Q‐functions. First, multi‐input value functions are presented, and Nash equilibrium is analyzed. A complex Hamilton–Jacobi–Issacs (HJI) equation is constructed with the multi‐input system and the zero‐sum‐game‐based value function. It is a challenging task to solve the HJI equation for nonlinear system. Thus, A transformation of the HJI equation is constructed as a Q‐function. The neural network (NN) is applied to learn the solution of the transformed Q‐functions based on the adaptive IRL scheme. Moreover, an error information is added to the Q‐function for the issue of insufficient initial incentives to relax the persistent excitation (PE) condition. Simultaneously, an IRL signal of the critic networks is introduced to study the saddle‐point intractable solution, such that the system drift and NN derivatives in the HJI equation are relaxed. The convergence of weight parameters is proved, and the closed‐loop stability of the multi‐system with the proposed IRL Q‐function scheme is analyzed. Finally, a two‐engine driven F‐16 aircraft plant and a nonlinear system are presented to verify the effectiveness of the proposed adaptive IRL Q‐function scheme.

On the Exact Parameter Estimation for Robot Manipulators Without Persistence of Excitation

Adaptive control is one of the most employed techniques to achieve trajectory tracking of robot manipulators. Although it is desirable to obtain exact parameter estimation, most adaptive schemes need the persistency of excitation (PE) condition on the regressor to be satisfied. In the recent years, the so called Dynamic Regressor Extension and Mixing (DREM) procedure was developed to provide an alternative in the design of adaptive laws with conditions different from PE. When met, the improvement in parameter estimation is remarkable, but when not, adaptation can simply stop, which might be unacceptable for control purposes. This paper proposes for the first time a composite scheme which combines the standard gradient adaptive law with a DREM based additional term with the following properties: 1) Trajectory tracking for joint desired positions and velocities is guaranteed; 2) In the absence of PE, if a new condition that can partially be verified online for the additional DREM based term is matched, then exact parameter estimation takes place in finite time. Simulation results are in good accordance with the developed theory.

Optimal Adaptive Tracking Control of Partially Uncertain Nonlinear Discrete-Time Systems Using Lifelong Hybrid Learning.

This article addresses a multilayer neural network (MNN)-based optimal adaptive tracking of partially uncertain nonlinear discrete-time (DT) systems in affine form. By employing an actor-critic neural network (NN) to approximate the value function and optimal control policy, the critic NN is updated via a novel hybrid learning scheme, where its weights are adjusted once at a sampling instant and also in a finite iterative manner within the instants to enhance the convergence rate. Moreover, to deal with the persistency of excitation (PE) condition, a replay buffer is incorporated into the critic update law through concurrent learning. To address the vanishing gradient issue, the actor and critic MNN weights are tuned using control input and temporal difference errors (TDEs), respectively. In addition, a weight consolidation scheme is incorporated into the critic MNN update law to attain lifelong learning and overcome catastrophic forgetting, thus lowering the cumulative cost. The tracking error, and the actor and critic weight estimation errors are shown to be bounded using the Lyapunov analysis. Simulation results using the proposed approach on a two-link robot manipulator show a significant reduction in tracking error by 44% and cumulative cost by 31% in a multitask environment.

Continuous-Time Reinforcement Learning: New Design Algorithms With Theoretical Insights and Performance Guarantees.

Continuous-time reinforcement learning (CT-RL) methods hold great promise in real-world applications. Adaptive dynamic programming (ADP)-based CT-RL algorithms, especially their theoretical developments, have achieved great successes. However, these methods have not been demonstrated for solving realistic or meaningful learning control problems. Thus, the goal of this work is to introduce a suite of new excitable integral reinforcement learning (EIRL) algorithms for control of CT affine nonlinear systems. This work develops a new excitation framework to improve persistence of excitation (PE) and numerical performance via input/output insights from classical control. Furthermore, when the system dynamics afford a physically-motivated partition into distinct dynamical loops, the proposed methods break the control problem into smaller subproblems, resulting in reduced complexity. By leveraging the known affine nonlinear dynamics, the methods achieve well-behaved system responses and considerable data efficiency. The work provides convergence, solution optimality, and closed-loop stability guarantees of the proposed methods, and it demonstrates these guarantees on a significant application problem of controlling an unstable, nonminimum phase hypersonic vehicle (HSV).

Adaptive Regulation With Global KL Guarantees

In the absence of persistency of excitation (PE), referring to adaptive control systems as “uniformly asymptotically stable” typically indicates insufficient understanding of stability concepts. While the state is indeed regulated to zero and the parameter estimate has some limit, namely, the overall state converges to some equilibrium, the equilibrium reached is not unique (and not even necessarily stable) but is dependent on the initial condition. The equilibrium set in the absence of PE is not uniformly attractive (from an open set containing the equilibrium set); hence, uniform asymptotic stability does not hold and KL-estimates are unavailable for the full state. In this paper we pursue adaptive control design with KL guarantees on the regulated state, something that is possible but previously unachieved with smooth, time-invariant and non-hybrid adaptive controllers. This property is referred to as Uniform Global Asymptotic Output Stability, where the regulated state is thought of as a system output. We provide designs for (i) systems with a matched uncertainty and (ii) systems in the parametric strict feedback form. We assume no a priori known bound for the parameters. To guarantee KL-estimates in the absence of PE, our designs employ time-invariant nonlinear damping terms, which depend both on the state and the parameter estimate. With an example, we illustrate the theory.

Strong Lyapunov Functions for Linear Time-Varying Systems Under Persistency of Excitation

Strong Lyapunov Functions for Linear Time-Varying Systems Under Persistency of Excitation

Composite Learning Based Adaptive Control of Linear 2×2 Hyperbolic PDE Systems.

This article considers the adaptive stability control of a class of 2×2 linear hyperbolic PDE systems. The PDE model is subject to constant but in-domain and boundary unknown parameters. A novel adaptive controller is developed by leveraging the swapping design technique and composite parameter learning law. With swapping design, several linear and static combinations, including carefully designed filters, unknown parameters, and error terms, are constructed to express the system states. From the static combinations, a composite learning based forgetting-factor least squares law is introduced to guarantee exponential parameter convergence without the persistent excitation (PE). Although inaccurate parameter estimation in the adaptive backstepping control results in asymptotic stability of the system, accurate parameter estimation ensures the exponential convergence of closed-loop system and concomitantly improves the transient performance. Finally, a comparative numerical simulation is performed to validate the effectiveness and advantage of the developed adaptive control scheme.

Similar Formation Control via Range and Odometry Measurements.

This article investigates the similar formation control problem for multirobot systems. Specifically, we propose an integrated relative localization and similar formation control scheme to navigate multirobot systems to a desired configuration, which is a similar transformation of a given template, based on interrobot and robot-landmark range measurements and odometry measurements of robots themselves. To achieve the exact relative localization, a persistent excitation (P.E.) signal is introduced in the controller which, however, perturbs the motion of each robot and affects the formation accuracy. To resolve the conflict, an autonomous system with its output regulated by a carefully designed function of range measurements is introduced to generate the persistent excitation. It is proved that the similar formation control problem can be solved by our proposed scheme with global asymptotic convergence for directed acyclic graphs (DAGs). Both numerical simulation and physical experiment are presented to verify and validate the effectiveness of our theoretical findings.

A New Neural Dynamic Learning Framework for Discrete-Time Strict-Feedback Systems: Internal Interaction-Based Weight Adaptive Laws.

This article investigates internal interaction-based dynamic learning control (LC) for uncertain discrete-time strict-feedback systems. On the basis of predict technology, the original system is converted into a common n -step-ahead input-output predict model. The predict model causes every estimated neural weight to converge to n different constants using the existing control framework. To solve such a problem, the predict model is further decomposed into n one-step-ahead subsystems, which can be viewed as n independent agents. Subsequently, the distributed cooperative weight adaptive laws are designed by introducing an undirected and connected interconnection topology among subsystems. By constructing the variable relationship between the subsystems and the n -step-ahead predict model, a new internal weight interaction-based neural dynamic LC framework is proposed for the whole closed-loop system, in which estimated weights at different times share their weight knowledge. The proposed framework ensures the ultimately uniform boundedness of the closed-loop system and achieves the excellent control performance. By combining the consensus theory and a cooperative persistent excitation condition, every estimated weight along the neural input orbit is verified to exponentially converge to a close vicinity of a unique ideal constant, rather than n different constants. Consequently, the developed LC framework facilitates constant weights storage, saves the knowledge storage space, and improves the robustness of knowledge utilization. These characteristics are verified by simulation results.

Recursive Identification of Binary-Valued Systems Under Uniform Persistent Excitations

Recursive Identification of Binary-Valued Systems Under Uniform Persistent Excitations

Regularized Iterative Weighted Total Least Squares for Vehicle Mass Estimation

This work addresses the challenge of vehicle mass estimation using a longitudinal vehicle dynamics model, which adopts an errors-in-variables formulation due to the presence of noise-contaminated measurements in both input and output variables. The reduced vehicle dynamics model is ill-conditioned by nature of correlated input variables and a lack of persistent excitation in the measured data. A regularized iterative weighted total least squares (RIWTLS) method is therefore developed and has the advantage of producing parameter uncertainty quantification and measurement bias estimation alongside the estimated system parameters. A complementary adaptive regularization scheme is developed and serves to control the numerical stability of the RIWTLS algorithm based on the conditioning of incoming data. Experimental tests using electric vehicle data and a batch estimation scheme highlight the performance of the proposed RIWTLS algorithm, estimating vehicle mass to within ±1% accuracy.

Dynamic Event-Driven Finite-Horizon Optimal Consensus Control for Constrained Multiagent Systems.

This article investigates the event-driven finite-horizon optimal consensus control problem for multiagent systems with symmetric or asymmetric input constraints. Initially, in order to overcome the difficulty that the Hamilton-Jacobi-Bellman equation is time-varying in finite-horizon optimal control, a single critic neural network (NN) with time-varying activation function is applied to obtain the approximate optimal control. Meanwhile, for minimizing the terminal error to satisfy the terminal constraint of the value function, an augmented error vector containing the Bellman residual and the terminal error is constructed to update the weight of the NN. Furthermore, an improved learning law is proposed, which relaxes the tricky persistence excitation condition and eliminates the requirement of initial stability control. Moreover, a specific algorithm is designed to update the historical dataset, which can effectively accelerate the convergence rate of network weight. In addition, to improve the utilization rate of the communication resource, an effective dynamic event-triggering mechanism (DETM) composed of dynamic threshold parameters (DTPs) and auxiliary dynamic variables (ADVs) is designed, which is more flexible compared with the ADV-based DETM or DTP-based DETM. Finally, to support the effectiveness of the proposed method and the superiority of the designed DETM, a simulation example is provided.

Finite-dimensional adaptive observer design for linear parabolic systems

A new finite-dimensional adaptive observer is proposed for some linear parabolic systems. The observer is based on the modal decomposition approach and uses a classical persistent excitation condition to ensure practical exponential convergence of both states and parameters estimation.

Double-layer prescribed performance control of free-floating space manipulators after capturing unknown targets

This paper studies the parameter identification and prescribed performance control of the free-floating space manipulator after capturing unknown targets. A novel finite-time concurrent learning (FT-CL) parameter identification algorithm utilizing suitable historical data is proposed. This algorithm guarantees the finite-time convergence of the parameter estimation error without using the persistent excitation condition. Then, we apply the dual modeling approach and view the end-effector states as generalized coordinates of novel virtual dual dynamics. This technique bypasses solving the inverse kinematics and calculating the Jacobi matrix, laying the foundation for Jacobi-free pose control of the end-effector. Besides, stabilizers of prescribed performance reference acceleration are carefully designed in a double-layer form for regulating end-effector pose and base attitude. Finally, the Jacobi-free controllers satisfying the double-layer performance constraints are developed. The efficacy of the proposed parameter identification scheme and control strategy is validated through numerical simulations.

Optimized intelligent tracking control for a quadrotor unmanned aerial vehicle with actuator failures

This work investigates the implementation of tracking control for a quadrotor unmanned aerial vehicle exposed to disturbances and actuator failures. Cost reduction in control processes is not usually considered in conventional control methods. To overcome this limitation, this work proposes an optimized intelligent control scheme that utilizes adaptive dynamic programming. Firstly, a critic-only structure is employed to learn the nominal control strategies, including two critic neural network update laws, lifting the persistent excitation condition. Subsequently, two specifically designed observers are developed on the radial basis function neural network (RBFNN) to reconstruct the disturbances and actuator failures and compensate for the nominal strategies. The observers achieve fixed-time convergence by employing the fixed-time update laws of the RBFNNs. Moreover, two variable vectors using the Gaussian error function are introduced to reduce the impact caused by the observation errors. Additionally, the superior performance of the proposed control scheme is validated through numerical simulations.

Adaptive Control With Global Exponential Stability for Parameter-Varying Nonlinear Systems Under Unknown Control Gains.

It is nontrivial to achieve exponential stability even for time-invariant nonlinear systems with matched uncertainties and persistent excitation (PE) condition. In this article, without the need for PE condition, we address the problem of global exponential stabilization of strict-feedback systems with mismatched uncertainties and unknown yet time-varying control gains. The resultant control, embedded with time-varying feedback gains, is capable of ensuring global exponential stability of parametric-strict-feedback systems in the absence of persistence of excitation. By using the enhanced Nussbaum function, the previous results are extended to more general nonlinear systems where the sign and magnitude of the time-varying control gain are unknown. In particular, the argument of the Nussbaum function is guaranteed to be always positive with the aid of nonlinear damping design, which is critical to perform a straightforward technical analysis of the boundedness of the Nussbaum function. Finally, the global exponential stability of parameter-varying strict-feedback systems, the boundedness of the control input and the update rate, and the asymptotic constancy of the parameter estimate are established. Numerical simulations are carried out to verify the effectiveness and benefits of the proposed methods.

Convergence analysis of a synchronous gradient estimation scheme for time-varying parameter systems

For the problem of time-varying parameter estimation and its convergence, this paper proposes an invariant matrix to represent the changing laws of time-varying parameters, which suggests a novel expression to describe the parameters by employing finite terms. A joint recursive least squares algorithm is derived by minimizing the criterion function associated with the unknown invariant matrix. For a class of time-varying systems, a joint stochastic gradient algorithm with the computational efficiency is obtained by analogy to achieve the estimation of invariant matrix and system parameters. The convergence analysis is explored to demonstrate that the time-varying parameter estimation error is bounded under the persistent excitation conditions. The simulation results also validate the effectiveness of the proposed synchronous parameter estimation approach.

Distributed Adaptive Inverse Differential Game Approach to Leader's Behavior Learning for Multiple Autonomous Followers

In this study, the leader's behavior learning problem is investigated for a class of leader-follower systems, where the leader's behavior is assumed not a priori known to all followers that are autonomous. A multi-player nonzero-sum differential game is introduced to indicate the control interaction between agents, where the leader's behavior is modelled by an unknown cost function. The autonomous followers aim to retrieve the weighting matrix in the leader's cost function collaboratively. A distributed online adaptive inverse differential game (IDG) approach to the leader's behavior learning is proposed for the autonomous followers. Specifically, a concurrent learning (CL) based adaptive law and an interactive game controller are first developed for each autonomous follower to learn the leader's feedback gain matrix online, while the feedback Nash equilibrium of the game can be achieved. Then, a linear matrix inequality (LMI) optimization problem is formulated to determine the weighting matrix of the leader's cost function for each autonomous follower. The proposed method simply requires that all followers share their interactive feedback gain matrices, not their private intents, and use only the system state data, not both the system state data and the leader's control input data. The main advantages of the proposed method are that it can be implemented online without requiring the persistent excitation condition and needs less computational power for all followers. Finally, numerical simulations are provided to demonstrate the effectiveness and feasibility of the developed method.

Dynamic learning from adaptive neural control for full-state constrained strict-feedback nonlinear systems

This study focuses on the learning and control issues of strict-feedback systems with full-state constraints. To achieve learning capability under constraints, transformation mapping is utilized to convert the original system with full-state constraints into a quasi-pure-feedback unconstrained system. Utilizing the system transformation technique, only a single neural network (NN) is required to identify the unknown dynamics within the transformed system. Combining the dynamic surface control design, a novel adaptive neural control scheme is developed to ensure that all closed-loop signals are uniformly bounded, and every system state remains within the predefined constraint range. In addition, the precise convergence of NN weights is further transformed into an exponential stability problem for a category of linear time-varying systems under persistent excitation conditions. Subsequently, the converged NN weights are efficiently stored and utilized to create a learning controller to achieve better control performance while abiding by the full-state constraints. The viability of this control strategy is demonstrated via simulations.