Persistent Excitation Condition Research Articles

Neural network-based adaptive critic control for saturated nonlinear systems with full state constraints via a novel event-triggered mechanism

This paper investigates the issue of event-triggered optimal control for saturated nonlinear systems with full state constraints. A smooth function is defined to map the constrained states, then the considered system can be transformed into a state unconstrained system with unknown approximate errors. In addition, a novel event-triggered mechanism is proposed via an adaptive dynamic programming algorithm, which can obtain the optimal control law without constructing the so-called event-triggered HJB equation. Moreover, in order to conquer the difficulty caused by the persistence of excitation conditions, the experience replay technique is utilized to design the critic update law. Meanwhile, it is demonstrated that all signals are uniformly ultimately bounded. Finally, two simulation examples are given to demonstrate the effectiveness of the control strategy.

Self-error learning framework-based algorithm for parameter recovery of extended Wiener–Hammerstein systems subject to quantised measurements

In this study, a novel estimation scheme is proposed for identifying extended Wiener–Hammerstein systems with hysteresis nonlinearity subject to quantised measurements. The proposed scheme is established in a self-error learning framework to achieve high-performance parameter estimation compared with classic error feedback learning estimation algorithms. Initially, the useful identification data can be extracted from contaminated system data by introducing an adaptive filter. Then, with the help of the filtered data, the identification error expression used to establish the estimator is derived. Subsequently, an online compensation estimation error variable is proposed to eliminate the effect of the regression vector on the convergence performance. A new adaptive law is designed with adaptive recursive gain, considering the compensation estimation error data and parameter initial error data. Under general persistent excitation (PE) condition, the PE condition of the regressor information is verified online, and the estimator convergence is strictly proven. Finally, the statistical results of two illustrated examples and a real-word example are provided to validate the positive features and effectiveness of the proposed estimation scheme.

Concurrent Learning Robust Adaptive Fault Tolerant Boundary Regulation of Hyperbolic Distributed Parameter Systems.

This article develops a robust adaptive boundary output regulation approach for a class of complex anticollocated hyperbolic partial differential equations subjected to multiplicative unknown faults in both the boundary sensor and actuator. The regulator design is based on the internal model principle, which amounts to stabilize a coupled cascade system, which consists of a finite-dimensional internal model driven by a hyperbolic distributed parameter system (DPS). To this end, a systematic sliding mode equipped with a backstepping approach is developed such that the robust state feedback control can be realized. Moreover, since the available information is a faulty boundary measurement at the right side point, state estimation is required. However, due to the presence of boundary unknown faults, we need to solve an issue of joint fault-state estimation. Restrictive persistent excitation conditions are usually required to guarantee the exact estimation of faults but are unrealistic in practice. To this end, a novel concurrent learning (CL) adaptive observer is proposed so that exponential convergence is obtained. It is the first time that the spirit of CL is introduced to the field of DPSs. Consequently, the observer-based adaptive boundary fault tolerant control scheme is developed, and rigorous theoretical analysis is given such that the exponential output regulation can be achieved. Finally, the effectiveness of the proposed methodology is demonstrated via comparative simulations.

Adaptive optimized backstepping tracking control for full‐state constrained nonlinear strict‐feedback systems without using barrier Lyapunov function method

AbstractIn this article, the problem of adaptive optimal tracking control is studied for nonlinear strict‐feedback systems. While not directly measurable, the states of these systems are subject to both time‐varying and asymmetric constraints. Bypassing the conventional barrier Lyapunov function method, the constrained system is transformed into its unconstrained counterpart, thereby obviating the need for feasibility conditions. A specially designed reinforcement learning (RL) algorithm, featuring an observer‐critic‐actor architecture, is deployed in an adaptive optimal control scheme to ensure the stabilization of the converted unconstrained system. Within this architecture, the observer estimates the unmeasurable system states, the critic evaluates the control performance, and the actor executes the control actions. Furthermore, enhancements to the RL algorithm lead to relaxed conditions of persistent excitation, and the design methodology for the observer overcomes the restrictions imposed by the Hurwitz equation. The Lyapunov stability theorem is applied for two primary purposes: to ascertain the boundedness of all signals within the closed‐loop system, and to ensure the accuracy of the output signal in tracking the desired reference trajectory. Finally, numerical and practical simulations are provided to corroborate the effectiveness of the proposed control strategy.

Concurrent Learning-Based Two-Stage Predefined-Time System Identification

This paper proposes a novel two-stage predefined-time system identification algorithm for uncertain nonlinear systems based on concurrent learning. The main feature of the algorithm is that the convergence time of estimation error is an exact predefined parameter, which can be known and adjusted directly by users. Historic identification data are stored in the first stage to guarantee that a finite-rank condition is satisfied. In the second stage, the estimation error converges to zero for linearly parameterized uncertain systems, or it is regulated into the neighborhood of zero for unknown systems modeled by neural networks. The identification algorithm takes effect without the restrictive requirement of the persistent excitation condition. Simulation examples verify the effectiveness of the proposed method.

Model-driven full system dynamics estimation of PMSM-driven chain shell magazine

Based on the system dynamic model, a full system dynamics estimation method is proposed for a chain shell magazine driven by a permanent magnet synchronous motor (PMSM). An adaptive extended state observer (AESO) is proposed to estimate the unmeasured states and disturbance, in which the model parameters are adjusted in real time. Theoretical analysis shows that the estimation errors of the disturbances and unmeasured states converge exponentially to zero, and the parameter estimation error can be obtained from the extended state. Then, based on the extended state of the AESO, a novel parameter estimation law is designed. Due to the convergence of AESO, the novel parameter estimation law is insensitive to controllers and excitation signal. Under persistent excitation (PE) condition, the estimated parameters will converge to a compact set around the actual parameter value. Without PE signal, the estimated parameters will converge to zero for the extended state. Simulation and experimental results show that the proposed method can accurately estimate the unmeasured states and disturbance of the chain shell magazine, and the estimated parameters will converge to the actual value without strictly continuous PE signals.

Sliding-mode surface-based adaptive optimal nonzero-sum games for saturated nonlinear multi-player systems with identifier-critic networks

This paper addresses the sliding-mode surface-based adaptive optimal nonzero-sum games control problem for continuous-time nonlinear systems with input constraints. By constructing a value function associated with the sliding-mode surface, the original nonzero-sum games problem can be transformed into constructing multiple optimal control policies. Then, an adaptive dynamic programming method under identifier-critic networks is introduced to deal with the Hamilton–Jacobi equation, where the identifier networks are applied to approximate the unknown dynamics. A remarkable feature is that the critic updating laws are designed through experience replay and gradient descent methodologies, such that the persistence of excitation condition can be excluded. Based on the Lyapunov stability theory, all signals of the close-loop system are verified to be uniformly ultimately bounded. Finally, simulation results are given to illustrate the effectiveness of the proposed optimal nonzero-sum games scheme.

Adaptive observer and control of spatiotemporal delayed neural fields

An adaptive observer is proposed to estimate the synaptic distribution between neurons asymptotically from the measurement of a part of the neuronal activity and a delayed neural field evolution model. The convergence of the observer is proved under a persistency of excitation condition. Then, the observer is used to derive a feedback law ensuring asymptotic stabilization of the neural fields. Finally, the feedback law is modified to ensure simultaneously practical stabilization of the neural fields and asymptotic convergence of the observer under additional restrictions on the system. Numerical simulations confirm the relevance of the approach.

Reactionless control of free‐floating space manipulators with parameter uncertainty and input disturbance

AbstractThis paper studies the control problem of free‐floating space manipulators, and a control scheme is proposed to solve reactionless control of the end‐effector pose tracking with parameter uncertainty and input disturbance. First, based on dual modeling to treat the end‐effector as a virtual base spacecraft, the dynamics with uncertainties are established which map the joints' torque to the end‐effector pose and base spacecraft attitude, while the inverse kinematics and the derivative of the generalized Jacobi matrix can be avoided in controller design. Then, the reference acceleration stabilization schemes satisfying prescribed performance constraints are carefully designed for tracking errors, and based on these schemes the steady‐state and transient performance of the tracking control can be guaranteed. Further, the radial basis function neural network is adopted to estimate modeling errors caused by parameter uncertainty and input disturbance. In addition, a concurrent learning method is introduced in the network weights matrix update law, which allows the estimation errors to converge a neighborhood of zeros without the need for satisfying the persistent excitation condition. The simulation results verify the effectiveness of the proposed control scheme.

A general update rule for Lyapunov‐based adaptive control of mobile robots with wheel slip

SummaryIn this article, we introduce a novel family of Lyapunov‐based adaptive kinematic control laws developed to solve the trajectory tracking problem for a differential‐drive mobile robot under the influence of both longitudinal and lateral wheel slip. Each adaptive controller in this family is constructed by augmenting a nonadaptive nominal controller, originally designed for the slip‐free case, with an update rule capable of estimating the longitudinal slip. In the absence of lateral slip and under constant longitudinal slip, we establish the convergence of the trajectory tracking error to zero and, assuming a persistent excitation condition, we also demonstrate the convergence of the slip estimate error to zero. When lateral slip is present, we analyse a particular control law from our family of adaptive controllers. This law ensures the trajectory tracking error is uniformly ultimately bounded around the origin, demonstrating the robustness of our adaptive control scheme in dealing with time‐varying longitudinal and lateral slip. The validity of our approach is assessed through comprehensive numerical simulations.

Cooperative Finitely Excited Learning for Dynamical Games.

In this article, we propose a way to enhance the learning framework for zero-sum games with dynamics evolving in continuous time. In contrast to the conventional centralized actor-critic learning, a novel cooperative finitely excited learning approach is developed to combine the online recorded data with instantaneous data for efficiency. By using an experience replay technique for each agent and distributed interaction amongst agents, we are able to replace the classical persistent excitation condition with an easy-to-check cooperative excitation condition. This approach also guarantees the consensus of the distributed actor-critic learning on the solution to the Hamilton-Jacobi-Isaacs (HJI) equation. It is shown that both the closed-loop stability of the equilibrium point and convergence to the Nash equilibrium can be guaranteed. Simulation results demonstrate the efficacy of this approach compared to previous methods.

Dynamic event-triggered robust optimal tracking control for multi-player nonzero-sum games with mismatched uncertainties and asymmetric constrained inputs

In this paper, a novel dynamic event-triggered robust optimal tracking control method is proposed for multi-player nonzero-sum game problem with mismatched uncertainties and asymmetric constrained inputs. First, the original asymmetric constrained-input multi-player system is transformed into a new multi-player system with symmetric constrained inputs implemented by the state-space transformation technique, and an augmented system is established based on the new system and the desired trajectory. Next, an auxiliary augmentation system is constructed to handle the robust tracking control problem. Then, a critic neural network with a novel modified weight updating law is established for each player to solve the event-triggered coupled Hamilton–Jacobi equations, which relaxes the restrictions of initial admissible control and persistence of excitation condition. Furthermore, efficient communication and computing are obtained by the newly established dynamic event-triggered mechanism, and the Zeno behavior can be excluded by introducing an exponential term. The uniformly ultimately bounded properties of all signals are proved. Finally, two numerical examples verify the effectiveness of the designed method.

Identification of Parameters of Conservative Multispecies Lotka-Volterra System Based on Sampled Data

In this paper, we propose a method for identifying the parameters of the multispecies Lotka-Volterra model based on sampled data. The conditions (nonhyperplanarity, recurrence) under which the “Stripe” algorithm achieves the identification goals are found. The relation between nonhyperplanar recurrent trajectories and the persistent excitation condition for systems of general form is studied.

Robust optimal control of the multi‐input systems with unknown disturbance based on adaptive integral reinforcement learning Q‐function

AbstractConsidering overshoot and chatter caused by the unknown interference, this article studies the adaptive robust optimal controls of continuous‐time (CT) multi‐input systems with an approximate dynamic programming (ADP) based Q‐function scheme. An adaptive integral reinforcement learning (IRL) scheme is proposed to study the optimal solutions of Q‐functions. First, multi‐input value functions are presented, and Nash equilibrium is analyzed. A complex Hamilton–Jacobi–Issacs (HJI) equation is constructed with the multi‐input system and the zero‐sum‐game‐based value function. It is a challenging task to solve the HJI equation for nonlinear system. Thus, A transformation of the HJI equation is constructed as a Q‐function. The neural network (NN) is applied to learn the solution of the transformed Q‐functions based on the adaptive IRL scheme. Moreover, an error information is added to the Q‐function for the issue of insufficient initial incentives to relax the persistent excitation (PE) condition. Simultaneously, an IRL signal of the critic networks is introduced to study the saddle‐point intractable solution, such that the system drift and NN derivatives in the HJI equation are relaxed. The convergence of weight parameters is proved, and the closed‐loop stability of the multi‐system with the proposed IRL Q‐function scheme is analyzed. Finally, a two‐engine driven F‐16 aircraft plant and a nonlinear system are presented to verify the effectiveness of the proposed adaptive IRL Q‐function scheme.

Human-in-the-Loop Behavior Modeling via an Integral Concurrent Adaptive Inverse Reinforcement Learning.

One goal of artificial intelligence (AI) research is to teach machines how to learn from humans, such that they can perform a certain task in a natural human-like way. In this article, an online adaptive inverse reinforcement learning (IRL) approach to human behavior modeling is proposed to enhance machine intelligence for a class of linear human-in-the-loop (HiTL) systems using the state data only, where the human behavior is described by a linear quadratic optimal control model with an unknown weighting matrix for the quadratic cost function. First, an integral concurrent adaptive law is developed to learn the human feedback gain matrix online using the demonstrated state data only, which removes the persistent excitation (PE) conditions required by traditional adaptive estimation approaches and thus is more in line with real applications. Then, with the learned feedback gain matrix, the IRL problem is formulated as a linear matrix inequality (LMI) optimization problem, which can be efficiently solved to retrieve the weighting matrix of the human cost function. Finally, a simulation example is provided to illustrate the effectiveness of the proposed approach.

Adaptive Inverse Nonlinear Optimal Control Based on Finite-Time Concurrent Learning and Semidefinite Programming.

This article investigates the problem of inverse optimal control (IOC) for a class of nonlinear affine systems. An adaptive IOC approach is proposed to recover the cost functional using only the system state data, which integrates the finite-time concurrent learning (FTCL) technique and the semidefinite programming (SDP) technique. First, an identifier neural network (NN) is employed to approximate the unknown nonlinear control policy, and an FTCL-based update law is proposed to estimate the weights of the identifier NN online, which removes the traditional persistent excitation (PE) condition. Moreover, the finite-time convergence as well as the uniformly ultimately boundness (UUB) of estimation error of the identifier NN weights are analysed according to whether or not there exists the identifier NN approximation error. Then, with the help of a value NN for approximating the value function, an SDP problem with a quadratic objective function can be set up for determining the weighting matrices of the cost functional. Finally, simulation results are presented to validate the proposed method.

A New Neural Dynamic Learning Framework for Discrete-Time Strict-Feedback Systems: Internal Interaction-Based Weight Adaptive Laws.

This article investigates internal interaction-based dynamic learning control (LC) for uncertain discrete-time strict-feedback systems. On the basis of predict technology, the original system is converted into a common n -step-ahead input-output predict model. The predict model causes every estimated neural weight to converge to n different constants using the existing control framework. To solve such a problem, the predict model is further decomposed into n one-step-ahead subsystems, which can be viewed as n independent agents. Subsequently, the distributed cooperative weight adaptive laws are designed by introducing an undirected and connected interconnection topology among subsystems. By constructing the variable relationship between the subsystems and the n -step-ahead predict model, a new internal weight interaction-based neural dynamic LC framework is proposed for the whole closed-loop system, in which estimated weights at different times share their weight knowledge. The proposed framework ensures the ultimately uniform boundedness of the closed-loop system and achieves the excellent control performance. By combining the consensus theory and a cooperative persistent excitation condition, every estimated weight along the neural input orbit is verified to exponentially converge to a close vicinity of a unique ideal constant, rather than n different constants. Consequently, the developed LC framework facilitates constant weights storage, saves the knowledge storage space, and improves the robustness of knowledge utilization. These characteristics are verified by simulation results.

Double-layer prescribed performance control of free-floating space manipulators after capturing unknown targets

This paper studies the parameter identification and prescribed performance control of the free-floating space manipulator after capturing unknown targets. A novel finite-time concurrent learning (FT-CL) parameter identification algorithm utilizing suitable historical data is proposed. This algorithm guarantees the finite-time convergence of the parameter estimation error without using the persistent excitation condition. Then, we apply the dual modeling approach and view the end-effector states as generalized coordinates of novel virtual dual dynamics. This technique bypasses solving the inverse kinematics and calculating the Jacobi matrix, laying the foundation for Jacobi-free pose control of the end-effector. Besides, stabilizers of prescribed performance reference acceleration are carefully designed in a double-layer form for regulating end-effector pose and base attitude. Finally, the Jacobi-free controllers satisfying the double-layer performance constraints are developed. The efficacy of the proposed parameter identification scheme and control strategy is validated through numerical simulations.

Convergence analysis of a synchronous gradient estimation scheme for time-varying parameter systems

For the problem of time-varying parameter estimation and its convergence, this paper proposes an invariant matrix to represent the changing laws of time-varying parameters, which suggests a novel expression to describe the parameters by employing finite terms. A joint recursive least squares algorithm is derived by minimizing the criterion function associated with the unknown invariant matrix. For a class of time-varying systems, a joint stochastic gradient algorithm with the computational efficiency is obtained by analogy to achieve the estimation of invariant matrix and system parameters. The convergence analysis is explored to demonstrate that the time-varying parameter estimation error is bounded under the persistent excitation conditions. The simulation results also validate the effectiveness of the proposed synchronous parameter estimation approach.

Optimized intelligent tracking control for a quadrotor unmanned aerial vehicle with actuator failures

This work investigates the implementation of tracking control for a quadrotor unmanned aerial vehicle exposed to disturbances and actuator failures. Cost reduction in control processes is not usually considered in conventional control methods. To overcome this limitation, this work proposes an optimized intelligent control scheme that utilizes adaptive dynamic programming. Firstly, a critic-only structure is employed to learn the nominal control strategies, including two critic neural network update laws, lifting the persistent excitation condition. Subsequently, two specifically designed observers are developed on the radial basis function neural network (RBFNN) to reconstruct the disturbances and actuator failures and compensate for the nominal strategies. The observers achieve fixed-time convergence by employing the fixed-time update laws of the RBFNNs. Moreover, two variable vectors using the Gaussian error function are introduced to reduce the impact caused by the observation errors. Additionally, the superior performance of the proposed control scheme is validated through numerical simulations.