Persistent Excitation Condition Research Articles

Event-triggered critic learning impedance control of lower limb exoskeleton robots in interactive environments

In this paper, we present an event-triggered critic learning impedance control algorithm for a lower limb rehabilitation exoskeleton robot in an interactive environment, where the control objective is specified by a desired impedance model. In comparison to many other traditional impedance controller design algorithms, in this paper, the impedance control problem is transformed into an optimal control problem. Firstly, the interactive environment accounts for the interaction between the exoskeleton, the human, and the environment, and is modeled by a linear time-invariant exogenous system. Secondly, in contrast to time-triggered control design mechanisms, the event-triggered controller is updated only when the system states deviate from prescribed threshold values. To obtain the event-triggered optimal controller, a critic neural network is developed through the framework of reinforcement learning. A modified gradient descent method is introduced to update the weights of the critic network with an additional stable term employed to eliminate the need for an initial admissible control. Meanwhile, with the simultaneous application of historical and transient state data to the critic neural network, the persistent excitation conditions are relaxed. The Lyapunov method is used to rigorously demonstrate the stability of the overall system. Finally, the effectiveness of the proposed algorithm is demonstrated via simulation.

Application of High Gain and Sliding Mode Observers for Estimating Key Variables in Bioprocesses

The monitoring and control of bioreactors is essential for any bioprocess. Therefore, measuring some essential biochemical key variables is crucial, such as the reaction rates, whose structure is usually barely known. Although many high gain and sliding mode observers-based estimators were designed to estimate these variables, these strategies are not robust against external disturbances. This work extends the high gain and sliding mode techniques to simultaneously estimate the reaction rates and an external disturbance under a persistent excitation condition that is easy to verify. Furthermore, the proposed algorithms are exponentially and finite-time stable, respectively. Finally, both observers are tested in a continuous fermentation process simulation.

Stable nullspace adaptive parameter identification of 6 degree-of-freedom plant and actuator models for underactuated vehicles: Theory and experimental evaluation

Model-based approaches to navigation, control, and fault detection that utilize precise nonlinear models of vehicle plant dynamics will enable more accurate control and navigation, assured autonomy, and more complex missions for such vehicles. This paper reports novel theoretical and experimental results addressing the problem of parameter estimation of plant and actuator models for underactuated underwater vehicles operating in 6 degrees-of-freedom (DOF) whose dynamics are modeled by finite-dimensional Newton-Euler equations. This paper reports the first theoretical approach and experimental validation to identify simultaneously plant-model parameters (parameters such as mass, added mass, hydrodynamic drag, and buoyancy) and control-actuator parameters (control-surface models and thruster models) in 6-DOF. Most previously reported studies on parameter identification assume that the control-actuator parameters are known a priori. Moreover, this paper reports the first proof of convergence of the parameter estimates to the true set of parameters for this class of vehicles under a persistence of excitation condition. The reported adaptive identification (AID) algorithm does not require instrumentation of 6-DOF vehicle acceleration, which is required by conventional approaches to parameter estimation such as least squares. Additionally, the reported AID algorithm is applicable under any arbitrary open-loop or closed-loop control law. We report simulation and experimental results for identifying the plant-model and control-actuator parameters for an L3 OceanServer Iver3 autonomous underwater vehicle. We believe this general approach to AID could be extended to apply to other classes of machines and other classes of marine, land, aerial, and space vehicles.

Learning from output‐feedback control of sampled‐data systems in normal form

AbstractThis paper investigates the learning and control problem of sampled‐data systems with only output measurements. A unified approach is presented by integrating the sampled‐data observer and deterministic learning. First, an adaptive radial basis function network (RBFN) learning controller with a sampled‐data observer is designed to track a recurrent reference model. Along the trajectory estimated by the observer, it is proven that the RBFN weights can exponentially converge to their ideal values with the satisfaction of a persistent excitation (PE) condition and the closed‐loop dynamics can be accurately learned during the output‐feedback process. Second, by using the learning results, a knowledge‐based output‐feedback controller is developed to improve the tracking performance. Further research shows that choosing appropriate parameters for the observer and RBFN can guarantee learning and control performance. The significance of the proposed approach is that the closed‐loop dynamics of the output‐feedback process can be accurately learned and further utilized to improve control performance. Simulation studies indicate the effectiveness and advantages of the learning control approach.

Adaptive critic learning for approximate optimal event-triggered tracking control of nonlinear systems with prescribed performances

This paper studies the approximate optimal event-triggered tracking control problem for a class of second-order nonlinear systems with prescribed performances. By employing a prescribed performances function, restrictions on tracking errors are removed for prescribed performances including convergence rate, maximum overshoot, and steady-state error. On this basis, a complex optimal tracking control problem can be transformed into an optimal stabilising control problem by constructing an augmented system. Subsequently, an event-triggered mechanism is introduced to reduce communication resources. Then, an adaptive critic learning algorithm is used to solve the Hamilton-Jacobi-Bellman equation, where the weights in the critic networks are tuned through the gradient descent approach and experience replay technology. Through the developed learning method, the persistence of excitation condition is released, and the data efficiency is improved. The Lyapunov stability theory is introduced to verify the stability of the close-loop system and the tracking errors are uniformly ultimately bounded. Finally, simulation results are presented to show the effectiveness of the proposed approach.

Adaptive Actuator Failure Compensation Control: A New Scheme Based on Fault Detection and Isolation

In flight control systems, multiple actuator stuck (or hardover) failures would cause saturation of the remaining healthy actuators and result in system instability. In this article, a new adaptive actuator failure compensation control scheme is proposed for a class of nonlinear systems with multiple redundant inputs. Under a mild assumption on the time interval between two successive failures, an online fault detection and isolation (FDI) algorithm is introduced, with which an actuator failure can be exactly identified in a given finite time without using the persistent excitation (PE) condition. The main advantage of the proposed scheme is that with the help of the FDI algorithm, an actuator with stuck failure (SF) can be deactivated before a new failure occurs and therefore, the drawback of the current adaptive failure compensation control approach that may cause more control effort or even saturation with multiple actuators being stuck can be overcome.

Deterministic Learning-Based Adaptive Neural Control for Nonlinear Full-State Constrained Systems.

In this article, an adaptive neural learning method is introduced for a category of nonlinear strict-feedback systems with time-varying full-state constraints. The two challenging problems of state constraints and learning capability are investigated and solved in a unified framework. To obtain the learning of unknown functions and satisfy full-state constraints, three main steps are considered. First, an adaptive dynamic surface controller (DSC) based on barrier Lyapunov functions (BLFs) is structured to implement that the closed-loop systems signals are bounded and full-state variables remain within the prescribed time-varying intervals. Moreover, the radial basis function neural networks (RBF NNs) are used to identify unknown functions. The output of the first-order filter, instead of virtual control derivatives, is used to simplify the complexity of the RBF NN input variables. Second, the state transformation is used to obtain a class of linear time-varying subsystems with small perturbations such that the recurrence of the RBF NN input variables and the partial persistent excitation condition are actualized. Therefore, the unknown functions can be accurately approximated, and the learned knowledge is kept as constant NN weights. Third, the obtained constant weights are borrowed into an adaptive learning scheme to achieve the batter control performance. Finally, simulation studies illustrate the advantage of the reported adaptive learning method on higher tracking accuracy, faster convergence rate, and lower computational expense by reusing learned knowledge.

Behavior learning based distributed tracking control for human-in-the-loop multi-agent systems

In this study, the distributed tracking problem for human-in-the-loop multi-agent systems (HiTL MASs) has been investigated. First, we construct an HiTL MAS model with a non-autonomous leader which can receive the control signal from a human operator and generate the desired trajectory. The human control signal is assumed to be generated by a leader’s state feedback control law with an unknown gain matrix that represents the control behavior of the human operator. Then, we propose a fully distributed adaptive control method that enables all followers to simultaneously track the human-controlled leader and online learn the unknown human operator’s feedback gain matrix. Furthermore, the parameter estimation error is also discussed, and all followers will learn the true value of the human operator’s feedback gain matrix when the state of the leader satisfies the persistent excitation (PE) condition. Moreover, a novel distributed adaptive control law is developed for each follower to remove the PE condition by utilizing the concurrent learning (CL) technique. Finally, simulated examples demonstrating the effectiveness of the proposed methodologies are presented.

Data-based trackability criteria and control design for disturbed learning systems

This paper aims to establish a data-based control design framework for linear iterative learning control (ILC) systems subject to disturbances. Through the proper selection of the test inputs, some output data are collected to exploit two trackability criteria of the desired reference for ILC systems such that the implementability of the perfect tracking objective is ensured, where the persistent excitation condition may not be imposed. Furthermore, two classes of data-based ILC updating laws are presented by incorporating the design idea of the observer, for which the collected input and output data are only leveraged. It is shown that given any trackable desired reference, the perfect tracking objective can be realized for ILC systems under the proposed data-based ILC updating laws in the absence of any model information.

Autonomous Game Control for Spacecraft Rendezvous via Adaptive Perception and Interaction

This article studies the autonomous game control problem for spacecraft rendezvous by using the adaptive perception and interaction. The rendezvous mission involves two spacecraft (named by autonomous spacecraft <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">a</i> , and the other spacecraft <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">o</i> , respectively), in which the control strategy of spacecraft <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">o</i> is determined by only itself and cannot be intervened, but the control law of spacecraft <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">a</i> needs to be designed to achieve the rendezvous under the condition that only the relative position and velocity of spacecraft in the rendezvous system are available. By formulating the rendezvous system as a two-player linear quadratic differential game (DG) with unknown intent of spacecraft <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">o</i> , an autonomous game control method is proposed for spacecraft <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">a</i> by using the adaptive perception and interaction. This game control approach removes the requirement for knowing the intent of spacecraft <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">o</i> , relaxes the persistent excitation condition in traditional adaptive estimation methods, and guarantees the asymptotic stability of the rendezvous system. Furthermore, the intent (i.e., the weighting matrix in the cost function) of spacecraft <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">o</i> is inferred by solving a residual minimization problem. Finally, a simulation study is carried out to verify the effectiveness of the developed autonomous game control strategy.

Adaptive neural control with fast approximation for uncertain nonlinear systems: A novel composite learning approach

AbstractIn existing adaptive neural control approaches, only when the regressor satisfies the persistent excitation (PE) or interval excitation (IE) conditions, the constant optimal weights of neural network (NN) can be identified, which can be used to establish uncertainties in nonlinear systems. This paper proposes a novel composite learning approach based on adaptive neural control. The focus of this approach is to make the NN approximate uncertainties in nonlinear systems quickly and accurately without identifying the constant optimal weights of the NN. Hence, the regressor does not need to satisfy the PE or IE conditions. In this paper, regressor filtering scheme is adopted to generate prediction error, and then the prediction error and tracking error simultaneously drive the update of NN weights. Under the framework of Lyapulov theory, the proposed composite learning approach can ensure that approximation error of the uncertainty and tracking error of the system states converge to an arbitrarily small neighborhood of zero exponentially. The simulation results verify the effectiveness and advantages of the proposed approach in terms of fast approximation.

Sliding-mode surface-based decentralized event-triggered control of partially unknown interconnected nonlinear systems via reinforcement learning

This paper studies a sliding-mode surface (SMS)-based decentralized event-triggered control problem for a class of partially unknown interconnected nonlinear systems within an identifier-critic neural networks framework. By assigning an especial cost function associated with SMS for each nominal subsystem, the original control issue is equivalently converted into designing a number of optimal controllers updating under an event-triggered manner, which considerably saves the communication resources. To derive these optimal event-based control policies, the corresponding Hamilton-Jacobi-Bellman equations are solved via the reinforcement learning algorithm. Different from traditional reinforcement learning algorithms, the identifier-critic network framework can obviate the requirement for the knowledge of system internal dynamics, and remove the errors generated by approximating actor networks. The weight vector in the critic network is updated through the gradient descent method and the experience replay technology, such that the persistence of excitation condition is relaxed. Under the proposed SMS-based decentralized control scheme, the considered system has a faster control response while minimizing the cost function. After that, according to the Lyapunov stability theory, all signals of the interconnected nonlinear systems are strictly proved to be bounded. Finally, the validity of the proposed control scheme is demonstrated via a simulation example.

Adaptive observer design for a class of delayed parabolic systems

A new adaptive observer is proposed for a class of uncertain time-delay parabolic systems. The uncertainties are located in both the state equation and the output equation which are also subjected to a distributed delay. Our algorithm is based on backstepping-like transformation and a classical persistent excitation condition. To illustrate the performances of this observer, simulation results are provided.

Composite adaptive online inverse optimal control approach to human behavior learning

There have been broad ranges of human-in-the-loop (HiTL) control systems, like fly-by-wire aircraft, share driving, and energy management. To develop advanced HiTL control systems with augmented intelligence, it is desirable that the machine is empowered to understand how a skilled human operator determines the control actions. It is common to model the human operator as an optimal controller with an unknown weighting matrix which depicts the tradeoff between multiple control objectives. Our aim is thus to learn the weighting matrix of the human objective function using only the system state measurement. Accordingly, for the HiTL system, a composite adaptive inverse optimal control (IOC) approach, which integrates concurrent learning (CL) based composite adaptive estimation and linear matrix inequality (LMI) optimization techniques, is proposed to learn human behavior online. The proposed composite adaptive IOC method consists of two steps: first, a composite adaptive update law is proposed to online estimate the human feedback gain matrix, which removes the persistent excitation condition required by traditional adaptive estimation methods, and then, an LMI optimization problem is solved to retrieve the weighting matrix of the human objective function with the learned feedback gain. The applicability and effectiveness of the proposed method are demonstrated with a steering control simulation.

Finite-time prescribed performance optimal attitude control for quadrotor UAV

In this paper, the optimal attitude control problem is investigated for quadrotor UAV with time-varying control effectiveness faults based on adaptive dynamic programming. Initially, a novel nonlinear conversion function is introduced to achieve the finite-time prescribed performance on tracking errors of three attitude angles of the quadrotor UAV. Furthermore, given that the controller may experience a degree of time-varying faults due to internal component aging or external environmental influences, an improved value function with a discount factor is proposed to resist the aforementioned time-varying control effectiveness faults. Then, by integrating the concurrent learning technique into adaptive dynamic programming algorithm, a valid approximate optimal control protocol is presented. Such an algorithm not only effectively improves the utilization of historical data in the history stack but also loosens the strict persistent excitation condition into a rank condition that is easy to check online. Moreover, it can be proved that the uniformly ultimately bounded stability of the attitude system is ensured even in the presence of time-varying control effectiveness faults. Meanwhile, tracking errors of attitude angles successfully converge to pre-defined symmetric or asymmetric precision intervals with an adjustable convergence rate in a user-defined finite time. Finally, a simulation example of the quadrotor UAV attitude system is given to verify the validity of the developed control scheme.

Event-triggered distributed robust optimal control of nonholonomic mobile agents with obstacle avoidance formation, input constraints and external disturbances

This paper proposes an event-triggered distributed robust optimal control algorithm for nonholonomic mobile agents (NMA) systems with collision avoidance, asymmetrically saturated inputs and external disturbances. By employing the limit cycle principle and developing the adaptive dynamic programming, zero-sum game theory, and an event-triggering mechanism, the algorithm can optimize an H∞ cost function and relax both the perfect velocity tracking assumption and the persistent excitation condition. First, the graph theory is used to establish the communication configuration, followed by the introduction of the distributed kinematic control for collision avoidance. Second, as a perfect velocity tracking assumption cannot be guaranteed under dynamics changes, input constraints and external disturbances, the distributed optimal robust control algorithm is designed. Third, by developing a new event-triggering mechanism, the burden of computational complexity and communications is reduced. The closed-loop stability and the zeno-behavior exclusion are proven by the Lyapunov theory. Finally, a numerical simulation is conducted with comparison to the sampling period–based control algorithm to demonstrate the effectiveness of the proposed algorithm.

Fault diagnosis for a class of nonlinear uncertain systems using deterministic learning approach

In this paper, we propose a fault diagnosis (FD) approach for a class of nonlinear uncertain systems based on the deterministic learning approach (DLA). Specifically, an adaptive learning observer is constructed, in which the adaptive neural networks (NNs) are constructed to approximate the unknown system dynamics under normal and fault modes. Based on the strictly positive real (SPR) condition, the convergence of the state estimation can be guaranteed. When the system is undergoing a periodic or periodic-like (recurrent) motion, the states of the observer will also become recurrent. Thus through DLA, the partial persistent excitation (PE) condition of the associated subvectors of NNs is satisfied. By utilizing the partial (PE) condition, the uniformly completely observable (UCO) property of the identification system is analyzed and the exponential convergence condition of the identification system is derived. Under this condition, the unknown dynamics under normal and fault modes can be accurately identified along the system trajectory. And by utilizing the knowledge obtained in the identification phase, the fault can be detected in the diagnosis phase. The main attraction of this paper lies in the analytical result, which shows that the exponential convergence condition of the learning observer not only depends on the observer gain matrix, but also depends on the PE level of the regressor subvector of NN. Simulation results are included to illustrate the effectiveness of the proposed scheme.

Robust adaptive model predictive control with persistent excitation conditions

For constrained linear systems with bounded disturbances and parametric uncertainty, we propose a robust adaptive model predictive control strategy with online parameter estimation. Constraints enforcing persistently exciting closed loop control actions are introduced for a set-membership parameter identification scheme. The algorithm requires the online solution of a convex program, satisfies constraints robustly, and ensures recursive feasibility and input-to-state stability. Almost sure convergence to the actual system parameters is demonstrated under assumptions on stabilizability, reachability, and tight disturbance bounds.

Deterministic learning-based neural control for output-constrained strict-feedback nonlinear systems

This paper studies learning from adaptive neural control of output-constrained strict-feedback uncertain nonlinear systems. To overcome the constraint restriction and achieve learning from the closed-loop control process, there are several significant steps. Firstly, a state transformation is introduced to convert the original constrained system output into an unconstrained one. Then an equivalent n-order affine nonlinear system is constructed based on the transformed unconstrained output state in norm form by the system transformation method. By combining dynamic surface control (DSC) technique, an adaptive neural control scheme is proposed for the transformed system. Then all closed-loop signals are uniformly ultimately bounded and the system output tracks the expected trajectory well with satisfying the constraint requirement. Secondly, the partial persistent excitation condition of the radial basis function neural network (RBF NN) could be verified to achieve. Therefore, the uncertain dynamics can be precisely approximated by RBF NN. Subsequently, the learning ability of RBF NN is achieved, and the knowledge acquired from the neural control process is stored in the form of constant neural networks (NNs). By reutilizing the knowledge, a novel learning controller is established to improve the control performance when facing the similar or same control task. The proposed learning control (LC) scheme can avoid repeating the online adaptation of neural weight estimates, which saves computing resources and improves transient performance. Meanwhile, the LC method significantly raises the tracking accuracy and the speed of error convergence while satisfying of the constraint condition simultaneously. Simulation studies demonstrate the efficiency of this proposed control scheme.

Nash Equilibrium Seeking for High-Order Multiagent Systems With Unknown Dynamics

In this article, we consider a Nash equilibrium seeking problem for a class of high-order multiagent systems with unknown dynamics. Different from existing results for single integrators, we aim to steer the outputs of this class of uncertain high-order agents to the Nash equilibrium of some noncooperative game in a distributed manner. To overcome the difficulties brought by the high-order structure, unknown nonlinearities, and the regulation requirement, we first introduce a virtual player for each agent and solve an auxiliary noncooperative game for them. Then, we develop a distributed adaptive protocol by embedding this auxiliary game dynamics into some proper tracking controller for the original agent to resolve this problem. We also discuss the parameter convergence issue under certain persistence of excitation conditions. The efficacy of our algorithms is verified by numerical examples.