Continuous-time Uncertain Nonlinear Systems Research Articles

Event-Triggered Robust Adaptive Dynamic Programming for Multiplayer Stackelberg-Nash Games of Uncertain Nonlinear Systems.

In this article, an event-triggered robust adaptive dynamic programming (ETRADP) algorithm is developed to solve a class of multiplayer Stackelberg-Nash games (MSNGs) for uncertain nonlinear continuous-time systems. Considering the different roles of players in the MSNG, the hierarchical decision-making process is described as the designed value functions for the leader and all followers, which assist to transform the robust control problem of the uncertain nonlinear system into an optimal regulation problem of the nominal system. Then, an online policy iteration algorithm is formulated to solve the derived coupled Hamilton-Jacobi equation. Meanwhile, an event-triggered mechanism is designed to alleviate computational and communication burdens. Moreover, critic neural networks (NNs) are constructed to obtain the event-triggered approximate optimal control polices for all players, which constitute the Stackelberg-Nash equilibrium of the MSNG. By using Lyapunov's direct method, the stability of the closed-loop uncertain nonlinear system is guaranteed under the ETRADP-based control scheme in the sense of uniform ultimate boundedness. Finally, a numerical simulation is provided to demonstrate the effectiveness of the present ETRADP-based control scheme.

Optimal Adaptive Control of Uncertain Nonlinear Continuous-Time Systems With Input and State Delays.

In this article, an actor-critic neural network (NN)-based online optimal adaptive regulation of a class of nonlinear continuous-time systems with known state and input delays and uncertain system dynamics is introduced. The temporal difference error (TDE), which is dependent upon state and input delays, is derived using actual and estimated value function and via integral reinforcement learning. The NN weights of the critic are tuned at every sampling instant as a function of the instantaneous integral TDE. A novel identifier, which is introduced to estimate the control coefficient matrices, is utilized to obtain the estimated control policy. The boundedness of the state vector, critic NN weights, identification error, and NN identifier weights are shown through the Lyapunov analysis. Simulation results are provided to illustrate the effectiveness of the proposed approach.

Temporal logic guided safe model-based reinforcement learning: A hybrid systems approach

This paper studies the problem of synthesizing control policies for uncertain continuous-time nonlinear systems from linear temporal logic (LTL) specifications using model-based reinforcement learning (MBRL). Rather than taking an abstraction-based approach, we view the interaction between the LTL formula’s corresponding Büchi automaton and the nonlinear system as a hybrid automaton whose discrete dynamics match exactly those of the Büchi automaton. To find satisfying control policies, we pose a sequence of optimal control problems associated with states in the accepting run of the automaton and leverage control barrier functions (CBFs) to prevent specification violation. Since solving many optimal control problems for a nonlinear system is computationally intractable, we take a learning-based approach in which the value function of each problem is learned online in real-time. Specifically, we propose a novel off-policy MBRL algorithm that allows one to simultaneously learn the uncertain dynamics of the system and the value function of each optimal control problem online while adhering to CBF-based safety constraints. Unlike related approaches, the MBRL method presented herein decouples convergence, stability, and safety, allowing each aspect to be studied independently, leading to stronger safety guarantees than those developed in related works. Numerical results are presented to validate the efficacy of the proposed method.

Robust approximate symbolic models for a class of continuous-time uncertain nonlinear systems via a control interface

Discrete abstractions have become a standard approach to assist control synthesis under complex specifications. Most techniques for the construction of a discrete abstraction for a continuous-time system require time–space discretization of the concrete system, which constitutes property satisfaction for the continuous-time system non-trivial. In this work, we aim at relaxing this requirement by introducing a control interface. Firstly, we connect the continuous-time uncertain concrete system with its discrete deterministic state-space abstraction with a control interface. Then, a novel stability notion called η-approximately controlled globally practically stable, and a new simulation relation called robust approximate simulation relation are proposed. It is shown that the uncertain concrete system, under the condition that there exists an admissible control interface such that the augmented system (composed of the concrete system and its abstraction) can be made η-approximately controlled globally practically stable, robustly approximately simulates its discrete abstraction. The effectiveness of the proposed results is illustrated by two simulation examples.

Higher-Order Sliding Mode design with Bounded Integral Control generation

In this paper uncertain continuous-time nonlinear systems affine in the control variable and with saturated actuators are considered. The finite-time regulation problem of the system output to zero is then solved by proposing a generic Higher-Order Sliding Mode (HOSM) controller equipped with a novel mechanism to encounter the saturation limits, thus extending previous results on saturated control inputs valid only in case of specific r-order sliding mode algorithms. The so-called Bounded Integral Control (BIC) method is reformulated into the framework of continuous HOSM, so as to replace the traditional integrator used to generate the continuous signal directly fed into the plant. Stability conditions for tuning the proposed algorithm are provided, and a numerical example finally assesses the effectiveness of the proposed technique.

Robust Tracking Control for Non-Zero-Sum Games of Continuous-Time Uncertain Nonlinear Systems

In this paper, a new adaptive critic design is proposed to approximate the online Nash equilibrium solution for the robust trajectory tracking control of non-zero-sum (NZS) games for continuous-time uncertain nonlinear systems. First, the augmented system was constructed by combining the tracking error and the reference trajectory. By modifying the cost function, the robust tracking control problem was transformed into an optimal tracking control problem. Based on adaptive dynamic programming (ADP), a single critic neural network (NN) was applied for each player to solve the coupled Hamilton–Jacobi–Bellman (HJB) equations approximately, and the obtained control laws were regarded as the feedback Nash equilibrium. Two additional terms were introduced in the weight update law of each critic NN, which strengthened the weight update process and eliminated the strict requirements for the initial stability control policy. More importantly, in theory, through the Lyapunov theory, the stability of the closed-loop system was guaranteed, and the robust tracking performance was analyzed. Finally, the effectiveness of the proposed scheme was verified by two examples.

Robustness Improvement of Computationally Efficient Cooperative Fuzzy Model Predictive-Integral Sliding Mode Control of Nonlinear Systems

This paper introduces a systematic and comprehensive method to design a constrained fuzzy model predictive control (MPC) cooperated with integral sliding mode control (ISMC) based on the Takagi-Sugeno (T-S) fuzzy model for uncertain continuous-time nonlinear systems subject to external disturbances. The proposed controller benefits from the robustness, optimality, and practical constraints considerations. The robustness against the uncertainties and matched external disturbances is achieved by the proposed ISMC without iterative calculation for obtaining the robust invariant set. The MPC schemes are designed separately based on the both quadratic and non-quadratic Lyapunov functions. By the proposed MPC, the states of the system reach the desired values in the optimal, constrained, and robust manner against the unmatched external disturbances. New linear matrix inequalities (LMIs) conditions are proposed to design both the proposed MPC schemes. Also, the practical constraints on the control signals are guaranteed in the design procedure based on the invariant ellipsoid set. To evaluate the effectiveness of the suggested strategy, some simulation and experimental tests were run.

Optimal control based on neural observer with known final time for fractional order uncertain non-linear continuous-time systems

In this paper, an optimal control scheme with known final time is presented for continuous time fractional order nonlinear systems with an unknown term in the dynamics of the system. Fractional derivative is considered based on Caputo concept and fractional order is between 0 and 1. First, a neural network observer with fractional order dynamics is designed to estimate system states. Weights of the neural network are updated adaptively and the update laws are presented as equations of fractional order. By using the Lyapunov method, it is shown that state estimation error and weight estimation error are limited. Then, the optimal control problem with known final time for fractional order nonlinear systems is presented based on observed states. Finally, the simulation results show efficiency of the proposed method.

A new robust LMI-based model predictive control for continuous-time uncertain nonlinear systems

ABSTRACT This paper presents a new robust predictive controller for a special class of continuous-time non-linear systems with uncertainty. These systems have bounded disturbances with unknown upper bound as well as constraints on input states. The controller is designed in the form of an optimization problem of the ‘worst-case’ objective function over an infinite moving horizon. Through this objective function, constraints and uncertainties can be applied explicitly on the controller design, which guarantees the system stability. Next, LMI tool is used to improve the calculation time and complexity. To do this, in order to find the optimum gain for state-feedback, the optimization problem is solved using LMI method in each time step. Finally, to show the efficiency and effectiveness of the proposed algorithm, a surge phenomenon avoidance problem in centrifugal compressors is solved.

Sliding mode learning algorithm based adaptive neural observer strategy for fault estimation, detection and neural controller of an aircraft

In this paper, two different adaptive strategies are presented for continuous time uncertain nonlinear systems with unknown disturbances and faults. In first strategy, a sliding mode control based adaptive neural observer approach is anticipated for estimation of unknown disturbances and faults by using the multi-layer perceptron, the weight parameters are updated by using the sliding mode online learning strategy. Conventionally, gradient descent back-propagation adaptation methods are used for neural networks training, within these adaptation methods a new theory of sliding mode control is added to conventional gradient descent back-propagation procedure. In this nonlinear control concept, the Sliding Mode Control is employed as a learning strategy, in which the neural network is considered as a control process and computes the stable and dynamic learning rates of neural network. By considering the unknown faults approximation and reconstruction, this online learning strategy shows a rapid sensor fault detection, approximation, and reconstruction with high preciseness and rapidness compared to conventional strategy and algorithms presented in literature. Approaches used in literature do not have much higher preciseness and fast response to fault occurrence compared to the strategy proposed in this study. In second strategy, the neural network controller strategy is proposed with concept of filtered error scheme. Online weight updating strategy comprise of additional term to back-propagation, plus an additional robustifying term, assures the stability and rapid convergence of the faulty system. The stability analysis of the proposed fault tolerance control is also provided. While considering stability of system, this robust online adaptive fault tolerance control shows a fast convergence in the presence of unknown disturbances and faults. The robust adaptive neural controller is compared with the conventional gradient descent based controller in the existence of various sensor faults and failures. The proposed strategies are validated on Boeing 747 100/200 aircraft, results show the efficiency, preciseness and robustness of strategies compared to the algorithm presented in literature.

Finite-horizon optimal control for continuous-time uncertain nonlinear systems using reinforcement learning

This paper investigates finite-horizon optimal control problem of continuous-time uncertain nonlinear systems. The uncertainty here refers to partially unknown system dynamics. Unlike the infinite-horizon, the difficulty of finite-horizon optimal control problem is that the Hamilton–Jacobi–Bellman (HJB) equation is time-varying and must meet certain terminal boundary constraints, which brings greater challenges. At the same time, the partially unknown system dynamics have also caused additional difficulties. The main innovation of this paper is the proposed cyclic fixed-finite-horizon-based reinforcement learning algorithm to approximately solve the time-varying HJB equation. The proposed algorithm mainly consists of two phases: the data collection phase over a fixed-finite-horizon and the parameters update phase. A least-squares method is used to correlate the two phases to obtain the optimal parameters by cyclic. Finally, simulation results are given to verify the effectiveness of the proposed cyclic fixed-finite-horizon-based reinforcement learning algorithm.

Memory-Augmented System Identification With Finite-Time Convergence

This letter presents a memory-augmented system identifier with finite-time convergence for continuous-time uncertain nonlinear systems. A memory of events with significant effect on the performance of the identifier is formed, and reuse of historic data is leveraged in the identifier's update law to guarantee that the identifier's error converges to zero in finite time. An easy-to-check and verifiable metric defined on samples collected along the system's trajectories is provided to certify the finite-time convergence. The robustness of the proposed identifier to mismatched modeling error is analyzed. Finally, a simulation example verifies the efficiency of the proposed identifier.

Event-Triggered Adaptive Dynamic Programming for Unmatched Uncertain Nonlinear Continuous-Time Systems

In this article, an event-triggered adaptive dynamic programming (ADP) method is proposed to solve the robust control problem of unmatched uncertain systems. First, the robust control problem with unmatched uncertainties is transformed into the optimal control design for an auxiliary system. Subsequently, to reduce controller executions and save computational and communication resources, an event-triggering mechanism is introduced. By using a critic neural network (NN) to approximate the value function, novel concurrent learning is developed to learn NN weights, which avoids the requirement of an initial admissible control and the persistence of excitation condition. Moreover, it is proven that the developed event-triggered ADP controller guarantees the robustness of the uncertain system and the uniform ultimate boundedness of the NN weight estimation error. Finally, by using the F-16 aircraft and the inverted pendulum with unmatched uncertainties as examples, the simulation results show the effectiveness of the developed event-triggered ADP method.

Optimal Control Based on Neuro Estimator for Fractional Order Uncertain Non-linear Continuous-Time Systems

In this paper, a novel method is presented for optimal control of fractional order systems in the presence of an unknown term in system dynamic where fractional order derivative is considered to be between zero and one. In this method, neural network is used to estimate the unknown term in system dynamic. Neural network coefficients are updated adaptively and online. Updating laws are presented considering system requirements to achieve a homogeneous fractional order system. Another problem is formulating optimal control laws for fractional order system which is solved through fractional differential calculus. Since optimal fractional order control is non-causal and does not have a online solution, step-by-step progression and predictive control idea are used to obtain control signal and combine optimal controller and estimator. This method results in an optimal run-time control and resolves unknown terms in system dynamic. In addition, the closed loop system being uniform ultimate bounded is proved through direct Lyapunov method. Finally, simulation results are given to show efficiency of the proposed method.

Neural Network-Based Optimal Tracking Control of Continuous-Time Uncertain Nonlinear System via Reinforcement Learning

In this note, optimal tracking control for uncertain continuous-time nonlinear system is investigated by using a novel reinforcement learning (RL) scheme. The uncertainty here refers to unknown system drift dynamics. Based on the nonlinear system and reference signal, we firstly formulate the tracking problem by constructing an augmented system. The optimal tracking control problem for original nonlinear system is thus transformed into solving the Hamilton–Jacobi–Bellman (HJB) equation of the augmented system. A new single neural network (NN)-based online RL method is proposed to learn the solution of tracking HJB equation while the corresponding optimal control input that minimizes the tracking HJB equation is calculated in a forward-in-time manner without requiring any value, policy iterations and the system drift dynamics. In order to relax the dependence of the RL method on traditional Persistence of Excitation (PE) conditions, a concurrent learning technique is adopted to design the NN tuning laws. The Uniformly Ultimately Boundedness of NN weight errors and closed-loop augmented system states are rigorous proved. Three numerical simulation examples are given to demonstrate the effectiveness of the proposed scheme.

Optimal Guaranteed Cost Tracking of Uncertain Nonlinear Systems Using Adaptive Dynamic Programming with Concurrent Learning

In this paper, based on adaptive dynamic programming (ADP) with concurrent learning, the problem of optimal guaranteed cost tracking is studied for a class of uncertain continuous-time nonlinear systems. First, the original uncertain system and a reference system are combined into an augmented uncertain system, and the performance index is transformed into the corresponding augmented form. Afterwards, the solution of the optimal control problem consisting of the nominal augmented system with the modified performance index is proven to be the optimal guaranteed cost tracking control law of the original uncertain system. Moreover, a concurrent learning tuning algorithm based on ADP is presented to approximate the solution of corresponding Hamilton-Jacobi-Bellman (HJB) equation, which relaxes the condition of persistent excitation (PE). A neural network-based approximate optimal guaranteed cost tracking design is developed not only to ensure tracking error convergence to zero for all admissible uncertainties but also to achieve the minimal guaranteed cost. Finally, two simulation examples are considered to verify the effectiveness of the theoretical results.

Observer-Based Adaptive Control of Uncertain Nonlinear Systems Via Neural Networks

In this paper, a novel observer-based control strategy is proposed for a class of uncertain continuous-time nonlinear systems based on the Hamilton–Jacobi–Bellman (HJB) equation. Due to the complexity of nonlinear systems, the approximately optimal control for affine uncertain continuous-time nonlinear systems is pursued. Considering that only the output variables can be measured in the control practice, the state observer is designed to reconstruct all system states by using the output variables. The observer-based policy iteration algorithm can solve the HJB equation within the adaptive dynamic programming framework for the unknown-state uncertain nonlinear systems, where a critic neural network is constructed to approximate the optimal cost function, and then, the approximate expression of the optimal control policy can be directly derived from solving the HJB equation. In addition, the stability of the whole closed-loop system is provided based on the Lyapunov analysis.

Fuzzy Resilient Energy-to-Peak Filtering for Continuous-Time Nonlinear Systems

The problem of resilient energy-to-peak filtering for a class of uncertain continuous-time nonlinear systems is investigated in this paper. A Takagi–Sugeno fuzzy model with norm-bounded uncertainties is used to represent the nonlinear plant. Attention is focused on the design of an energy-to-peak filter such that the filtering error system is asymptotically stable and the prescribed energy-to-peak filtering performance is guaranteed, where the designed filter is assumed to have additive gain variations. The proposed design is aimed at all filter matrices with gain variations, which improves the existing results on resilient energy-to-peak filtering for continuous-time systems. A simulation example is provided to show the effectiveness of the proposed methods.

Event-Driven Adaptive Robust Control of Nonlinear Systems With Uncertainties Through NDP Strategy

In this paper, we construct an event-driven adaptive robust control approach for continuous-time uncertain nonlinear systems through a neural dynamic programming (NDP) strategy. Through system transformation and theoretical analysis, the robustness of the original uncertain system can be achieved by designing an event-driven optimal controller with respect to the nominal system under a suitable triggering condition. In addition, it is also observed that the event-driven controller has a certain degree of gain margin. Then, the NDP technique is employed to perform the main controller design task, followed by the uniform ultimate boundedness stability proof with the feedback action of the event-driven adaptive control law. The comparative effect of the present control strategy is also illustrated via two simulation examples. The established method provides a new avenue of combining adaptive dynamic programming-based self-learning control, event-triggered adaptive control, and robust control, to investigate the nonlinear adaptive robust feedback design under uncertain environment.

Adaptive tracking control for a class of continuous-time uncertain nonlinear systems using the approximate solution of HJB equation

In this paper, an adaptive tracking control scheme is designed for a class of continuous-time uncertain nonlinear systems based on the approximate solution of the Hamilton–Jacobi–Bellman (HJB) equation. Considering matched uncertainties, the tracking control of the continuous-time uncertain nonlinear system can be transformed to the optimal tracking control of the associated nominal system. By building the nominal error system and modifying its cost function, the solution of the relevant HJB equation can be contributed to the adaptive tracking control of the continuous-time uncertain nonlinear system. In view of the complexity on solving the HJB equation, its approximate solution is pursued by the policy iteration algorithm under the adaptive dynamic programming (ADP) framework, where a critic neural network is constructed to approximate the optimal cost function, and an action network is used to directly calculate the approximate optimal control law, which constitutes the tracking control law for the original uncertain system together with the steady control law. The weight convergence of the critic network and the stability of the closed-loop system are provided as the theoretical guarantee based on the Lyapunov theory. Two simulation examples are studied to verify the theoretical results and the effectiveness of the proposed tracking control scheme.