Continuous-time Nonlinear Systems Research Articles

Improving extensions for the global uniform asymptotic stability of continuous-time nonlinear systems

This paper considers the classical problems of global asymptotic stability (GAS) of nonlinear autonomous (time-invariant) systems and global uniform asymptotic stability (GUAS) of nonlinear non-autonomous (time-varying) systems. By imposing new conditions on the Lyapunov function and its derivative, the state convergence to the origin from any initial condition with bounded norm in the classical setting is extended to the initial conditions with unbounded norms in the present work. The proposed new notions of stability are called extended global asymptotic stability (EGAS) and extended global uniform asymptotic stability (EGUAS) in this paper. The proposed Lyapunov-based criteria yield the global asymptotic stability of the origin and fixed-time/predefined-time attractiveness of any selected neighbourhood of the origin. The result is that the dynamical behaviour of the state trajectories from the convergence time point of view is improved considerably. The relationship between the smoothness and input-to-state stability (ISS) properties is also studied from a quantitative point of view. Under some new sufficient conditions, it is shown that dynamical systems with smaller Lipschitz constants represent smaller ultimate bounds, and as a result, are more robust. The desirable dynamical behaviour and robustness property of the proposed stability notion makes it comparable with the finite/fixed/predefined/prescribed (exact) stable systems that are naturally non-Lipschitz at the origin. Numerical results illustrate the effectiveness of the proposed theoretical results.

A Novel Parallel Control Method for Optimal Consensus of Nonlinear Multiagent Systems.

This work concentrates on the initial introduction of parallel control to investigate an optimal consensus control strategy for continuous-time nonlinear multiagent systems (MASs) via adaptive dynamic programming (ADP). First, the control input is integrated into the feedback system for parallel control, facilitating an augmented system's optimal consensus control with an appropriate augmented performance index function to be established, which is identical to the original system's suboptimal control with a conventional performance index. Second, the feasibility of the proposed control scheme is evaluated based on the policy iteration algorithm, and the convergence of the algorithm is demonstrated. Then, an online learning algorithm becomes available to implement the ADP-based optimal parallel consensus control protocol without prior knowledge of the system. The Lyapunov approach is employed to indicate that the signals are convergent. Ultimately, the experimental data support the theoretical results.

A new observer-based controller for continuous-time nonlinear systems with applications on electrical energy systems

In this paper a new observer-based controller is developed for continuous-time input constrained nonlinear systems. Firstly, a new developed continuous time Regularized Least Square (CRLS) estimator is presented. The estimated states are then used to generate the desired constrained state feedback control strategies to regulate the performance of a perturbed system to the desired steady state. Unlike the well-known continuous-time nonlinear state estimators, such as the high gain observer and others, the developed observer does not show large overshoots at the start of the estimation process, the estimation errors reach zero steady state very shortly, has no pre-specified limitations on the class of nonlinear systems to be estimated, …etc. As a result, when used as an element in observer-based controlled systems, then the desired behaviors of the state trajectories of the controlled system are reached very shortly. The asymptotic stability of the proposed constrained observer-based controlled nonlinear system is rigorously analyzed. Simulation results are presented to justify the applicability and the efficiency of the proposed approach. Firstly, to show the superiority of the developed CRLS estimator, the states of illustrative practical problems are firstly estimated using the proposed approach. The achieved results are then compared with those achieved from applying the most famous and widely used observers in the literature. Then, two nonlinear practical control problems are used to demonstrate the effectiveness of the developed observer- based control approach.

Induced ℒ∞ control for continuous-time nonlinear systems

In this paper, the problem of induced L ∞ control in the presence of disturbance signals in the control output of continuous-time nonlinear systems is considered. Under the condition that the system control output has a disturbance signal, the expected design conditions of the controller that satisfy the asymptotic stability and the specified performance are given. The sufficient conditions for such an induced L ∞ controller are expressed in terms of linear matrix inequalities (LMIs). Finally, the effectiveness and feasibility of the proposed induced L ∞ control design strategy is illustrated by a tunnelling diode circuit.

Optimal trajectory tracking of uncertain nonlinear continuous-time strict-feedback systems with dynamic constraints

A novel method for optimal trajectory tracking of uncertain nonlinear systems by using adaptive dynamic programming (ADP)-based integral reinforcement learning (IRL) and neural networks (NNs) is proposed. The method utilises an actor-critic framework with optimal backstepping to minimise a discounted value function and uses a single-layer NN identifier for estimating the unknown dynamics. For safety assurance, a time-varying barrier Lyapunov function (TVBLF) is used in the control design to handle the constraints. A novel weight-tuning law by using the control input error, integral Bellman error, and the NN identifier is used for the actor-critic framework. A novel lifelong learning (LL)-based method for critic NN is utilised in an optimal framework by incorporating the Bellman error to mitigate catastrophic forgetting in multitasking scenarios. Stability analysis using Lyapunov stability theory is obtained for the overall closed-loop system. Simulations of leader-follower mobile robot formation control show a 25% reduction in the cost in multitasking scenarios.

Advanced stability assessment for control systems: Gelfand’s method with iterated Shanks transformation

ABSTRACT In this paper, we present an efficient method for evaluating the stability of both linear time-invariant (LTI) systems and continuous-time nonlinear systems. Our approach centres around the modulus of the spectral radius derived from a specific matrix transformation of the original system state matrix in the case of LTI systems, or the Jacobian of the nonlinear system. We enhance computational speed by employing the iterated Shanks transformation on Gelfand’s formula, eliminating the need to rely on the system’s characteristic equation. Notably, our stability criterion avoids the computation of all eigenvalues of the system matrix. Illustrative examples, including a fluidised bed reactor, a continuous stirred tank reactor and a hypothetical 20th-order dynamical system, are provided. Additionally, we conduct a CPU-time analysis, revealing that our method demonstrates several orders of magnitude faster computational speed compared to the classical Routh-Hurwitz stability test for systems ranging from two to 80 dimensions.

Integral sliding mode-based event-triggered optimal fault tolerant tracking control of continuous-time nonlinear systems

In this paper, the integral sliding mode-based event-triggered optimal fault tolerant tracking control of continuous-time nonlinear systems is investigated via adaptive dynamic programming. The developed control scheme consists of two parts, i.e., integral sliding mode control and event-triggered optimal tracking control. For the first part, an integral sliding mode controller is designed to eliminate the affect of actuator fault and the dynamics of nominal nonlinear systems is obtained. For the second part, a novel quadratic cost function with respect to the tracking error and its dynamics is developed such that the feedforward control law or the discount factor is not required, which reduces the complexity of the control method and guarantees the tracking performance. Moreover, a critic-only structure is established to obtain the solution of tracking Hamilton–Jacobi–Bellman equation. It should be noted that the optimal tracking control law is updated only at triggering moments in order to preserve computing and communication resources. Finally, the effectiveness of the present approach is demonstrated through simulation examples of a robotic arm system and a Van der Pol circuit system.

Integral reinforcement learning-based dynamic event-triggered safety control for multiplayer Stackelberg–Nash games with time-varying state constraints

In this article, an integral reinforcement learning-based dynamic event-triggered safety control scheme is proposed to tackle the multiplayer Stackelberg–Nash games (MSNG) problem in continuous-time nonlinear systems with time-varying state constraints. Initially, a new barrier function (BF) is introduced by integrating traditional barrier functions with a novel smooth function to address the challenge of time-varying state constraints. Then, the constrained MSNG system is transformed into an unconstrained system through the application of the state transformation technique, which helps to characterize hierarchical decision problems as MSNG problems with the leader and followers. Meanwhile, the integral reinforcement learning (IRL) technique is also applied to ease the demand for precise system dynamics. Moreover, a new dynamic event-triggered control (DETC) mechanism is designed, resulting in coupled dynamic event-triggered Hamilton–Jacobi (HJ) equations. A single critic neural network (NN) is constructed to learn the optimal control laws for the leader and followers. By employing Lyapunov theory, the integral reinforcement learning-based dynamic event-triggered safety control method ensures that the system state and the critic NN weight errors are uniformly ultimately bounded (UUB). Finally, two simulation examples are provided to demonstrate the effectiveness of the proposed algorithm.

Sliding-mode surface-based adaptive optimal nonzero-sum games for saturated nonlinear multi-player systems with identifier-critic networks

This paper addresses the sliding-mode surface-based adaptive optimal nonzero-sum games control problem for continuous-time nonlinear systems with input constraints. By constructing a value function associated with the sliding-mode surface, the original nonzero-sum games problem can be transformed into constructing multiple optimal control policies. Then, an adaptive dynamic programming method under identifier-critic networks is introduced to deal with the Hamilton–Jacobi equation, where the identifier networks are applied to approximate the unknown dynamics. A remarkable feature is that the critic updating laws are designed through experience replay and gradient descent methodologies, such that the persistence of excitation condition can be excluded. Based on the Lyapunov stability theory, all signals of the close-loop system are verified to be uniformly ultimately bounded. Finally, simulation results are given to illustrate the effectiveness of the proposed optimal nonzero-sum games scheme.

Combining hybrid metaheuristic algorithms and reinforcement learning to improve the optimal control of nonlinear continuous-time systems with input constraints

This paper proposes an innovative method for achieving optimal tracking control in nonlinear continuous-time systems with input constraints. The method combines reinforcement learning and hybrid metaheuristics to enhance the controller’s performance. Specifically, a hybrid metaheuristic algorithm is employed to optimize the hyperparameters of a critic neural network, which serves as the system’s controller. The proposed approach is evaluated through extensive simulation studies on a nonlinear system with input constraints. Results demonstrate its superiority over traditional control techniques in terms of accuracy, timeliness, and stability. Notably, the approach effectively eliminates overshoot and steady-state error while providing precise and prompt tracking and showcasing remarkable robustness against model uncertainties. By synergistically integrating reinforcement learning and hybrid metaheuristics, this approach represents a significant advancement in enhancing the control performance of complex nonlinear systems. The simulation studies confirm superiority of the proposed approach over existing techniques, offering a promising solution for achieving optimal tracking control in nonlinear systems with input constraints. This approach holds potential for a wide range of applications, including robotics, aerospace, and manufacturing, where precise and prompt tracking control is critical.

Convergence Analysis of Value Iteration Adaptive Dynamic Programming for Continuous-Time Nonlinear Systems.

This article is concerned with the convergence property and error bounds analysis of value iteration (VI) adaptive dynamic programming for continuous-time (CT) nonlinear systems. The size relationship between the total value function and the single integral step cost is described by assuming a contraction assumption. Then, the convergence property of VI is proved while the initial condition is an arbitrary positive semidefinite function. Moreover, the accumulated effects of approximation errors generated in each iteration are taken into consideration while using approximators to implement the algorithm. Based on the contraction assumption, the error bounds condition is proposed, which ensures the approximated iterative results converge to a neighborhood of the optimum, and the relation between the optimal solution and approximated iterative results is also derived. To make the contraction assumption more concrete, an estimation way is proposed to derive a conservative value of the assumption. Finally, three simulation cases are given to validate the theoretical results.

Robust stability of moving horizon estimation for continuous-time systems

Abstract We consider a moving horizon estimation (MHE) scheme involving a discounted least squares objective for general nonlinear continuous-time systems. Provided that the system is detectable (incrementally integral input/output-to-state stable, i-iIOSS), we show that there exists a sufficiently long estimation horizon that guarantees robust global exponential stability of the estimation error in a time-discounted L 2-to-L ∞ sense. In addition, we show that i-iIOSS Lyapunov functions can be efficiently constructed by verifying certain linear matrix inequality conditions. In combination, we propose a flexible Lyapunov-based MHE framework in continuous time, which particularly offers more tuning possibilities than its discrete-time analog, and provide sufficient conditions for stability that can be easily verified in practice. Our results are illustrated by a numerical example.

Continual online learning-based optimal tracking control of nonlinear strict-feedback systems: application to unmanned aerial vehicles

A novel optimal trajectory tracking scheme is introduced for nonlinear continuous-time systems in strict feedback form with uncertain dynamics by using neural networks (NNs). The method employs an actor-critic-based NN backstepping technique for minimizing a discounted value function along with an identifier to approximate unknown system dynamics that are expressed in augmented form. Novel online weight update laws for the actor and critic NNs are derived by using both the NN identifier and Hamilton-Jacobi-Bellman residual error. A new continual lifelong learning technique utilizing the Fisher Information Matrix via Hamilton-Jacobi-Bellman residual error is introduced to obtain the significance of weights in an online mode to overcome the issue of catastrophic forgetting for NNs, and closed-loop stability is analyzed and demonstrated. The effectiveness of the proposed method is shown in simulation by contrasting the proposed with a recent method from the literature on an underactuated unmanned aerial vehicle, covering both its translational and attitude dynamics.

Nearly optimal stabilization of unknown continuous-time nonlinear systems: A new parallel control approach

This paper develops a novel online nearly optimal control (ONOC) method for unknown continuous-time (CT) nonaffine nonlinear systems without recovering unknown systems. First, a dynamic control law is proposed for CT nonaffine nonlinear systems using parallel control. To achieve the proposed dynamic control law, an affine augmented system (AAS) is constructed according to the original system, and an augmented performance index (API) is constructed on the basis of the original performance index (OPI). Then, the stability relationship between the original system and the AAS is provided, and it is proven that, by selecting a suitable parameter in the API, optimal control of the AAS with the API is equivalent to near-optimal control of the original system with the OPI. Subsequently, based on the proposed dynamic control law, we extend integral reinforcement learning (IRL) to completely unknown CT nonaffine systems, and it is further proved that closed-loop signals are uniformly ultimately bounded (UUB) without the assumption that the input dynamics are bounded. Furthermore, the OPI can be set to an arbitrary positive-definite form, and the UUB bound for the state vector can be predetermined. Lastly, simulations are offered to exhibit the correctness of the developed ONOC method. Source code of this paper is available at: https://github.com/lujingweihh/Adaptive-dynamic-programming-algorithms/tree/main/model_free_integral_reinforcement_learning.

Output information-based intermittent optimal control for continuous-time nonlinear systems with unmatched uncertainties via adaptive dynamic programming

Intermittent control stands as a valuable strategy for resource conservation and cost reduction across diverse systems. Nonetheless, prevailing research is intractable to address the challenges posed by robust optimal intermittent control of nonlinear input-affine systems with unmatched uncertainties. This paper aims to fill this gap. Initially, we introduce an enhanced finite-time intermittent control approach to ensure stability within nonlinear dynamic systems harboring bounded errors. A neural networks (NNs) state observer is constructed to estimate system information. Subsequently, an optimal intermittent controller that operates within a finite time span, guaranteeing system stability by employing the Hamilton–Jacobi–Bellman (HJB) methodology. Furthermore, we devise an output information-based event-triggered intermittent (ETI) approach rooted in the robust adaptive dynamic programming (ADP) algorithm, furnishing an optimal intermittent control law. In this process, a critic NNs is introduced to estimate the cost function and optimal intermittent controller. Simulation results show that our proposed method is superior to existing intermittent control strategies.

Hierarchical Sliding-Mode Surface-Based Adaptive Actor-Critic Optimal Control for Switched Nonlinear Systems With Unknown Perturbation.

This article studies the hierarchical sliding-mode surface (HSMS)-based adaptive optimal control problem for a class of switched continuous-time (CT) nonlinear systems with unknown perturbation under an actor-critic (AC) neural networks (NNs) architecture. First, a novel perturbation observer with a nested parameter adaptive law is designed to estimate the unknown perturbation. Then, by constructing an especial cost function related to HSMS, the original control issue is further converted into the problem of finding a series of optimal control policies. The solution to the HJB equation is identified by the HSMS-based AC NNs, where the actor and critic updating laws are developed to implement the reinforcement learning (RL) strategy simultaneously. The critic update law is designed via the gradient descent approach and the principle of standardization, such that the persistence of excitation (PE) condition is no longer needed. Based on the Lyapunov stability theory, all the signals of the closed-loop switched nonlinear systems are strictly proved to be bounded in the sense of uniformly ultimate boundedness (UUB). Finally, the simulation results are presented to verify the validity of the proposed adaptive optimal control scheme.

Optimal Robust Control of Nonlinear Systems with Unknown Dynamics via NN Learning with Relaxed Excitation.

This paper presents an adaptive learning structure based on neural networks (NNs) to solve the optimal robust control problem for nonlinear continuous-time systems with unknown dynamics and disturbances. First, a system identifier is introduced to approximate the unknown system matrices and disturbances with the help of NNs and parameter estimation techniques. To obtain the optimal solution of the optimal robust control problem, a critic learning control structure is proposed to compute the approximate controller. Unlike existing identifier-critic NNs learning control methods, novel adaptive tuning laws based on Kreisselmeier's regressor extension and mixing technique are designed to estimate the unknown parameters of the two NNs under relaxed persistence of excitation conditions. Furthermore, theoretical analysis is also given to prove the significant relaxation of the proposed convergence conditions. Finally, effectiveness of the proposed learning approach is demonstrated via a simulation study.

Distributed Lebesgue Approximation Model for Distributed Continuous-Time Nonlinear Systems.

Approximation models play a crucial role in model-based methods, as they enhance both accuracy and computational efficiency. This article studies distributed and asynchronous discretized models to approach continuous-time nonlinear systems. The considered continuous-time system consists of some distributed but physically coupled nonlinear subsystems that exchange information. We propose two Lebesgue approximation models (LAMs): 1) the unconditionally triggered LAM (CT-LAM) and 2) the CT-LAM. In both approaches, a specific LAM approximates an individual subsystem. The iteration of each LAM is triggered by either itself or its neighbors. The collection of different LAMs executing asynchronously together form the approximation of the overall distributed continuous-time system. The aperiodic nature of LAMs allows for a reduction in the number of iterations in the approximation process, particularly when the system has slow dynamics. The difference between the unconditionally and CT-LAMs is that the latter checks an "importance" condition, further reducing the computational effort in individual LAMs. Furthermore, the proposed LAMs are analyzed by constructing a distributed event-triggered system which is proved to have the same state trajectories as the LAMs with linear interpolation. Through this specific event-triggered system, we derive conditions on the quantization sizes in LAMs to ensure asymptotic stability of the LAMs, boundedness of the state errors, and prevention of Zeno behavior. Finally, simulations are carried out on a quarter-car suspension system to show the advantage and efficiency of the proposed approaches.

Robust Optimal Parallel Tracking Control Based on Adaptive Dynamic Programming.

This article focuses on a novel robust optimal parallel tracking control method for continuous-time (CT) nonlinear systems subject to uncertainties. First, the designed virtual controller facilitates the transformation of the original nonlinear system into an affine system with an augmented state vector, which promotes the introduction of the optimal parallel tracking control problem. Then, this article generates fresh insight into counteracting the effects of uncertainty by developing a novel parallel control system that invokes the formulated virtual control law and an auxiliary variable obtained from the relationship between the solutions of the optimal control problems for the uncertain system and the nominal one. Next, critic neural networks (NNs) approximate the Hamilton-Jacobi-Bellman (HJB) equations' solution to implement the proposed robust optimal control method via adaptive dynamic programming (ADP). Finally, simulation experiments demonstrate the proposed method's remarkable effectiveness.

Event-Triggered Robust Adaptive Dynamic Programming for Multiplayer Stackelberg-Nash Games of Uncertain Nonlinear Systems.

In this article, an event-triggered robust adaptive dynamic programming (ETRADP) algorithm is developed to solve a class of multiplayer Stackelberg-Nash games (MSNGs) for uncertain nonlinear continuous-time systems. Considering the different roles of players in the MSNG, the hierarchical decision-making process is described as the designed value functions for the leader and all followers, which assist to transform the robust control problem of the uncertain nonlinear system into an optimal regulation problem of the nominal system. Then, an online policy iteration algorithm is formulated to solve the derived coupled Hamilton-Jacobi equation. Meanwhile, an event-triggered mechanism is designed to alleviate computational and communication burdens. Moreover, critic neural networks (NNs) are constructed to obtain the event-triggered approximate optimal control polices for all players, which constitute the Stackelberg-Nash equilibrium of the MSNG. By using Lyapunov's direct method, the stability of the closed-loop uncertain nonlinear system is guaranteed under the ETRADP-based control scheme in the sense of uniform ultimate boundedness. Finally, a numerical simulation is provided to demonstrate the effectiveness of the present ETRADP-based control scheme.