Continuous-time Nonlinear Systems Research Articles

Robust adaptive MPC using control contraction metrics

We present a robust adaptive model predictive control (MPC) framework for nonlinear continuous-time systems with bounded parametric uncertainty and additive disturbance. We utilize general control contraction metrics (CCMs) to parameterize a homothetic tube around a nominal prediction that contains all uncertain trajectories. Furthermore, we incorporate model adaptation using set-membership estimation. As a result, the proposed MPC formulation is applicable to a large class of nonlinear systems, reduces conservatism during online operation, and guarantees robust constraint satisfaction and convergence to a neighborhood of the desired setpoint. One of the main technical contributions is the derivation of corresponding tube dynamics based on CCMs that account for the state and input dependent nature of the model mismatch. Furthermore, we online optimize over the nominal parameter, which enables general set-membership updates for the parametric uncertainty in the MPC. Benefits of the proposed homothetic tube MPC and online adaptation are demonstrated using a numerical example involving a planar quadrotor.

Robust safe reinforcement learning control of unknown continuous-time nonlinear systems with state constraints and disturbances

Industrial processes must operate safely and optimally in practice, which typically entails solving a constrained optimal control (COC) problem. Finding a control policy for such a problem, however, is challenging especially when the system dynamics are unknown. In this paper, a novel safe reinforcement learning (RL) algorithm is proposed to deal with the COC problem for the continuous-time nonlinear system with unknown dynamics and disturbances. The presented method can ensure that the system operates within a predefined safe region in the presence of system model uncertainties and external disturbances, which goes beyond the results of typical RL methods. To handle the system state constraints, the problem is first transformed into an unconstrained one via the proposed data-driven slack function approach. Then an improved model-based RL method is devised to learn a near-optimal and safe control policy in real-time. Finally, a robust compensator and the composite neural-network updating rule are designed to eliminate the influence of disturbances and uncertainties. Theoretical analysis based on the Lyapunov approach is conducted to prove the closed-loop system stability and the algorithm convergence, which further guarantees the satisfaction of safety constraints. The effectiveness of the proposed RL algorithm is verified through simulations.

Robust stabilization and adaptive H∞ control of nonlinear singular systems via static output feedback

This article studies robust stabilization and adaptive [Formula: see text] control problems for a class of continuous-time nonlinear singular systems by static output feedback. First, under sufficient conditions which the system is impulse controllable, the static output feedback controller is designed, and the stabilization problem of the system is discussed. Second, the [Formula: see text] and adaptive [Formula: see text] controllers are designed for the continuous-time nonlinear singular systems with external disturbances. Compared with the existing conclusions, the controller can be designed for more nonlinear singular systems. Finally, an example of a nonlinear singular circuit is given, and the simulation results verify the validity of the proposed controllers.

Optimal Adaptive Control of Uncertain Nonlinear Continuous-Time Systems With Input and State Delays.

In this article, an actor-critic neural network (NN)-based online optimal adaptive regulation of a class of nonlinear continuous-time systems with known state and input delays and uncertain system dynamics is introduced. The temporal difference error (TDE), which is dependent upon state and input delays, is derived using actual and estimated value function and via integral reinforcement learning. The NN weights of the critic are tuned at every sampling instant as a function of the instantaneous integral TDE. A novel identifier, which is introduced to estimate the control coefficient matrices, is utilized to obtain the estimated control policy. The boundedness of the state vector, critic NN weights, identification error, and NN identifier weights are shown through the Lyapunov analysis. Simulation results are provided to illustrate the effectiveness of the proposed approach.

Output Bipartite Consensus for Second-Order Heterogeneous Uncertain Agents With State-Dependent Cooperation-Competition Interactions

For state-dependent cooperation-competition networks, the output bipartite consensus problem of heterogeneous uncertain agents is studied in this paper. In our framework, all the agents are described by second-order continuous-time nonlinear systems with different intrinsic dynamics, and the agents' uncertainties are characterized by unknown parameters in the intrinsic term. Then, the edge evolution rules with hysteresis coefficients are proposed via the classification strategy. To solve this problem, the distributed Lyapunov-based redesign method with the potential function term is first applied to such second-order heterogeneous uncertain multi-agent system. Through the three-step correlation design, the explicit expressions of the distributed controllers and the unknown parameter estimators are obtained, and it is shown that the state-dependent cooperation-competition network is always connected and maintains structural balance for all time if an initial structurally balanced and connected network topology is provided. On this basis, it follows from the total Lyapunov function that output bipartite consensus can be achieved asymptotically for the heterogeneous uncertain multi-agent system. Finally, a numerical simulation is provided to validate the structural balance of state-dependent networks and output bipartite consensus of heterogeneous uncertain agents.

Partial stability of nonlinear dissipative feedback systems

Partially stable systems involve dynamical systems whose motion lie in a subspace of the state space resulting in system stability with respect to part of the system's states. In this paper, we develop partial stability theorems for nonlinear continuous-time and discrete-time dissipative feedback systems. Specifically, by invoking additional structural constraints on the forward loop and feedback loop system storage functions, we develop feedback interconnection partial stability results for dissipative nonlinear dynamical systems. Our results provide extensions of the positivity and small gain theorems for guaranteeing partial stability of feedback interconnected systems.

Online Reinforcement Learning Control of Nonlinear Dynamic Systems: A State-action Value Function Based Solution

In this paper, we present an online reinforcement learning-based solution to the optimal control problem of continuous-time nonlinear input-affine systems. The proposed approach contains a concurrent identifier that estimates time derivatives of states of the system in some arbitrary points. The identifier is utilized to simulate a so-called Bellman error in some unvisited points. The simulated errors together with errors obtained along the trajectory of the system are used to estimate the state-action value function, which is then employed to derive the estimated optimal controller. The designed approach does not explicitly require the input dynamics, which is hard to segregate it from the drift dynamics in optimal regulation problems. In addition, the simulated Bellman errors relax the confining persistence of excitation condition, which is needed for convergence in deterministic systems. A Lyapunov-based analysis was conducted to derive convergence conditions. Simulation studies demonstrated the effectiveness of the developed control scheme.

Relaxed Actor-Critic With Convergence Guarantees for Continuous-Time Optimal Control of Nonlinear Systems

This paper presents the Relaxed Continuous-Time Actor-critic (RCTAC) algorithm, a method for finding the nearly optimal policy for nonlinear continuous-time (CT) systems with known dynamics and infinite horizon, such as the path-tracking control of vehicles. RCTAC has several advantages over existing adaptive dynamic programming algorithms for CT systems. It does not require the ``admissibility" of the initialized policy or the input-affine nature of controlled systems for convergence. Instead, given any initial policy, RCTAC can converge to an admissible, and subsequently nearly optimal policy for a general nonlinear system with a saturated controller. RCTAC consists of two phases: a warm-up phase and a generalized policy iteration phase. The warm-up phase minimizes the square of the Hamiltonian to achieve admissibility, while the generalized policy iteration phase relaxes the update termination conditions for faster convergence. The convergence and optimality of the algorithm are proven through Lyapunov analysis, and its effectiveness is demonstrated through simulations and real-world path-tracking tasks.

Ellipsoidal approximations of the minimal robust positively invariant set

An optimization algorithm is presented in this paper for the minimal robust positively invariant (mRPI) set approximations via sums-of-squares (SOS) optimization. The mRPI set is an effective tool for robust analysis of uncertain systems under bounded disturbances. The approximation of the mRPI set is always characterized by a polyhedron computed after finite time iterations. In this paper, an mRPI set is characterized by an ellipsoidal set while bounded parametric uncertainties act on states. The proposed algorithm optimizes the shape matrix of the ellipsoidal set approximation by minimizing the volume of the ellipsoidal set. The algorithm is designed for discrete-time and continuous-time nonlinear systems respectively. The algorithm has the ability to further minimize the mRPI set by optimizing the state-feedback control law. Examples are employed to validate the effectiveness of the proposed algorithms.

Fuzzy Observer-Based Output Feedback Control of Continuous-Time Nonlinear Two-Dimensional Systems

The fuzzy observer-based output feedback control of continuous-time nonlinear two-dimensional systems is studied by T-S fuzzy models in this work. The plant information of the two-dimensional systems evolves along two independent directions dynamically. The nonlinear two-dimensional systems are firstly expressed by Takagi-Sugeno fuzzy models with parameter uncertainties. By a Lyapunov method together with some convexification techniques, two methods are developed for fuzzy observer-based output feedback controller synthesis of the underlying fuzzy two-dimensional systems, and novel output feedback controller synthesis results are proposed within a convex optimization setup. Simulation studies are provided to illustrate the validity of the proposed methods.

Adaptive learning control for nonlinear systems: A single learning estimation scheme is enough

In this brief, continuous-time nonlinear systems with extended matching uncertainties are considered. The problem of designing a state-feedback adaptive learning control of reduced complexity — just including a single adaptive learning estimation scheme in the upper subsystem and a high-gain proportional action in the input channel — is addressed. By properly setting the control parameters, exponential output tracking of (sufficiently smooth) periodic reference signals with a known period is achieved. Fourier series expansions are used and estimates of the resulting Fourier coefficients are continuously adapted based on the persistency of excitation conditions that naturally hold due to the orthogonal nature of the sinusoidal basis functions.

Hierarchical Reinforcement Learning-based Supervisory Control of Unknown Nonlinear Systems

A supervisory control approach using hierarchical reinforcement learning (HRL) is developed to approximate the solution to optimal regulation problems for a control-affine, continuous-time nonlinear system with unknown drift dynamics. This result contains two objectives. The first objective is to approximate the optimal control policy that minimizes the infinite horizon cost function of each approximate dynamic programming (ADP) sub-controller. The second objective is to design a switching rule, by comparing the approximated optimal value functions of the ADP sub-controllers, to ensure that switching between subsystems yields a lower cost than using one subsystem. An integral concurrent learning-based parameter identifier approximates the unknown drift dynamics. Uniformly ultimately bounded regulation of the system states to a neighborhood of the origin, and convergence of the approximate control policy to a neighborhood of the optimal control policy, are proven using a Lyapunov-based stability and dwell-time analysis.

Stochastic Models to Mitigate Sparse Sensor Attacks in Continuous-Time Non-Linear Cyber-Physical Systems

Stochastic Models to Mitigate Sparse Sensor Attacks in Continuous-Time Non-Linear Cyber-Physical Systems

Sampled-Data Model-Free Adaptive Sliding Mode Control for Piezoelectric Actuators Subject to Time Delay

This paper proposes a model-free sampled-data sliding mode control of piezoelectric actuators considering time delay. As time delay is a challenging subject in controller design, we try to eradicate its effect on the tracking performance of the control system as much as possible. For this goal, an equivalent sampled-data data-based model from the continuous-time nonlinear system of a piezoelectric actuator is first derived. Then, we include the delay of the control system into the sliding surface to enrich the controller performance. Next, the sliding mode controller is designed using the equivalent model and the delayed sliding surface. No observer is used to compensate for the total disturbances. Thus, the scheme is easy to design and implement. The stability of the closed-loop system is theoretically studied, and the controller performance is confirmed through comparative experimental tests. Based on the experimental results, the proposed controller successfully obtains control objectives.

Decentralized tracking control design based on intelligent critic for an interconnected spring-mass-damper system

In this paper, the decentralized tracking control (DTC) problem is investigated for a class of continuous-time nonlinear systems with external disturbances. First, the DTC problem is resolved by converting it into the optimal tracking controller design for augmented tracking isolated subsystems (ATISs). %It is investigated in the form of the nominal system. A cost function with a discount is taken into consideration. Then, in the case of external disturbances, the DTC scheme is effectively constructed via adding the appropriate feedback gain to each ATIS. %Herein, we aim to obtain the optimal control strategy for minimizing the cost function with discount. In addition, utilizing the approximation property of the neural network, the critic network is constructed to solve the Hamilton-Jacobi-Isaacs equation, which can derive the optimal tracking control law and the worst disturbance law. Moreover, the updating rule is improved during the process of weight learning, which removes the requirement for initial admission control. Finally, through the interconnected spring-mass-damper system, a simulation example is given to verify the availability of the DTC scheme.

Almost Fixed Time Observers for Parameters and for State Variables of Nonlinear Systems

We provide new finite-time and fast-converging observers for continuous-time nonlinear systems that contain unknown constant parameters and measurement delays. When the dynamics are affine in the unmeasured state, we obtain fixed time observers that identify unknown states and unknown model parameters. When the dynamics contain a nonlinearity that depends on the unmeasured state, we instead obtain asymptotic convergence of our observers, whose rate of convergence converges to infinity as the growth rate of this nonlinearity converges to zero, so in this case we call the observers almost fixed time observers. Our examples illustrate how our assumptions can be easily checked.

Influence of Discretization in Dynamically Embedded Model Predictive Control

The paper analyzes the closed-loop stability of Dynamically Embedded Model Predictive Control for input-constrained continuous-time nonlinear systems. Given a stabilizing continuous-time optimal control problem, the proposed method performs a discrete approximation to obtain a finite number of optimization variables. The resulting optimization problem is then embedded into a dynamic feedback law that evolves in parallel to the system. Using Input-to-State Stability, it is shown that the dynamic interconnection between the ideal continuous-time model predictive controller and the dynamically embedded solver is asymptotically stable for a sufficiently small discretization step and sufficiently fast solver dynamics. Numerical results, however, highlight a counter-intuitive behavior: as the discretization step decreases, the stability of the closed-loop system tends to deteriorate. This suggests that, although the discretization should be sufficiently accurate to correctly capture the behavior of the system, oversampling the system dynamics may be just as harmful as undersampling.

Design of Switching-type Dynamic Quantizers for Continuous-Time Nonlinear Systems

This paper focuses on designing discrete-valued input systems using dynamic quantizers, which convert continuous-valued signals into discrete ones. Although the optimal design of dynamic quantizer is much studied, there are some unresolved problems. This study focuses on one of the unresolved problems: the design of a switching-type discrete-time dynamic quantizer for continuous-time nonlinear systems. In this paper, we first reduced the quantizer design problem to a selection problem of continuous-time linear models for a given nonlinear system. Then, we evaluated this approach via simulations of the swing-up problem for a cart-type inverted pendulum.

Iteratively-Linearized Set-Based Parameter Estimation for Uncertain Nonlinear Systems

This study presents a set-based method to estimate unknown parameters of continuous-time structured nonlinear systems under uncertainties. Sets containing the parameters are found by using data sets and structures of the systems. Such estimation is challenging because the uncertainties are caused by disturbances, data noises, and partially unknown dynamics. In addition, the estimation for continuous-time nonlinear systems faces nonlinear problems that are difficult to address. To overcome these issues, the proposed set-based method solves linear programming (LP) problems iteratively instead of the nonlinear problems. The inequalities in the LP are derived such that they are consistent with the true parameters, using the boundedness of the uncertainties. A theoretical analysis guarantees that solving the iterative LP gives sets containing the true parameters.

Unknown input observer based distributed fault detection for nonlinear multi-agent systems with probabilistic time delay

This paper focuses on the fault detection (FD) problem for nonlinear continuous-time multi-agent systems (MASs) with external disturbances and random time-varying delay. The communication topology is assumed to be undirected and connected. Firstly, a stochastic variable satisfying the Bernoulli distribution is utilized to describe the random time-varying delay and transfer the random time-varying delay to a deterministic time-varying delay. To avoid the constraints of the rank conditions for unknown input observer (UIO), the external disturbances and faults from other agents are treated as the unknown inputs which are partitioned into the decoupled part and the non-decoupled part. Sufficient conditions composed of linear matrix inequalities (LMIs) are derived to ensure the stochastic stability of the augmented system with a desired H∞ performance index and extra freedom of design. By utilizing the properties of UIOs properly, the method proposed in this paper can detect multiple faulty nodes simultaneously. Finally, a numerical example demonstrates the effectiveness and feasibility of the proposed approach.