Affine Nonlinear Control Systems Research Articles

Accelerated Gradient Approach For Deep Neural Network-Based Adaptive Control of Unknown Nonlinear Systems.

Recent connections in the adaptive control literature to continuous-time analogs of Nesterov's accelerated gradient method have led to the development of new real-time adaptation laws based on accelerated gradient methods. However, previous results assume that the system's uncertainties are linear-in-the-parameters (LIP). To compensate for non-LIP uncertainties, our preliminary results developed a neural network (NN)-based accelerated gradient adaptive controller to achieve trajectory tracking for nonlinear systems; however, the development and analysis only considered single-hidden-layer NNs. In this article, a generalized deep NN (DNN) architecture with an arbitrary number of hidden layers is considered, and a new DNN-based accelerated gradient adaptation scheme is developed to generate estimates of all the DNN weights in real-time. A nonsmooth Lyapunov-based analysis is used to guarantee the developed accelerated gradient-based DNN adaptation design achieves global asymptotic tracking error convergence for general nonlinear control affine systems subject to unknown (non-LIP) drift dynamics and exogenous disturbances. A comprehensive set of simulation studies are conducted on a two-state nonlinear system, a robotic manipulator, and a complex 20-D nonlinear system to demonstrate the improved performance of the developed method. Our simulation studies demonstrate enhanced tracking and function approximation performance from both DNN architectures and accelerated gradient adaptation.

Deadzone-adapted disturbance suppression control for global practical IOS and zero asymptotic gain to matched uncertainties

For control-affine nonlinear systems with a disturbance and unknown parameters satisfying a matching condition but not having a known bound, we introduce a direct adaptive control scheme with three design elements: (a) nonlinear damping, (b) single-gain adjustment, and (c) deadzone in the update law. Combining these three ingredients for the first time, we guarantee the following previously unattained property: an attenuation of the plant state to an assignable small level, despite unknown bounds on the parameters and disturbance, without a drift of the gain, state, and input. Given the tools and results, we refer to the proposed scheme as a Deadzone-Adapted Disturbance Suppression (DADS) Controller.

Neuroadaptive Tracking Control of Affine Nonlinear Systems Using Echo State Networks Embedded With Multiclustered Structure and Intrinsic Plasticity.

In this article, we present an echo state network (ESN)-based tracking control approach for a class of affine nonlinear systems. Different from the most existing neural-network (NN)-based control methods that are focused on the feedforward NN, the proposed method adopts a bioinspired recurrent NN fusing with multiple cluster and intrinsic plasticity (IP) to deal with modeling uncertainties and coupling nonlinearities in the systems. The key features of this work can be summarized as follows: 1) the proposed control is built upon the ESN embedded with multiclustered reservoir inspired from the hierarchically clustered organizations of cortical connections in mammalian brains; 2) the developed neuroadaptive control scheme utilizes unsupervised learning rules inspired from the neural plasticity mechanism of the individual neuron in nervous systems, called IP; 3) a multiclustered reservoir with IP is integrated into the algorithm to enhance the approximation performance of NN; and 4) the multiclustered reservoir is constructed offline and is task-independent, rendering the proposed method less expensive in computation. The effectiveness of the method is also confirmed by comparison with the existing neuroadaptive methods via numerical simulations, demonstrating that better tracking precision is achieved by the proposed method.

Zero-Sum Differential Game-Based Fault-Tolerant Control for a Class of Affine Nonlinear Systems.

This article proposes a zero-sum differential game-based control scheme for a class of affine nonlinear control systems with actuator faults, including loss of control, loss of effectiveness, and bias faults. In the control design, the bias faults and the control signal are chosen as the two opposite sides. The Nash equilibrium is achieved while the optimal control signal and the upper bound of bias faults can be derived from the Hamilton-Jacobi-Isaacs (HJI) equation. An adaptive dynamic programming (ADP) method is employed to estimate the weight of the critic network. The developed zero-sum differential game-based control scheme can ensure the system stability and optimal performance. Two simulation results on a numerical system and a rigid spacecraft model illustrate the effectiveness of the proposed control scheme.

Decomposition of Equations of Nonlinear Affine Control Systems and its Application to the Synthesis of Regulators

The article deals with the decomposition of nonlinear differential equations based on the group-theoretic approach. At the beginning, the decomposition of differential equations of linear systems using a transition matrix of state is presented, and then, based on the theory of continuous groups (Lie groups), the process of decomposition of differential equations of nonlinear systems is shown. The decomposition approach is based on the isomorphism theorem of the space of vector fields and Lie derivatives, which allows us to consider vector fields as differential operators of smooth functions. A formula is derived about the adjoin representation of a Lie group in its Lie algebra, which actually determines the finding of a vector field that characterizes the interaction of two or more vector fields. The Lie algebra of derivatives makes it possible to determine the infinitesimal action of the Lie group, i.e. the linearization of this action is carried out (transformation of the points of the trajectory space of the original system in a small neighborhood). Decomposition allows, as in the linear case, to separate the finding of an action (only locally) of a group of transformations from the transformed points themselves. For linear systems, this separation is global. It is also shown that the decomposition of linear equations is a particular case of the decomposition of nonlinear equations. An algorithm of the method of model predictive control with Gramian weighting using this decomposition is presented. A practical example of decomposition and application of the model predictive control for stabilization of a nonstationary nonlinear system is considered.

Continuous-Time Reinforcement Learning Control: A Review of Theoretical Results, Insights on Performance, and Needs for New Designs.

This exposition discusses continuous-time reinforcement learning (CT-RL) for the control of affine nonlinear systems. We review four seminal methods that are the centerpieces of the most recent results on CT-RL control. We survey the theoretical results of the four methods, highlighting their fundamental importance and successes by including discussions on problem formulation, key assumptions, algorithm procedures, and theoretical guarantees. Subsequently, we evaluate the performance of the control designs to provide analyses and insights on the feasibility of these design methods for applications from a control designer's point of view. Through systematic evaluations, we point out when theory diverges from practical controller synthesis. We, furthermore, introduce a new quantitative analytical framework to diagnose the observed discrepancies. Based on the analyses and the insights gained through quantitative evaluations, we point out potential future research directions to unleash the potential of CT-RL control algorithms in addressing the identified challenges.

On the robustification of digital event-based stabilizers for nonlinear time-delay systems

In this paper, the robust stabilization problem by means of quantized sampled-data event-based (QSE) controllers is investigated for nonlinear systems affected by state delays and unknown disturbances. In particular, a methodology for the design of robust QSE stabilizers is provided for control-affine nonlinear systems affected by unknown actuation disturbances and unknown measurement errors. Firstly, the notion of Steepest Descent Feedback (SDF), continuous or not, is suitably revised in order to deal with the robustification of event-based controllers. Then, Input-to-State Stability (ISS) redesign methodologies are used to provide the robustification term which is added to the SDF at hand in order to arbitrarily attenuate the effects of unknown external disturbances affecting the considered control scheme. A spline approximation approach is used in order to cope with the problem of the possible non-availability in the buffer of suitable past values of the system state required for the correct application of the proposed robust QSE controller. It is proved that there exist a suitably fast sampling and an accurate quantization of the input/output channels such that: the robust QSE implementation of SDFs, continuous or not, ensures the semi-global practical stability of the related closed-loop system, regardless of the above disturbances, provided that the observation errors affects marginally the new added control term. The stabilization in the sample-and-hold sense theory is used as a tool to prove the results. The provided results include the case of non-uniform quantization of the input/output channels and the case of aperiodic sampling. Applications are presented in order to validate the results.

Off‐policy model‐based end‐to‐end safe reinforcement learning

AbstractSafety and stability considerations play a crucial role in the development of learning based strategies for control design of systems that require high levels of safety. Safe reinforcement learning (RL) based approaches traditionally seek learning of the control laws that are optimal with respect to system performance whilst ensuring system stability and safety. In this article, an off‐policy safe RL based approach is proposed for nonlinear systems affine in control in continuous time. In this novel work, safety and stability are guaranteed during initialization and exploration phases by adjusting the control input with the solution of a quadratic programming problem combining both input to state stable‐control Lyapunov function and robust control barrier function (R‐CBF) conditions. Moreover, the safety of the learned policy is assured by augmenting the cost function with a CBF to maintain safety and optimize performance simultaneously. Novel mathematically rigorous proofs are provided to establish the stability and safety guarantees, offering a sound theoretical foundation for the approach. To demonstrate the effectiveness of the algorithm, two examples are presented: engine surge and stall dynamics, and an unstable nonlinear system.

On the undesired equilibria induced by control barrier function based quadratic programs

In this paper, we analyze the system behavior for general nonlinear control-affine systems when a control barrier function-induced quadratic program-based controller is employed for feedback. In particular, we characterize the existence and locations of possible equilibrium points of the closed-loop system and also provide analytical results on how design parameters affect them. Based on this analysis, a simple modification on the existing quadratic program-based controller is provided, which, without any assumptions other than those taken in the original program, inherits the safety set forward invariance property, and further guarantees the complete elimination of undesired equilibrium points in the interior of the safety set as well as one type of boundary equilibrium points, and local asymptotic stability of the origin. Numerical examples are given alongside the theoretical discussions.

Motion planning and stabilization of nonholonomic systems using gradient flow approximations

Nonlinear control-affine systems described by ordinary differential equations with time-varying vector fields are considered in the paper. We propose a unified control design scheme with oscillating inputs for solving the trajectory tracking and stabilization problems under the bracket-generating condition. This methodology is based on the approximation of a gradient-like dynamics by trajectories of the designed closed-loop system. As an intermediate outcome, we characterize the asymptotic behavior of solutions of the considered class of nonlinear control systems with oscillating inputs under rather general assumptions on the generating potential function. These results are applied to examples of nonholonomic trajectory tracking and obstacle avoidance.

Adaptive optimal control of affine nonlinear systems via identifier–critic neural network approximation with relaxed PE conditions

This paper considers an optimal control of an affine nonlinear system with unknown system dynamics. A new identifier–critic framework is proposed to solve the optimal control problem. Firstly, a neural network identifier is built to estimate the unknown system dynamics, and a critic NN is constructed to solve the Hamiltonian–Jacobi–Bellman equation associated with the optimal control problem. A dynamic regressor extension and mixing technique is applied to design the weight update laws with relaxed persistence of excitation conditions for the two classes of neural networks. The parameter estimation of the update laws and the stability of the closed-loop system under the adaptive optimal control are analyzed using a Lyapunov function method. Numerical simulation results are presented to demonstrate the effectiveness of the proposed IC learning based optimal control algorithm for the affine nonlinear system.

A Learning-Based Method for Computing Control Barrier Functions of Nonlinear Systems With Control Constraints

Verifying the safety of states and designing a safety controller are very important for safety critical systems (for example, robotic and automotive systems). Since the reachable set of a state is hard to calculate online, it is difficult to determine whether the current state will enter the unsafe set in the future. Thus, the safe set of a system is usually obtained by calculating the forward invariant set offline. For control affine systems, the control barrier function provides a linear safety constraint on the control, which helps us obtain safety control online. However, the computation of control barrier functions is recognized as difficult, and existing methods make over-approximations or are only applicable to problems without control constraints. In this paper, we propose a learning-based method to compute the control barrier function for nonlinear control affine systems with control constraints. The method consists of a constraint satisfaction (CS) algorithm and a set expansion (SE) algorithm. The CS algorithm computes the control barrier function that satisfies the constraints, and the SE algorithm expands the volume of the safe set. The numerical results demonstrate that our algorithm outperforms existing methods and can improve existing control barrier functions.

Finite‐time sub‐optimal control design for control affine nonlinear systems

AbstractA finite‐time suboptimal control strategy named technique is proposed for the control of a class of control‐affine nonlinear dynamic systems in this study. The nonlinear dynamic is expressed as a pseudo‐linear form without linearization. By adding vanishing perturbation terms into the quadratic cost functional and using a power series to represent the co‐state, an approximate solution to the Hamilton–Jacobi–Bellman (HJB) equation can be obtained through solving a differential Riccati equation offline and a series of linear Lyapunov equations with analytical solutions. The obtained suboptimal controller is in a state feedback closed‐form. By tuning the parameters in the perturbation terms, semi‐global stability can be guaranteed. In contrast to the finite‐time state‐dependent Riccati equation technique, the proposed method does not need intensive and iterative numerical computations, which facilitates real‐time implementation. Two examples are utilized to demonstrate the effectiveness and efficiency of this proposed technique.

On Sontag’s formula for the sampled-data observer-based stabilization of nonlinear time-delay systems

In this paper, a new methodology for the design of sampled-data dynamic output feedback (DOF) stabilizers for control-affine nonlinear time-delay systems is proposed. In particular, the methodology here developed is based on the Artstein’s theory of control Lyapunov functions and related Sontag’s formula, here extended, for the first time in the literature, to the case of DOF controllers. Firstly, the new notion of Steepest Descent Error Dynamics (SDED) is introduced and used to suitably re-formulate the well-known Sontag’s universal formula for the case of DOF controllers. Then, it is shown that, under suitably fast sampling, a proposed sampled-data DOF controller based on the new provided version of Sontag’s universal formula ensures the semiglobal practical stability of the related sampled-data closed-loop system. Time-varying sampling intervals, as well as the intersampling system behavior, are taken into account. In the theory here developed, the case of delay-free systems is addressed as a special case. Applications are presented for the validation of the proposed theoretical results.

A Survey on the Control Lyapunov Function and Control Barrier Function for Nonlinear-Affine Control Systems

This survey provides a brief overview on the control Lyapunov function (CLF) and control barrier function (CBF) for general nonlinear-affine control systems. The problem of control is formulated as an optimization problem where the optimal control policy is derived by solving a constrained quadratic programming (QP) problem. The CLF and CBF respectively characterize the stability objective and the safety objective for the nonlinear control systems. These objectives imply important properties including controllability, convergence, and robustness of control problems. Under this framework, optimal control corresponds to the minimal solution to a constrained QP problem. When uncertainties are explicitly considered, the setting of the CLF and CBF is proposed to study the input-to-state stability and input-to-state safety and to analyze the effect of disturbances. The recent theoretic progress and novel applications of CLF and CBF are systematically reviewed and discussed in this paper. Finally, we provide research directions that are significant for the advance of knowledge in this area.

Neural Koopman Lyapunov control

Learning and synthesizing stabilizing controllers for unknown nonlinear control systems is a challenging problem for real-world and industrial applications. Koopman operator theory allows one to analyze nonlinear systems through the lens of linear systems and nonlinear control systems through the lens of bilinear control systems. The key idea of these methods lies in the transformation of the coordinates of the nonlinear system into the Koopman observables, which are coordinates that allow the representation of the original system (control system) as a higher dimensional linear (bilinear control) system. However, for nonlinear control systems, the bilinear control model obtained by applying Koopman operator based learning methods is not necessarily stabilizable. Simultaneous identification of stabilizable lifted bilinear control systems as well as the associated Koopman observables is still an open problem. In this paper, we propose a framework to construct such stabilizable bilinear models and identify their associated observables from data by simultaneously learning a bilinear Koopman embedding for the underlying unknown control affine nonlinear system as well as a Control Lyapunov Function (CLF) for the Koopman based bilinear model using a learner and falsifier. Our proposed approach thereby provides provable guarantees of asymptotic stability for the Koopman based representation of the unknown control affine nonlinear control system as a bilinear system. Numerical simulations are provided to validate the efficacy of our proposed class of stabilizing feedback controllers for unknown control-affine nonlinear systems.

Degradation Tolerant Control Learning for Discrete-Time Affine Nonlinear Systems

This paper develops a degradation tolerant optimal control in the framework of Reinforcement Learning (RL). Safety-critical and mission-critical systems require the development of new control designs that maintain system stability and performance specifications but also address incipient degradation. The aim of this work is to decelerate the speed of degradation by minimizing a cost function that includes the rate of evolution of degradation and the performance requirements. The controller is developed for discrete-time nonlinear systems affine in control, where the system's states are affected by a nonlinear degradation. Value iteration (VI) algorithm based approach is developed to find suitable approximations of both optimal control policy and optimal cost, while guaranteeing closed-loop stability and minimization of degradation rate. Offline model-based Adaptive Dynamic Programming (ADP) algorithm is developed and implemented using actor-critic structure which involves training of both actor and critic neural networks (NN). After training the actor NN with the optimal policy, the NN is implemented in real time to generate the input of the system. Simulation example shows the efficiency and feasibility of the algorithm.

Critical Nash Value Nodes for Control Affine Embedded Graphon Mean Field Games*

Graphon Mean Field Games (GMFGs) (Caines and Huang (2021)) constitute generalizations of Mean Field Games to the case where the agents form subpopulations associated with the nodes of large graphs. The work in (Foguen-Tchuendom et al. (2021), Foguen-Tchuendom et al. (2022a)) analyzed the stationarity of equilibrium Nash values with respect to node location for large populations of non-cooperative agents with linear dynamics on large graphs embedded in Euclidean space together with their limits (termed embedded graphons). That analysis is extended in this investigation to agent systems lying in the class of control affine non-linear systems (see Isidori (1985)). Specifically, control affine GMFG systems are treated where (i) at each node αeV the drift of each generic agent system is affine in the control function, and (ii) the running costs at each node α ϵ V ⊂ Rm are exponentiated negative inverse quadratic (ENIQ) functions of the difference between a generic state and the local graphon weighted mean Zα,µG where µG:= {µβ,β ϵ V ⊂ Rm} is the globally distributed family of mean fields. The infinite cardinality node and edge limits are considered, where it is assumed that the limit embedded graphon g(α, β), (α, β) ϵ V x V, is continuously differentiable. It is shown that the equilibrium Nash value Vα is stationary with respect to the nodal location α ϵ V if and only if the corresponding mean Zα,µG is stationary with respect to nodal location.

Global Controllability Criteria and Motion Planning of Regular Affine Systems With Drifts

In this article, we give a condition for the global controllability of affine nonlinear control systems with drifts on Euclidean spaces. Under regularity assumptions, the condition is necessary and sufficient in the codimension-1 and codimension-2 cases, and holds for systems of higher codimensions under mild restrictions. We then investigate motion planning problems for codimension-1 affine systems, and give proof of the global existence of the lift to control curves for certain drifted systems using the homotopy continuation method.

Control Barriers in Bayesian Learning of System Dynamics

This article focuses on learning a model of system dynamics online, while satisfying safety constraints. Our objective is to avoid offline system identification or hand-specified models and allow a system to safely and autonomously estimate and adapt its own model during operation. Given streaming observations of the system state, we use Bayesian learning to obtain a distribution over the system dynamics. Specifically, we propose a new matrix variate Gaussian process (MVGP) regression approach with an efficient covariance factorization to learn the drift and input gain terms of a nonlinear control-affine system. The MVGP distribution is then used to optimize the system behavior and ensure safety with high probability, by specifying control Lyapunov function (CLF) and control barrier function (CBF) chance constraints. We show that a safe control policy can be synthesized for systems with arbitrary relative degree and probabilistic CLF-CBF constraints by solving a second-order cone program. Finally, we extend our design to a self-triggering formulation, adaptively determining the time at which a new control input needs to be applied in order to guarantee safety.