In this work, an adaptive dynamic programming (ADP)-based event-triggered infinite-horizon (H∞) control algorithm is proposed for high-precision speed regulation of permanent magnet synchronous motors (PMSMs). The H∞ control problem of PMSM can be formulated as a two-player zero-sum differential game, and only a single critic neural network is needed to approximate the solution of the Hamilton–Jacobi–Isaacs (HJI) equations online, which significantly simplifies the control structure. Dynamically balancing control accuracy and update frequency through adaptive event-triggering mechanism significantly reduces the computational burden. Through theoretical analysis, the system state and critic weight estimation error are rigorously proved to be uniform ultimate boundedness, and the Zeno behavior is theoretically precluded. The simulation results verify the high accuracy tracking capability and the strong robustness of the algorithm under both load disturbance and shock load, and the event-triggering mechanism significantly reduces the computational resource consumption.

This paper investigates the robust tracking control of unmanned aerial vehicles (UAVs) against external time-varying disturbances. First, by introducing a virtual position controller, we innovatively decouple the UAV dynamics into independent position and attitude error subsystems, transforming the robust tracking problem into two zero-sum differential games. This approach contrasts with conventional methods by treating disturbances as strategic “players”, enabling a systematic framework to address both external disturbances and model uncertainties. Second, we develop an integral reinforcement learning (IRL) framework that approximates the optimal solution to the Hamilton–Jacobi–Isaacs (HJI) equations without relying on precise system models. This model-free strategy overcomes the limitation of traditional robust control methods that require known disturbance bounds or accurate dynamics, offering superior adaptability to complex environments. Third, the proposed recursive Ridge regression with a forgetting factor (R3F2 ) algorithm updates actor-critic-disturbance neural network (NN) weights in real time, ensuring both computational efficiency and convergence stability. Theoretical analyses rigorously prove the closed-loop system stability and algorithm convergence, which fills a gap in existing data-driven control studies lacking rigorous stability guarantees. Finally, numerical results validate that the method outperforms state-of-the-art model-based and model-free approaches in tracking accuracy and disturbance rejection, demonstrating its practical utility for engineering applications.

This article introduces a software package release for geometrically reasoning about the safety desiderata of (complex) dynamical systems via level set methods. In emphasis, safety is analyzed with the Hamilton–Jacobi equations. In scope, we provide implementations of numerical algorithms for the resolution of Hamilton–Jacobi–Isaacs equations: the spatial derivatives of the associated value function via upwinding, the Hamiltonian via Lax–Friedrichs schemes, and the integration of the Hamilton–Jacobi equation altogether via total variation diminishing Runge–Kutta schemes. Since computational speed and interoperability with other modern scientific computing libraries (typically written in the Python language) are of essence, we capitalize on modern computational frameworks such as CUPY and NUMPY and move heavy computations to GPU devices to aid parallelization and improve bring-up time in safety analysis. We hope that this package can aid users to quickly iterate on ideas and evaluate all possible safety desiderata of a system via geometrical simulation in modern engineering problems.

We investigate an infinite dimensional partial differential equation of Isaacs’ type, which arises from a zero-sum differential game between two masses. The evolution of the two masses is described by a controlled transport/continuity equation, where the control is given by the velocity vector field. Our study is set in the framework of the viscosity solutions theory in Hilbert spaces, and we prove the uniqueness of the value functions as solutions of the Isaacs equation.

The problem of order execution is cast as a relative entropy-regularized robust optimal control problem in this article. The order execution agent's goal is to maximize an objective functional associated with his profit-and-loss of trading and simultaneously minimize the inventory risk associated with market's liquidity and uncertainty. We model the market's liquidity and uncertainty by the principle of least relative entropy associated with the market trading rate. The problem of order execution is made into a relative entropy-regularized stochastic differential game. Standard argument of dynamic programming yields that the value function of the differential game satisfies a relative entropy-regularized Hamilton–Jacobi–Isaacs (rHJI) equation. Under the assumptions of linear-quadratic model with Gaussian prior, the rHJI equation reduces to a system of Riccati and linear differential equations. Further imposing constancy of the corresponding coefficients, the system of differential equations can be solved in closed form, resulting in analytical expressions for optimal strategy and trajectory as well as the posterior distribution of market trading rate. Numerical examples illustrating the optimal strategies and the comparisons with conventional trading strategies are conducted.

Sliding-mode surface-based fixed-time adaptive critic tracking control for zero-sum game of switched nonlinear systems

This paper concentrates on the fixed-time optimal consensus issue of multi-agent systems (MASs) under false data injection (FDI) attacks. To mitigate FDI attacks on sensors and actuators that may cause systems to deviate from the reference trajectory, a zero-sum game framework is established, where the secure control protocol aims at the better system performance, yet the attacker plays a contrary role. By solving the Hamilton–Jacobi–Isaacs (HJI) equation related to the zero-sum game, an optimal secure tracking controller based on the event-triggered mechanism (ETM) is obtained to decrease the consumption of system resources while the fixed-time consensus can be guaranteed. Moreover, a critic-only online reinforcement learning (RL) algorithm is proposed to approximate the optimal policy, in which the critic neural networks are constructed by the experience replay-based approach. The unmanned aerial vehicle (UAV) systems are adopted to verify the feasibility of the presented approach.

To intercept a maneuvering target with a predetermined impact angle, a computational intelligence guidance law was proposed in this paper. Based on the theory of two-player zero-sum differential games, this problem is resolved efficiently by solving the Hamilton–Jacobi–Isaacs (HJI) equation. The Nash equilibrium solution of HJI equation can be solved with a policy iteration (PI) algorithm. Instead of using the offline PI algorithm, an online PI algorithm is introduced, in which the disturbance and control policies can be updated simultaneously. It can be proved that the online PI algorithm is a replacement for Newton’s iterative algorithm, the convergence of which is ensured by Kantorovich’s theorem. In the scenario of missiles intercepting targets, an adaptive critic structure based on a neural network (NN) is proposed to implement the online PI algorithm. Only one critic NN approximator is used in the PI algorithm to calculate a value function and the approximate Nash equilibrium solution. It is not necessary to acquire the exact internal dynamics information of nonlinear systems on the basis of online data sampling. The effectiveness of the computational intelligence guidance law is proven by simulation results.

Stochastic Differential Games of Mean-Field Dynamics and Second-Order Bellman–Isaacs Equations on the Wasserstein Space

SummaryThis paper addresses the problem of a hierarchical sliding mode surface (HSMS) control design for nonlinear systems via a dynamic event‐triggered mechanism. Initially, the HSMS containing the system states is constructed to enhance the system's response rate and robustness. By assigning a cost function associated with the HSMS, such an control problem is equivalently transformed into a zero‐sum game problem, where the control policy and the exogenous disturbance are treated as two players with opposite interests. Afterwards, a novel dynamic event‐triggered mechanism is designed, where the triggering condition depends on HSMS variables. To solve the corresponding event‐triggered Hamilton–Jacobi–Isaacs equation, a single‐critic reinforcement learning algorithm is developed, which removes the error generated by approximating the actor network in the actor‐critic network. According to the Lyapunov stability theory, all signals of the considered system are strictly proved to be bounded. Finally, the validity of the proposed control method is demonstrated through simulations of a tunnel diode circuit system and a mass‐spring‐damper system.

ABSTRACTThis paper proposes a model‐free and online off‐policy algorithm based on reinforcement learning (RL) for vibration attenuation of earthquake‐excited structures, through designing an optimal controller. This design relies on solving a two‐player zero‐sum game theory with a Hamilton–Jacobi–Isaacs (HJI) equation, which is extremely difficult, or often impossible, to be solved for the value function and the related optimal controller. The proposed strategy uses an actor‐critic‐disturbance structure to learn the solution of the HJI equation online and forward in time, without requiring any knowledge of the system dynamics. In addition, the control and disturbance policies and value function are approximated by the actor, the disturbance, and the critic neural networks (NNs), respectively.Implementing the policy iteration technique, the NNs’ weights of the proposed model are calculated using the least square (LS) method in each iteration. In the present study, the convergence of the proposed algorithm is investigated through two distinct examples. Furthermore, the performance of this off‐policy RL strategy is studied in reducing the response of a seismically excited nonlinear structure with an active mass damper (AMD) for two cases of state feedback. The simulation results prove the effectiveness of the proposed algorithm in application to civil engineering structures.

ABSTRACTIn this article, a novel robust consensus control protocol is proposed for multiple uncertain Euler–Lagrange systems with unknown external disturbances. First, the local adaptive control laws are derived under event‐triggered communication framework, and the robust consensus control problem is transformed into an equivalent optimal control problem with disturbance rejection. Event‐triggered mechanisms and distributed state estimators are designed by utilizing only triggered exchanging states, by which continuous monitoring neighbors' states is avoided and communication burdens are effectively reduced. Second, the Hamilton–Jacobi–Isaac equations are formulated based on zero‐sum differential game theory, and adaptive dynamic programming methods are employed to approximate the Nash‐equivalent solutions, by which the local robust optimal control laws are derived to eliminate the disturbance effects and improve the robustness of the proposed strategy. It is strictly proved that all signals in the closed‐loop systems are uniformly ultimately bounded and the cost functions are minimized. Finally, two practical examples are presented to validate the proposed strategy.

We study discretizations of fractional fully nonlinear equations by powers of discrete Laplacians. Our problems are parabolic and of order σ∈(0,2) since they involve fractional Laplace operators (-Δ)σ/2. They arise e.g. in control and game theory as dynamic programming equations – HJB and Isaacs equation – and solutions are non-smooth in general and should be interpreted as viscosity solutions. Our approximations are realized as finite-difference quadrature approximations and are 2nd order accurate for all values of σ. The accuracy of previous approximations of fractional fully nonlinear equations depend on σ and are worse when σ is close to 2. We show that the schemes are monotone, consistent, L∞-stable, and convergent using a priori estimates, viscosity solutions theory, and the method of half-relaxed limits. We also prove a second order error bound for smooth solutions and present many numerical examples.

ABSTRACTThis article investigates a multiplayer nonzero‐sum differential game (MNDG) with switching topology and input delay. The Hamilton–Jacobi–Isaacs (HJI) equation is used to derive the optimal control strategies. To solve the issue of switching topology, the control strategies are decoupled with the communication topology. Then, the stability of the system can be achieved regardless of changes in communication topology with the designed control strategies. This does not require a restriction on the dwell time, making the system stability conditions more relaxed. The Lyapunov–Krasovskii functional (LKF) is constructed to prove the stability of the system in the presence of time‐varying input delay. To cope with the real‐world situation, the input delay is considered to be disturbed by a time‐varying perturbation in this article. With these treatments, the system is more in line with real‐world applications. Simulation examples are given to validate the efficiency of the presented methods.

Rate of Convergence for First-Order Singular Perturbation Problems: Hamilton–Jacobi–Isaacs Equations and Mean Field Games of Acceleration

This paper investigates a parameter-free H∞ differential game approach for nonlinear active vehicle suspensions. The study accounts for the geometric nonlinearity of the half-car active suspension and the cubic nonlinearity of the damping elements. The nonlinear H∞ control problem is reformulated as a zero-sum game between two players, leading to the establishment of the Hamilton–Jacobi–Isaacs (HJI) equation with a Nash equilibrium solution. To minimize reliance on model parameters during the solution process, an actor–critic framework employing neural networks is utilized to approximate the control policy and value function. An off-policy reinforcement learning method is implemented to iteratively solve the HJI equation. In this approach, the disturbance policy is derived directly from the value function, requiring only a limited amount of driving data to approximate the HJI equation’s solution. The primary innovation of this method lies in its capacity to effectively address system nonlinearities without the need for model parameters, making it particularly advantageous for practical engineering applications. Numerical simulations confirm the method’s effectiveness and applicable range. The off-policy reinforcement learning approach ensures the safety of the design process. For low-frequency road disturbances, the designed H∞ control policy enhances both ride comfort and stability.

Optimal synchronization with [formula omitted]-gain performance: An adaptive dynamic programming approach

Abstract This article addresses the optimal tracking control problem with prescribed performance for uncertain nonlinear systems subject to input constraint and unknown disturbances. First, a fixed‐time monotonic convergence function is introduced to restrain tracking error, and a nonlinear mapping technique is employed to transform the constrained error into an unconstrained variable, then the fixed‐time output tracking issue is boiled down to the boundedness problem of the transformed variable. With the aid of a nonquadratic cost function, the input constraint is encoded into the optimization problem. To solve the unknown disturbances, an auxiliary system and an auxiliary disturbance policy are constructed, and the optimal control problem is formulated as a two‐player zero‐sum game. Moreover, a Hamilton–Jacobi–Isaacs (HJI) equation associated with this nonquadratic zero‐sum game is established to give the optimal control and the worst‐case disturbance policy solution. Subsequently, to avoid using knowledge of the system dynamics, three neural network approximators, namely, actor, critic, and disturbance, which are tuned online and simultaneously for approximating the solution of HJI, are constructed based on the integral reinforcement learning algorithm. Theoretical analysis shows that the reconstructed error system states and the weight estimation errors are semi‐globally uniformly ultimately bounded. Finally, the simulation study further tests the availability of the proposed control strategy.

By utilizing a neural-network-based adaptive critic mechanism, the optimal tracking control problem is investigated for nonlinear continuous-time (CT) multiplayer zero-sum games (ZSGs) with asymmetric constraints. Initially, we build an augmented system with the tracking error system and the reference system. Moreover, a novel nonquadratic function is introduced to address asymmetric constraints. Then, we derive the tracking Hamilton-Jacobi-Isaacs (HJI) equation of the constrained nonlinear multiplayer ZSG. However, it is extremely hard to get the analytical solution to the HJI equation. Hence, an adaptive critic mechanism based on neural networks is established to estimate the optimal cost function, so as to obtain the near-optimal control policy set and the near worst disturbance policy set. In the process of neural critic learning, we only utilize one critic neural network and develop a new weight updating rule. After that, by using the Lyapunov approach, the uniform ultimate boundedness stability of the tracking error in the augmented system and the weight estimation error of the critic network is verified. Finally, two simulation examples are provided to demonstrate the efficacy of the established mechanism.

In this paper, a distributed secure change control scheme for supply chain systems is presented under denial-of-service (DoS) attacks. To eliminate the effect of DoS attacks on supply chain systems, a secure change compensation is designed. A distributed policy iteration method is established to approximate the coupled Hamilton–Jacobi–Isaacs (HJI) equations. Based on the established reinforce–critic–actor (RCA) structure using reinforcement learning (RL), the reinforced signals, performance indicators, and disturbance input are proposed to update the traditional time-triggered mechanism, and the control input is proposed to update the dynamic event-triggered mechanism (DETM). Stability is guaranteed based on the Lyapunov method under secure change control. The simulation results for supply chain systems show the effectiveness of the secure change control scheme and verify the results.