Zero-sum Differential Game Research Articles

Linear Quadratic Stochastic Differential Games: Open-Loop and Closed-Loop Saddle Points

In this paper, we consider a linear quadratic stochastic two-person zero-sum differential game. The controls for both players are allowed to appear in both drift and diffusion of the state equation. The weighting matrices in the performance functional are not assumed to be definite/non-singular. The existence of an open-loop saddle point is characterized by the existence of an adapted solution to a linear forward-backward stochastic differential equation with constraints, together with a convexity-concavity condition, and the existence of a closed-loop saddle point is characterized by the existence of a regular solution to a Riccati differential equation. It turns out that there is a significant difference between open-loop and closed-loop saddle points. Also, it is found that there is an essential feature that prevents a linear quadratic optimal control problem from being a special case of linear quadratic two-person zero-sum differential games.

Online Partially Model-Free Solution of Two-Player Zero Sum Differential Games

An online adaptive dynamic programming based iterative algorithm is proposed for a two-player zero sum linear differential game problem arising in the control of process systems affected by disturbances. The objective in such a scenario is to obtain an optimal control policy that minimizes the specified performance index or cost function in presence of worst case disturbance. Conventional algorithms for the solution of such problems require full knowledge of system dynamics. The algorithm proposed in this paper is partially model-free and solves the two-player zero sum linear differential game problem without knowledge of state and control input matrices.

Robust trajectory tracking: differential game/cheap control approach

A robust trajectory tracking problem is treated in the framework of a zero-sum linear-quadratic differential game of a general type. For the cheap control version of this game, a novel solvability condition is derived. The sufficient condition, guaranteeing that the tracking problem is solved by the optimal strategy of the minimiser in the cheap control game, is established. The boundedness of the time realisations of this strategy is analysed. An illustrative example is presented.

Stackelberg Solutions of Differential Games in the Class of Nonanticipative Strategies

The reverse Stackelberg solution of a two-person nonzero-sum differential game is considered. We assume that the leader plays in the class of nonanticipative strategies. The main result is the description of the Stackelberg solutions via an auxiliary zero-sum differential game. The case when the leader’s strategies depend on the actual control of the follower is compared with the case when the leader uses nonanticipative strategies.

Hamilton–Jacobi equations and two-person zero-sum differential games with unbounded controls

A two-person zero-sum differential game with unbounded controls is considered. Under proper coercivity conditions, the upper and lower value functions are characterized as the unique viscosity solutions to the corresponding upper and lower Hamilton–Jacobi–Isaacs equations, respectively. Consequently, when the Isaacs’ condition is satisfied, the upper and lower value functions coincide, leading to the existence of the value function of the differential game. Due to the unboundedness of the controls, the corresponding upper and lower Hamiltonians grow super linearly in the gradient of the upper and lower value functions, respectively. A uniqueness theorem of viscosity solution to Hamilton–Jacobi equations involving such kind of Hamiltonian is proved, without relying on the convexity/concavity of the Hamiltonian. Also, it is shown that the assumed coercivity conditions guaranteeing the finiteness of the upper and lower value functions are sharp in some sense.

Game theory approach to state and input simultaneous estimation for discrete-time LTV systems

This article aims to provide a simple approach realising state observation and input estimation simultaneously for discrete-time LTV systems in H∞ setting. Through solving a two-player zero sum differential game, appealing results are obtained in two folds. First, necessary and sufficient solvability conditions for state and input simultaneous estimation problem are given in terms of solution to a set of difference Riccati recursion. Second, one estimator is presented with special innovation structure, where innovation information is used to update state observation tuned by gain matrix and to provide input estimation through a projector matrix, where gain matrix and projector matrix are constructed from solution to difference Riccati recursion. At last, simulation results are provided to justify proposed approach.

On Zero-Sum Stochastic Differential Games with Jump-Diffusion Driven State: A Viscosity Solution Framework

A zero-sum differential game with controlled jump-diffusion driven state is considered and studied by a combination of dynamic programming and viscosity solution techniques. We prove, under certain conditions, that the value of the game exists. Moreover, the value function is shown to be the unique viscosity solution of a fully nonlinear integro-partial differential equation. In addition, we formulate and prove a verification theorem for such games within the viscosity solution framework for nonlocal equations.

On Numerical Solving of Differential Games with Nonterminal Payoff

A linear-convex zero-sum differential game with a nonterminal payoff in the form of the Euclidean norm of a set of deviations of the motion trajectory from given targets at given instants of time is considered. A numerical method of approximate computation of the game value and optimal control laws is elaborated. Details of the software implementation are discussed. Results of numerical simulations are given.

History-dependent modified sliding mode interception strategies with maximal capture zone

In order to construct the guidance strategy in a realistic nonlinear noise-corrupted interception endgame against a maneuverable target, a linearized zero-sum differential game is considered. Assuming perfect information in this game, sufficient conditions are established, which guarantee that a continuous interception strategy with memory (history-dependent) has the maximal capture zone. Two examples of such a strategy are analyzed: a modified super-twisting second-order sliding mode control and a modified integral sliding mode control. Simulation results of the original nonlinear interception endgame demonstrate that these strategies considerably reduce the chattering created by the classical game optimal bang-bang strategy without deteriorating the homing performance.

Adaptive dynamic programming for online solution of a zero-sum differential game

This paper will present an approximate/adaptive dynamic programming (ADP) algorithm, that uses the idea of integral reinforcement learning (IRL), to determine online the Nash equilibrium solution for the two-player zerosum differential game with linear dynamics and infinite horizon quadratic cost. The algorithm is built around an iterative method that has been developed in the control engineering community for solving the continuous-time game algebraic Riccati equation (CT-GARE), which underlies the game problem. We here show how the ADP techniques will enhance the capabilities of the offline method allowing an online solution without the requirement of complete knowledge of the system dynamics. The feasibility of the ADP scheme is demonstrated in simulation for a power system control application. The adaptation goal is the best control policy that will face in an optimal manner the highest load disturbance.

Numerical Solution of Orbital Combat Games Involving Missiles and Spacecraft

This research addresses the problem of the optimal interception of an optimally evasive orbital target by a pursuing spacecraft or missile. The time for interception is to be minimized by the pursuing space vehicle and maximized by the evading target. This problem is modeled as a two-sided optimization problem, i.e. as a two-player zero-sum differential game. The work incorporates a recently developed method, termed “semi-direct collocation with nonlinear programming”, for the numerical solution of dynamic games. The method is based on the formal conversion of the two-sided optimization problem into a single-objective one, by employing the analytical necessary conditions for optimality related to one of the two players. An approximate, first attempt solution for the method is provided through the use of a genetic algorithm in a “preprocessing” phase. Three qualitatively different cases are considered. In the first example the pursuer and the evader are represented by two spacecraft orbiting the Earth in two distinct orbits. The second and the third case involve two missiles, and a missile that pursues an orbiting spacecraft, respectively. The numerical results achieved in this work testify to the robustness and effectiveness of the method also in solving large, complex, three-dimensional problems.

Minimax algorithm for constructing an optimal control strategy in differential games with a Lipschitz payoff

For a zero-sum differential game, an algorithm is proposed for computing the value of the game and constructing optimal control strategies with the help of stepwise minimax. It is assumed that the dynamics can be nonlinear and the cost functional of the game is the sum of an integral term and a terminal payoff function that satisfies the Lipschitz condition but can be neither convex nor concave. The players’ controls are chosen from given sets that are generally time-dependent and unbounded. An error estimate for the algorithm is obtained depending on the number of partition points in the time interval and on the fineness of the spatial triangulation. Numerical results for an illustrative example are presented.

An iterative adaptive dynamic programming method for solving a class of nonlinear zero-sum differential games

In this paper, a new iterative adaptive dynamic programming (ADP) method is proposed to solve a class of continuous-time nonlinear two-person zero-sum differential games. The idea is to use the ADP technique to obtain the optimal control pair iteratively which makes the performance index function reach the saddle point of the zero-sum differential games. If the saddle point does not exist, the mixed optimal control pair is obtained to make the performance index function reach the mixed optimum. Stability analysis of the nonlinear systems is presented and the convergence property of the performance index function is also proved. Two simulation examples are given to illustrate the performance of the proposed method.

Some Recent Aspects of Differential Game Theory

This survey paper presents some new advances in theoretical aspects of differential game theory. We particular focus on three topics: differential games with state constraints; backward stochastic differential equations approach to stochastic differential games; differential games with incomplete information. We also address some recent development in nonzero-sum differential games (analysis of systems of Hamilton–Jacobi equations by conservation laws methods; differential games with a large number of players, i.e., mean-field games) and long-time average of zero-sum differential games.

Numerical construction of Nash and Stackelberg solutions in a two-player linear non-zero-sum positional differential game

Numerical methods are proposed for constructing Nash and Stackelberg solutions in a two-player linear non-zero-sum positional differential game with terminal cost functionals and geometric constraints on the players’ controls. The formalization of the players’ strategies and of the motions generated by them is based on the formalization and results from the theory of positional zero-sum differential games developed by N.N. Krasovskii and his school. It is assumed that the game is reduced to a planar game and the constraints on the players’ controls are given in the form of convex polygons. The problem of finding solutions of the game may be reduced to solving nonstandard optimal control problems. Several computational geometry algorithms are used to construct approximate trajectories in these problems, in particular, algorithms for constructing the convex hull as well as the union, intersection, and algebraic sum of polygons.

PARAMETER ANALYSIS OF A ZERO-SUM DIFFERENTIAL GAME APPROXIMATING A NONZERO-SUM GAME

In this paper we consider a zero-sum differential game in feedback setting with one state coordinate and finite time horizon. The problem is characterized by two parameters. This game arises as an approximation to a nonzero-sum game. Using analytical and numerical methods we solve the HJBI equation and give the description of the optimal feedback controls for both players. These feedbacks are given in terms of switching curves and singular universal lines. We also determine the change (bifurcations) of the optimal phase portrait of the game depending upon the parameters. In conclusion, we show that for some values of the parameters the solution of the considered zero-sum game leads to the exact solution of the corresponding nonzero-sum game, and for the other values of the parameters it supplies just some approximation.

Hybrid Estimation of State and Input for Linear Discrete Time-varying Systems: A Game Theory Approach

The H ∞ hybrid estimation problem for linear discrete time-varying systems is investigated in this paper, where estimated signals are linear combination of state and input. Design objective requires the worst-case energy gain from disturbance to estimation error to be less than a prescribed level. Optimal solution of the hybrid estimation problem is the saddle point of a two-player zero sum differential game. On the basis of the differential game approach, necessary and sufficient solvable conditions for the hybrid estimation problem are provided in terms of solutions to a Riccati differential equation. Moreover, one possible estimator is proposed if the solvable conditions are satisfied. The estimator is characterized by a gain matrix and an output mapping matrix, where the latter reflects the internal relations between unknown input and output estimation error. At last, a numerical example is provided to illustrate the proposed approach.

Representations Formulas for Some Differential Games with Asymmetric Information

We compute the value of several two-player zero-sum differential games in which the players have an asymmetric information on the random terminal payoff.

DETERMINISTIC DIFFERENTIAL GAMES UNDER PROBABILITY KNOWLEDGE OF INITIAL CONDITION

We study a zero-sum differential game where the players have only an unperfect information on the state of the system. In the beginning of the game only a random distribution on the initial state is available. The main result of the paper is the existence of the value obtained through an uniqueness result for Hamilton-Jacobi-Isaacs equations stated on the space of measure in ℝn. This result is the first step for future work on differential games with lack of information.

Subsolutions of an Isaacs Equation and Efficient Schemes for Importance Sampling

It was established in Dupuis and Wang [Dupuis, P., H. Wang. 2004. Importance sampling, large deviations, and differential games. Stoch. Stoch. Rep. 76 481–508, Dupuis, P., H. Wang. 2005. Dynamic importance sampling for uniformly recurrent Markov chains. Ann. Appl. Probab. 15 1–38] that importance sampling algorithms for estimating rare-event probabilities are intimately connected with two-person zero-sum differential games and the associated Isaacs equation. This game interpretation shows that dynamic or state-dependent schemes are needed in order to attain asymptotic optimality in a general setting. The purpose of the present paper is to show that classical subsolutions of the Isaacs equation can be used as a basic and flexible tool for the construction and analysis of efficient dynamic importance sampling schemes. There are two main contributions. The first is a basic theoretical result characterizing the asymptotic performance of importance sampling estimators based on subsolutions. The second is an explicit method for constructing classical subsolutions as a mollification of piecewise affine functions. Numerical examples are included for illustration and to demonstrate that simple, nearly asymptotically optimal importance sampling schemes can be obtained for a variety of problems via the subsolution approach.