Deterministic Differential Games Research Articles

Turnpikes and computation of piecewise open-loop equilibria in stochastic differential games

This paper is concerned with the computation of Nash-Cournot equilibria for a class of piecewise deterministic differential games, where modal jumps occur randomly in time according to a Markov chain. Global asymptotic stability of equilibrium trajectories is proved and a Turnpike Adjustment Algorithm is introduced. The performance of the algorithm is illustrated on a duopoly model with random market condition, representing the competition of two firms through their capacity accumulation processes.

Iterative computation of noncooperative equilibria in nonzero-sum differential games with weakly coupled players

We study the Nash equilibria of a class of two-person nonlinear, deterministic differential games where the players are weakly coupled through the state equation and their objective functionals. The weak coupling is characterized in terms of a small perturbation parameter e. With e=0, the problem decomposes into two independent standard optimal control problems, while for e≠0, even though it is possible to derive the necessary and sufficient conditions to be satisfied by a Nash equilibrium solution, it is not always possible to construct such a solution. In this paper, we develop an iterative scheme to obtain an approximate Nash solution when e lies in a small interval around zero. Further, after requiring strong time consistency and/or robustness of the Nash equilibrium solution when at least one of the players uses dynamic information, we address the issues of existence and uniqueness of these solutions for the cases when both players use the same information, either closed loop or open loop, and when one player uses open-loop information and the other player uses closed-loop information. We also show that, even though the original problem is nonlinear, the higher (than zero) order terms in the Nash equilibria can be obtained as solutions to LQ optimal control problems or static quadratic optimization problems.

Hamilton-Jacobi equations with state constraints

In the present paper we consider Hamilton-Jacobi equations of the form H ( x , u , ∇ u ) = 0 , x ∈ Ω H(x,u,\nabla u) = 0,\;x \in \Omega , where Ω \Omega is a bounded open subset of R n , H {R^n},H is a given continuous real-valued function of ( x , s , p ) ∈ Ω × R × R n (x,s,p) \in \Omega \times R \times {R^n} and ∇ u \nabla u is the gradient of the unknown function u u . We are interested in particular solutions of the above equation which are required to be supersolutions, in a suitable weak sense, of the same equation up to the boundary of Ω \Omega . This requirement plays the role of a boundary condition. The main motivation for this kind of solution comes from deterministic optimal control and differential games problems with constraints on the state of the system, as well from related questions in constrained geodesics.

Minimizing Escape Probabilities: A large Deviations Approach

This paper considers the problem of controlling a possibly degenerate diffusion process so as to minimize the probability of escape over a given time interval. It is assumed that the control acts on the process through the drift coefficient, and that the noise coefficient is small. Developing a large deviations type of theory for the controlled diffusion produces several results. The limit of the normalized log of the minimum exit probability is identified as the value I of an associated (deterministic) differential game. Furthermore, we identify a deterministic (and $\varepsilon $-independent) mapping g from the sample values $\varepsilon w(s)$, $0 \leqq s \leqq t$, into the control space such that if we define the control used at time t by $u(t) = g(\varepsilon w(s),0 \leqq s \leqq t)$, then the resulting control process is progressively measurable and ($\delta $-optimal (in the sense that the limit of the normalized log of the exit probability is within $\delta $ of I).

Total risk aversion, stochastic optimal control, and differential games

We present a connection between the theory of risk in the context of a stochastic optimal control problem and its relation to the theory of differential games. In particular, we define the notion of “total risk aversion” from the viewpoint of the upper value of a differential game. We prove that as the index of absolute risk aversion of a utility function in a stochastic control problem converges to infinity the (certainty equivalent) optimal payoff converges to the upper value of an associated deterministic differential game. The two main points of this paper are (1) a precise characterization oftotal risk aversion and (2) the construction of a stochastic optimal control problem intimately connected to a deterministic differential game.

Prior play in a deterministic differential game

This paper discusses a two person zero sum differential game, as treated, for example, in [4-7). It is known that, if the Isaac’s condition is not satisfied, there is not a single concept of value for the differential game, and several definitions of value are given in the literature, of which the upper value and lower value are the most important. For the upper value the maximizing player has the advantage of knowing instantaneously, at each time t, the value of the control chosen by his opponent at that time. This is obtained by a non-anticipative function, which we call a strategy, from the opponent’s control functions to his own control functions. If the minimizing player knows what strategy the maximizing player will employ he can determine the optimal control function that he should use. We see that, in this formulation, the maximizing player should choose his strategy first, and the questions investigated in this paper are the properties of optimal, or almost optimal, strategies. Even in the stochastic case, where things are sometimes smoother, and which was investigated in 131, the calculations are very delicate. The paper proceeds by first showing that the set of strategies is a complete metric space under a certain metric, and by then applying the non-convex minimization result of Ekeland 121. The dynamic programming result of 161 is next recalled. By declaring his strategy in advance the maximizing player is certainly giving his opponent information about his future behaviour, but, by using the dynamic programming identities, games played over shorter and shorter time intervals are considered. In these, the maximizing player gives less and less information about his future play. If the dynamics and payoff satisfy Lipschitz conditions, the upper value function is differentiable almost everywhere. At points of differentiability it is shown that an e-optimal strategy should, on average, almost maximize the Hamiltonian.

On the cheating problem in Stackelberg games†

Abstract This paper deals with the problem of cheating in two-person games. First we discuss different ways of cheating and give examples of practical economic game problems where cheating is a realistic policy alternative. We consider the Stackelberg setting where the leader makes his decision first and announces this to the follower. When the follower does not know the performance criterion of the leader then the leader can cheat by making a false announcement to the follower. Three different alternatives for cheating are described. Necessary conditions for the open-loop cheating strategies in deterministic linear-quadratic differential games are developed. One of the cheating formulations results in a three stage dynamic optimization problem. The solution of the TPBVP obtained in this case is considered in detail. The feasibility of obtaining the cheating solutions is demonstrated by solving a simple pursuit-evasion game.

Stochastic programmed design for a deterministic positional differential game

It is shown that under specific sufficiently general conditions the value of a positional differential game can be found from auxiliary programmed constructions which include a suitable random process. The paper is a continuation of the researches in /1–12/.

Sufficient conditions for Stackelberg and Nash strategies with memory

Sufficiency conditions for Stackelberg strategies for a class of deterministic differential games are derived when the players have recall of the previous trajectory. Sufficient conditions for Nash strategies when the players have recall of the trajectory are also derived. The state equation is linear, and the cost functional is quadratic. The admissible strategies are restricted to be affine in the information available.

Finite Energy Approximations in Stochastic and Deterministic Differential Games

By penalizing the players in a differential game of fixed duration for large energy use the game will always have (i) a value (ii) a saddle point in pure strategies and (iii) a well-formulated Hamilton–Jacobi equation. Furthermore, under certain regularity conditions, the saddle points can be characterized as the solution to a system of ordinary differential equations. The case where the control functions can be any measurable function have none of the properties (i)–(iii) above and are difficult to solve. However, by considering our finite energy games we can approximate the measureable case. It also seems more realistic to have an “inertial effect” in the controls. We consider both deterministic and stochastic versions.

Feedback strategies with finite memory in differential games

It is shown that, in differential games, strategies can be defined for the players in such a way that their controls depend, for each timet, on a finite section of the past of the trajectoryx(t). In particular,s-delay,r-memory strategies can be defined as in the Varaiya-Lin approach. It is shown that, for deterministic differential games with terminal payoff, the upper and lower values of the game so defined, are independent of the lengthr of the memory. Lettingr → 0, a feedback strategy is obtained which depends only on the present (and infinitesimal past) of the trajectoryx(t).

Differential‐game examination of optimal time‐sequential fire‐support strategies

AbstractOptimal time‐sequential fire‐support strategies are studied through a two‐person zero‐sum deterministic differential game with closed‐loop (or feedback) strategies. Lanchester‐type equations of warfare are used in this work. In addition to the max‐min principle, the theory of singular extremals is required to solve this prescribed‐duration combat problem. The combat is between two heterogeneous forces, each composed of infantry and a supporting weapon system (artillery). In contrast to previous work reported in the literature, the attrition structure of the problem at hand leads to force‐level‐dependent optimal fire‐support strategies with the attacker's optimal fire‐support strategy requiring him to sometimes split his artillery fire between enemy infantry and artillery (counterbattery fire). A solution phenomnon not previously encountered in Lanchester‐type differential games is that the adjoint variables may be discontinuous across a manifold of discontinuity for both players' strategies. This makes the synthesis of optimal strategies particularly difficult. Numerical examples are given.

Informationally Nonunique Equilibrium Solutions in Differential Games

This paper is concerned with two-person deterministic nonzero-sum differential games (NZSDG) characterized by quadratic objective functionals and with state dynamics described by linear differential equations. It is first shown that such games admit uncountably many (informationally nonunique) noncooperative (Nash) equilibrium solutions when at least one of the players has access to dynamic information. We provide a characterization of all Nash equilibrium solutions to the problem for a particular dynamic information pattern, and propose an optimal unique selection of an element of the Nash equilibrium set, which exhibits a robust behavior by being insensitive to additive random perturbations in the state dynamics. We model these random perturbations as a local martingale process and obtain the abovementioned optimal Nash strategy pair as the unique noncooperative equilibrium solution of a related stochastic NZSDG. With regard to the latter, it is shown that the unique Nash equilibrium strategy of the player with dynamic closed-loop information can be realized by affine control laws.

Necessary conditions of optimality for a differential game with bounded state variables

First-order necessary conditions of optimality are examined for a two-person zero-sum deterministic differential game (an extension of Isaacs' War of Attrition and Attack [4]) with bounded state variables. Both players use closed-loop strategies. Multiplier conditions associated with the state constraints are shown to play a key role in the determination of optimal strategies.

On open-loop and closed-loop nash strategies

For deterministic nonzero-sum differential games, every open-loop Nash strategy set is a closed-loop Nash strategy set. The converse, that the open-loop strategy set generated by a closed-loop Nash strategy set is Nash, is false.

Optimal stochastic linear systems with exponential performance criteria and their relation to deterministic differential games

Two stochastic optimal control problems are solved whose performance criteria are the expected values of exponential functions of quadratic forms. The optimal controller is linear in both cases but depends upon the covariance matrix of the additive process noise so that the certainty equivalence principle does not hold. The controllers are shown to be equivalent to those obtained by solving a cooperative and a noncooperative quadratic (differential) game, and this leads to some interesting interpretations and observations. Finally, some stability properties of the asymptotic controllers are discussed.

Contraction Mappings in the Theory Underlying Dynamic Programming

Contraction Mappings in the Theory Underlying Dynamic Programming

The Economics of Exhaustible Resources

The discussion is confined in scope to absolutely irreplaceable assets. Topics include peculiar problems of mineral wealth, free competition, maximum social value and state regulation, monopoly, value of a mine monopoly, retardation of production under monopoly, price effects from cumulated production, and the author's mathematically derived optimum solutions. (PCS)