Open-Loop Nash Equilibrium of Two-Person Non-zero-sum Differential Games with Time Inconsistency

The relationship amongst state-redundancy and time consistency of differential games is investigated. A class of state-redundant games is detected, where the state dynamics and the payoff functions of all players are additively separable w.r.t. control variables. We prove that, in this class of games, open-loop Nash and degenerate feedback Stackelberg equilibria coincide, both being subgame perfect. This allows us to bypass the issue of the time inconsistency that typically affects the open-loop Stackelberg solution.

Differential games with time-inconsistent preferences are studied. Noncooperative Markovian Nash equilibria are described. If players can cooperate at every instant of time, time-consistent equilibria are analyzed for the problem with partial cooperation. Cooperation is partial in the sense that, although players cooperate at every moment t forming a coalition, due to the time inconsistency of the time preferences, coalitions at different times value the future in a different way, and they are treated as different agents. Time-consistent equilibria are obtained by looking for the Markovian subgame perfect equilibria in the corresponding noncooperative sequential game. The issue of dynamic consistency is then considered. In order to guarantee the sustainability of cooperation, players should bargain at every instant of time their weight in the whole coalition, and nonconstant weights are introduced. The results are illustrated with two examples: a common property resource game and a linear state pollution differential game.

I. Zero-sum differential games: Theory and applications in worst-case controller design.- A Theory of Differential Games.- H?-Optimal Control of Singularly Perturbed Systems with Sampled-State Measurements.- New Results on Nonlinear H? Control Via Measurement Feedback.- Reentry Trajectory Optimization under Atmospheric Uncertainty as a Differential Game.- II. Zero-sum differential games: Pursuit-evasion games and numerical schemes.- Fully Discrete Schemes for the Value Function of Pursuit-Evasion Games.- Zero Sum Differential Games with Stopping Times: Some Results about its Numerical Resolution.- Singular Paths in Differential Games with Simple Motion.- The Circular Wall Pursuit.- III. Mathematical programming techniques.- Decomposition of Multi-Player Linear Programs.- Convergent Stepsizes for Constrained Min-Max Algorithms.- Algorithms for the Solution of a Large-Scale Single-Controller Stochastic Game.- IV. Stochastic games: Differential, sequential and Markov Games.- Stochastic Games with Average Cost Constraints.- Stationary Equilibria for Nonzero-Sum Average Payoff Ergodic Stochastic Games and General State Space.- Overtaking Equilibria for Switching Regulator and Tracking Games.- Monotonicity of Optimal Policies in a Zero Sum Game: A Flow Control Model.- V. Applications.- Capital Accumulation Subject to Pollution Control: A Differential Game with a Feedback Nash Equilibrium.- Coastal States and Distant Water Fleets Under Extended Jurisdiction: The Search for Optimal Incentive Schemes.- Stabilizing Management and Structural Development of Open-Access Fisheries.- The Non-Uniqueness of Markovian Strategy Equilibrium: The Case of Continuous Time Models for Non-Renewable Resources.- An Evolutionary Game Theory for Differential Equation Models with Reference to Ecosystem Management.- On Barter Contracts in Electricity Exchange.- Preventing Minority Disenfranchisement Through Dynamic Bayesian Reapportionment of Legislative Voting Power.- Learning by Doing and Technology Sharing in Asymmetric Duopolies.

The preceding chapter dealt with differential games in which all players make their moves simultaneously. We now turn to a class of differential games in which some players have priority of moves over other players. To simplify matters, we focus mostly on the case where there are only two players. The player who has the right to move first is called the leader and the other player is called the follower. A well-known example of this type of hierarchical-moves games is the Stackelberg model of duopoly, which is often contrasted with the Cournot model of duopoly.

Robust Markov Perfect Equilibria in a Dynamic Choice Model with Quasi-hyperbolic Discounting.- Stochastic Differential Games and Intricacy of Information Structures.- Policy Interactions in a Monetary Union: An Application of the OPTGAME Algorithm.- The Dynamics of Lobbying Under Uncertainty: On Political Liberalization in Arab Countries.- A Feedback Stackelberg Game of Cooperative Advertising in a Durable Goods Oligopoly.- Strategies of Foreign Direct Investment in the Presence of Technological Spillovers.- Differential Games and Environmental Economics.- Capacity Accumulation Games with Technology Constraints.- Dynamic Analysis of an Electoral Campaign.- Multi-Agent Optimal Control Problems and Variational Inequality Based Reformulations.- Time-consistent Equilibria in a Differential Game Model with Time Inconsistent Preferences and Partial Cooperation.- Interactions Between Fiscal and Monetary Authorities in a Three-Country New-Keynesian Model of a Monetary Union.- Subgame Consistent Cooperative Provision of Public Goods Under Accumulation and Payoff Uncertainties.

In this chapter, a new optimal coordination control for the consensus problem of heterogeneous multi-agent differential graphical games by iterative ADP is developed. The main idea is to use iterative ADP technique to obtain the iterative control law which makes all the agents track a given dynamics and simultaneously makes the iterative performance index function reach the Nash equilibrium. In the developed heterogeneous multi-agent differential graphical games, the agent of each node is different from the one of other nodes. The dynamics and performance index function for each node depend only on local neighbor information. A cooperative policy iteration algorithm for graphical differential games is developed to achieve the optimal control law for the agent of each node, where the coupled Hamilton–Jacobi (HJ) equations for optimal coordination control of heterogeneous multi-agent differential games can be avoided. Convergence analysis is developed to show that the performance index functions of heterogeneous multi-agent differential graphical games can converge to the Nash equilibrium. Simulation results will show the effectiveness of the developed optimal control scheme.

A nonstop stream of publications is devoted to the investigation of positive and negative properties of Nash equilibrium concept prevailing in economics (as solution of noncooperative game). Mostly they are related to non-uniqueness and, as a consequence, to the lack of equivalence, interchangeability, external stability as well as instability to simultaneous deviation of such solutions of two and more players. The game "dilemma of prisoners" also revealed the property of "ability to improve". The book Equilibrium Control of Multi-criteria Dynamic Problems (V.I. Zhukovskiy and N.T. Tynyanskiy, M.: MSU, 1984) is devoted to detailed analysis of such "negative" properties for differential positional games. The authors of this book com to the following conclusion: either make use of those situations of Nash equilibrium that are simultaneously free from some of the stated disadvantages, or introduce new solutions of noncooperative game. Such solutions having the merits of Nash equilibrium situation would allow to get rid of its certain disadvantages. The present article is devoted to one of such possibilities for differential games related to concepts of objections and counterobjections. The concepts of objections and counterobjections used in it are based on the concepts of objections and counterobjections well-known classical game theory. The papers of E.I. Wilkas [1973] are devoted to theoretical questions of this concept. The term "active equilibrium" suggested R.E. Smolyakov in 1983, the notion of equilibrium of objections and counterobjections in differential games was first used apparently by E.M. Vaisbord in 1974, and then it was picked up by the first author of the present article in the above mentioned book [1984], but this concept was applied and is being applied in differential games, in our opinion, insufficiently widely. This fact "called to life" the present paper. In it the class of differential games of two persons is revealed, where the usual Nash equilibrium situation is absent, but the equilibrium of objections and counterobjections is present

A fuzzy differential game theory is proposed to solve the n-person (or n-player) nonlinear differential noncooperative game and cooperative game (team) problems, which are not easily tackled by the conventional methods. In the paper, both noncooperative and cooperative quadratic differential games are considered. First, the nonlinear stochastic system is approximated by a fuzzy model. Based on the fuzzy model, a fuzzy controller is proposed to deal with the noncooperative differential game in the sense of Nash equilibrium strategies or with the cooperative game in the sense of Pareto-optimal strategies. Using a suboptimal approach, the outcomes of the fuzzy differential games for both the noncooperative and the cooperative cases are parameterized in terms of an eigenvalue problem. Since the state variables are usually unavailable, a suboptimal fuzzy observer is also proposed in this study to estimate the states for these differential game problems. Finally, simulation examples are given to illustrate the design procedures and to indicate the performance of the proposed methods.

A linear-quadratic positional differential game of N persons is considered. The solution of a game in the form of Nash equilibrium has become widespread in the theory of noncooperative differential games. However, Nash equilibrium can be internally and externally unstable, which is a negative in its practical use. The consequences of such instability could be avoided by using Pareto maximality in a Nash equilibrium situation. But such a coincidence is rather an exotic phenomenon (at least we are aware of only three cases of such coincidence). For this reason, it is proposed to consider the equilibrium of objections and counterobjections. This article establishes the coefficient criteria under which in a differential positional linear-quadratic game of N persons there is Pareto equilibrium of objections and counterobjections and at the same time no Nash equilibrium situation; an explicit form of the solution of the game is obtained.

This paper studies an overlapping generations (OLG) differential game on optimal emissions with continuous age structure and different types of individuals. At the (stochastic) arrival of a catastrophic climate change, the utility and the damage to the stock of pollution change for the rest of the time horizon. We derive the open-loop (OL) Nash equilibrium and show that it is subgame perfect and moreover equal to the feedback Stackelberg one. We compare the solution to the cooperative one (using the social welfare as objective function) and show the different dynamic evolutions of optimal emissions over time. Finally, we derive a time-consistent tax scheme that reaches the cooperative optimal solution in the OL Nash equilibrium. The tax scheme turns out to be heterogeneous with respect to age and type (anticipating and nonanticipating the catastrophic climate change). Setting taxes that are homogeneous across the individual type leads to an OL Nash solution that produces socially optimal total emissions, but lower individual utilities.

THIS paper gives an overview of the various equilibrium concepts used in non-cooperative difference games and their economic interpretation. Difference games are dynamic games in discrete time. The state of the economy at time t, say yt depends on the state of the economy at time t 1, yt1, and on the actions of the various players undertaken during this period. (Differential games are dynamic games in continuous time.) Difference games are unlike repeated games (supergames), because the latter refer to the repetition of a static game where the state of the economy in each game is independent of the state of the economy in previous games. History in repeated games matters only because players might condition their strategies on the history of play, but history in difference games matters also due to the dynamics of capital accumulation, wages, prices, etc. To illustrate the various concepts employed in difference games, it is useful to discuss a classic example where actions can take on only one of two values. Figure 1 gives a simple example of such a dynamic game (due to Simaan and Cruz (1973)). The economy starts off in the state yo = 0. Subsequently each player can either take the action L or H. Each player minimizes a welfare loss function, which is time separable. The welfare losses incurred during the transition from the state at time 0 to the state at time 1 are given above the actions. From each state at time 1, each player can again take the actions L or H and at time 2 arrive at four possible states. The Nash solution concept represents the standard approach to non-cooperative games and is applicable when both players have equal strength. The actions in a Nash equilibrium must be the best response of player 1 to the action of player 2 and the best response of player 2 to the action of player 1. In dynamic games one distinguishes between the open-loop Nash equilibrium and the feedback Nash or subgameperfect equilibrium. Two assumptions distinguish the feedback concept from the open-loop concept, namely information structure (Basar and Olsder (1982)) and period of commitment (Reinganum and Stokey (1985)). The open-loop Nash equilibrium presumes that the players at time 1 and 2 can only observe the initial state of the economy, yo, i.e. have open-loop information patterns. The open-loop Nash equilibrium also presumes that at time 0 each player can make binding commitments about the actions he or she announces to undertake

In differential games, open-loop Nash equilibrium and feedback Nash equilibrium (FNE) are, in general, different. Only in the case of a degenerate FNE do these equilibria coincide. So far, only a countable number of games with a degenerate FNE or a non-degenerate FNE have been found. In this paper, the author shows that (i) a class of games with a degenerate FNE can be constructed from every game which has a non-degenerate FNE, and (ii) a class of games with a non-degenerate FNE can be constructed from every game which has a degenerate FNE. Hence,for any differential game with a degenerate (non-degenerate) FNE, there is a corresponding class of games which have a non-degenerate (degenerate) FNE. >

The purpose of this chapter is to develop a theory for stochastic linear-quadratic two-person differential games. Open-loop and closed-loop Nash equilibria are explored in the context of nonzero-sum and zero-sum differential games. The existence of an open-loop Nash equilibrium is characterized in terms of a system of constrained forward-backward stochastic differential equations, and the existence of a closed-loop Nash equilibrium is characterized by the solvability of a system of coupled symmetric differential Riccati equations. It is shown that in the nonzero-sum case, the closed-loop representation for open-loop Nash equilibria is different from the outcome of closed-loop Nash equilibria in general, whereas they coincide in the zero-sum case when both exist. Some results for infinite-horizon zero-sum differential games are also established in terms of algebraic Riccati equation.

Nash equilibria in nonzero-sum differential games with impulse control

Correlated relaxed equilibria in nonzero-sum linear differential games