Open-loop Nash Equilibrium Research Articles

Open-loop and closed-loop Nash equilibria for the LQ stochastic difference game

This paper investigates the conditions for the existence of open-loop and closed-loop Nash equilibria for the linear quadratic (LQ) two-player stochastic difference game. It is shown that the former is characterized by the solvability of a forward–backward stochastic difference equation (FBSΔE), while the latter is described by the solvability of a class of coupled symmetric difference Riccati equations (ΔREs). Different from the existing works, the relationship between the closed-loop representation of open-loop Nash equilibrium and the closed-loop Nash equilibrium has been studied. It is found that the closed-loop representation of open-loop Nash equilibrium is different from the outcome of closed-loop Nash equilibrium for the general case. However, for the LQ zero-sum stochastic difference game, they are consistent.

Game-Theoretic Inverse Reinforcement Learning: A Differential Pontryagin's Maximum Principle Approach.

This brief proposes a game-theoretic inverse reinforcement learning (GT-IRL) framework, which aims to learn the parameters in both the dynamic system and individual cost function of multistage games from demonstrated trajectories. Different from the probabilistic approaches in computer science community and residual minimization solutions in control community, our framework addresses the problem in a deterministic setting by differentiating Pontryagin's maximum principle (PMP) equations of open-loop Nash equilibrium (OLNE), which is inspired by Jin et al. (2020). The differentiated equations for a multi-player nonzero-sum multistage game are shown to be equivalent to the PMP equations for another affine-quadratic nonzero-sum multistage game and can be solved by some explicit recursions. A similar result is established for two-player zero-sum games. Simulation examples are presented to demonstrate the effectiveness of our proposed algorithms.

Existence and generic stability of open-loop Nash equilibria for noncooperative fuzzy differential games

Differential game is a critical area of study not only in theory but also in application. Based on the concept of the fuzzy process introduced by Liu, a fuzzy differential game model described using fuzzy differential equations is studied. Firstly, the existence theorem of open-loop Nash equilibrium for fuzzy differential games is given using Ky Fan inequality, and an example shows the applicability of the theorem. Secondly, the fuzzy differential game space Γ1 is constructed, and the stability of open-loop Nash equilibria of the fuzzy differential game γ∈Γ1 is studied. The conclusion shows that the fuzzy differential games whose all the open-loop Nash equilibria are stable form a dense residual set, i.e., most of the fuzzy differential games are stable in the sense of Baire classification.

Economically exhaustible resources in an oligopoly-fringe model with renewables

We consider a game between oligopolistic and fringe suppliers of fossil fuel from an exhaustible resource, and producers of a renewable perfect substitute. Extraction costs are stock-dependent and strictly convex in the rate of extraction. We characterize the open-loop Nash equilibrium analytically and perform numerical simulations with calibrated parameter values. The effects of our cost assumptions are (i) to have asymptotic economical instead of physical exhaustion of the non-renewable resource and (ii) the existence of a limit-pricing phase in which both fossil and renewables suppliers are active. We decompose the welfare loss of imperfect competition in a conservation and a sequence effect, and show that both can be substantial: 3.8 and 4.2 trillion US$ in the calibrated model, respectively. We also examine Green Paradox effects and find that initial carbon emissions depend non-monotonically on the renewables subsidy rate.

Power Control and Interference Pricing Strategies for Heterogeneous Networks With Renewable Energy Sources

In this paper, the power control and interference pricing problems between the macro base station (MBS) and small cell base stations (SBSs) in Heterogeneous networks (HetNets) are investigated. To optimally allocate power resources and effectively manage the power resources to reduce interference in HetNets, an innovative Stackelberg differential game approach is proposed. In the proposed Stackelberg game, the SBSs control their transmit power to achieve cost minimization based on the interference price offered by the MBS. Given the optimal power control strategies of SBSs, the MBS can adjust the interference price to realize the objective optimization. Renewable energy sources are introduced into the proposed HetNets, and differential equations formulate the SBSs' dynamic energy. Meanwhile, the MBS's state is given by the dynamic spectrum resources. Then the differential game is imported to achieve optimal power control and interference pricing. The open-loop Nash equilibrium solutions are first analyzed for the MBS and SBSs. Then the feedback Nash equilibrium solutions are given. Under the feedback pattern, the proposed model's performances in the non-cooperative and cooperative situations are discussed. Numerical simulations are given to show the convergence and effectiveness of the proposed game approach.

Mixed Market Structure and R &D: A Differential Game Approach

We consider a dynamic model of an industry consisting of a few large firms, which can manipulate the market outcome, and a mass of small enterprises, each of which has a negligible impact on the market. The production costs of the respective firms depend on the stock of knowledge capital, which accumulates over time through research and development (R &D) investment made by large firms. The model is a variant of the differential game of voluntary provision of public goods, but in contrast to previous studies, we focus on the interaction between market competition and dynamic game outcomes. We derive both open-loop and Markov-perfect Nash equilibria. There exists a unique open-loop Nash equilibrium. By contrast, depending on the parameters of the model, there can be two linear Markov-perfect Nash equilibria. We also examine the short- and long-run effects of a change in the number of large firms. An increase in the number of large firms unambiguously harms both types of firms in the short run but may benefit them in the long run. In the open-loop Nash equilibrium, the relationship between the number of large firms and the steady-state stock of knowledge capital is inverted-U shaped. Concerning the Markov-perfect Nash equilibria, the effect of increased competition from large firms depends on the specific feedback strategy chosen in equilibrium.

Linear-Quadratic Non-Zero Sum Backward Stochastic Differential Game With Overlapping Information

This article studies a linear quadratic non-zero sum stochastic differential game with overlapping information, where the state dynamics are described by a backward stochastic differential equation and the information obtained by two players has a common part but no inclusion relation. The open-loop Nash equilibrium strategy is given by some conditional mean-field stochastic differential equations. In addition, coupled Riccati equations are introduced to express the state feedback form of the Nash equilibrium strategy.

Approximate Solutions for Nash Differential Games

In this paper, we are concerned with an open-loop Nash differential game. The necessary conditions for an open-loop Nash equilibrium solution are obtained, also the existence for the solution of the dynamical system of the differential game is studied. Picard method is used to find an approximate solution, and the uniform convergence is proved. Finally, we constructed figures for the analysis of the differential game. These results can be applied between economic and financial firms as well as industrial firms.

Decentralized Open-Loop Strategies of Linear Quadratic Mean Field Games: From Finite to Infinite Population

This paper studies the decentralized open-loop asymptotic Nash equilibria for linear quadratic mean field games following the direct approach. In the first step of the direct approach, the necessary and sufficient condition for the existence of centralized open-loop Nash equilibria is obtained by using the variational analysis. The Nash equilibria are characterized by an adapted solution to a system of forward-backward stochastic differential equations. A feedback representation of the open-loop Nash equilibrium is obtained by using the solution to a system of Riccati equations. In the second step of the direct approach, the decentralized asymptotic Nash equilibria are designed by considering the limit of the centralized Nash equilibria with a standard Riccati equation and a non-symmetric Riccati equation. The results show that the decentralized open-loop asymptotic Nash equilibria coincide with the feedback solutions which are designed by the fixed-point approach, provided both exist.

Large population games with interactions through controls and common noise: convergence results and equivalence between open-loop and closed-loop controls

In the presence of a common noise, we study the convergence problems in mean field game (MFG) and mean field control (MFC) problem where the cost function and the state dynamics depend upon the joint conditional distribution of the controlled state and the control process. In the first part, we consider the MFG setting. We start by recalling the notions of measure-valued MFG equilibria and of approximate closed-loop Nash equilibria associated to the corresponding N-player game. Then, we show that all convergent sequences of approximate closed-loop Nash equilibria, when N → ∞, converge to measure-valued MFG equilibria. And conversely, any measure-valued MFG equilibrium is the limit of a sequence of approximate closed-loop Nash equilibria. In other words, measure-valued MFG equilibria are the accumulation points of the approximate closed-loop Nash equilibria. Previous work has shown that measure-valued MFG equilibria are the accumulation points of the approximate openloop Nash equilibria. Therefore, we obtain that the limits of approximate closed-loop Nash equilibria and approximate open-loop Nash equilibria are the same. In the second part, we deal with the MFC setting. After recalling the closed-loop and open-loop formulations of the MFC problem, we prove that they are equivalent. We also provide some convergence results related to approximate closed-loop Pareto equilibria.

Decentralized strategies for finite population linear–quadratic–Gaussian games and teams

This paper is concerned with a new class of mean-field games which involve a finite number of agents. Necessary and sufficient conditions are obtained for the existence of the decentralized open-loop Nash equilibrium in terms of non-standard forward–backward stochastic differential equations (FBSDEs). By solving the FBSDEs, we design a set of decentralized strategies by virtue of two differential Riccati equations. Instead of the ɛ-Nash equilibrium in classical mean-field games, the set of decentralized strategies is shown to be a Nash equilibrium. For the infinite-horizon problem, a simple condition is given for the solvability of the algebraic Riccati equation arising from consensus. Furthermore, the social optimal control problem is studied. Under a mild condition, the decentralized social optimal control and the corresponding social cost are given.

A two-player portfolio tracking game

We study the competition of two strategic agents for liquidity in the benchmark portfolio tracking setup of Bank et al. (Math Financial Economics 11(2):215–239 2017). Specifically, both agents track their own stochastic running trading targets while interacting through common aggregated temporary and permanent price impact à la Almgren and Chriss (J Risk 3:5–39 2001). The resulting stochastic linear quadratic differential game with terminal state constraints allows for a unique and explicitly available open-loop Nash equilibrium. Our results reveal how the equilibrium strategies of the two players take into account the other agent’s trading targets: either in an exploitative intent or by providing liquidity to the competitor, depending on the relation between temporary and permanent price impact. As a consequence, different behavioral patterns can emerge as optimal in equilibrium. These insights complement and extend existing studies in the literature on predatory trading models examined in the context of optimal portfolio liquidation games.

Study of Multiple Target Defense Differential Games Using Receding Horizon-Based Switching Strategies

In this article, we study a variation of the active target-attacker-defender (ATAD) differential game involving multiple targets, an attacker, and a defender. Our model allows for 1) a capability of the defender to switch roles from <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">rescuer</i> (rendezvous with all the targets) to <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">interceptor</i> (intercepts the attacker) and vice versa and 2) the attacker to continuously pursue the closest target (which can change during the course of the game). We assume that the mode of the defender (rescue or interception) defines the mode of the game itself. Using the framework of Games of a Degree, we first analyze the game within each mode. More specifically, the objectives of the players are taken as a combination of weighted Euclidean distances and penalties on their control efforts. We model the interaction of the players within each mode as a linear quadratic differential game (LQDG) and obtain the open-loop Nash equilibrium strategies. We then use the receding horizon approach to enable switching between the modes to obtain switching strategies for the players. By partitioning the matrices associated with the Riccati differential equations we obtain geometric characterization of the trajectories of the players. Furthermore, under mild restrictions on the problem parameters and for a particular choice of the defender’s switching function we show that interception mode is invariant. We illustrate our results with numerical simulations. Experimental results involving multiple autonomous differential drive mobile robots are presented.

On the analysis of open-loop Nash equilibria admitting a feedback synthesis in nonlinear differential games

Open-loop Nash equilibrium strategies for differential games described by nonlinear, input-affine, systems and cost functionals that are quadratic with respect to the control input are studied. First it is shown that the computation of such strategies hinges upon the solution of a system of nonlinear, time-varying, partial differential equations (PDEs) obtained by building on arguments borrowed from Pontryagin’s Minimum Principle and combined with Dynamic Programming considerations. Then, by relying on a state/costate interpretation of the above characterization, a feedback synthesis of the underlying open-loop strategy is obtained by solving linear first-order PDEs that ensure invariance of certain submanifolds in the state–space of the extended state/costate dynamics. These PDEs are the nonlinear counterpart of the well-known asymmetric Algebraic Riccati Equations arising in the study of linear quadratic Nash games.

A numerical iterative method for solving two-point BVPs in infinite-horizon nonzero-sum differential games: Economic applications

In this work, the Nash equilibrium solution of nonlinear infinite-horizon nonzero-sum differential games with open-loop information is investigated numerically. For this class of games, some difficulties are involved in finding an open-loop Nash equilibrium. For instance, we must solve a nonlinear differential equations system with split boundary conditions, namely the two-point boundary value problem (TPBVP), in which some of the boundary conditions are specified at the initial time and some at the infinite final time. In the current study, we provide a combined numerical algorithm based on a new set of basis functions on the half-line, called the exponential Chelyshkov functions (ECFs), and quasilinearization (QL) method to solve TPBVPs. In the first step, we reduce the nonlinear TPBVP to a sequence of linear differential equations by using the QL method. Although we have now a linearized system, another difficulty is finding the approximate solution of this linear system such that the transversality conditions at the infinite final time are satisfied. So, in the second step, we apply a collocation method based on the ECFs to solve the obtained linear system in the semi-infinite domain. The convergence of the proposed method is discussed in detail. To confirm the validity and efficiency of the proposed scheme, we compute the approximate solution of TPBVP as well as the open-loop Nash equilibrium for four applications of differential games in economics and management science.

Comparative Dynamics and Envelope Theorems of Open-Loop Stackelberg Equilibria in Differential Games

A direct proof of the envelope theorems and intrinsic comparative dynamics of locally differentiable open-loop Stackelberg equilibria (OLSE) is given using an extended primal-dual method. It is shown that the follower’s envelope and comparative dynamics results agree in form with those of any player in an open-loop Nash equilibrium, while those of the leader differ. This difference allows, in principle, an empirical test of the leader–follower role in a differential game. Separability conditions are identified on the instantaneous payoff and transition functions under which, for certain parameters, the intrinsic comparative dynamics of the leader’s time-inconsistent OSLE and those in the corresponding optimal control problem are qualitatively identical. However, similar conditions do not exist for time-consistent OSLE.

Conditional LQ time-inconsistent Markov-switching stochastic optimal control problem for diffusion with jumps

The paper presents a characterization of equilibrium in a game-theoretic description of discounting conditional stochastic linear-quadratic (LQ for short) optimal control problem, in which the controlled state process evolves according to a multidimensional linear stochastic differential equation, when the noise is driven by a Poisson process and an independent Brownian motion under the effect of a Markovian regime-switching. The running and the terminal costs in the objective functional are explicitly dependent on several quadratic terms of the conditional expectation of the state process as well as on a nonexponential discount function, which create the time-inconsistency of the considered model. Open-loop Nash equilibrium controls are described through some necessary and sufficient equilibrium conditions. A state feedback equilibrium strategy is achieved via certain differential-difference system of ODEs. As an application, we study an investment–consumption and equilibrium reinsurance/new business strategies for mean-variance utility for insurers when the risk aversion is a function of current wealth level. The financial market consists of one riskless asset and one risky asset whose price process is modeled by geometric Lévy processes and the surplus of the insurers is assumed to follow a jump-diffusion model, where the values of parameters change according to continuous-time Markov chain. A numerical example is provided to demonstrate the efficacy of theoretical results.

An OLG Differential Game of Pollution Control with the Risk of a Catastrophic Climate Change

This paper studies an overlapping generations (OLG) differential game on optimal emissions with continuous age structure and different types of individuals. At the (stochastic) arrival of a catastrophic climate change, the utility and the damage to the stock of pollution change for the rest of the time horizon. We derive the open-loop (OL) Nash equilibrium and show that it is subgame perfect and moreover equal to the feedback Stackelberg one. We compare the solution to the cooperative one (using the social welfare as objective function) and show the different dynamic evolutions of optimal emissions over time. Finally, we derive a time-consistent tax scheme that reaches the cooperative optimal solution in the OL Nash equilibrium. The tax scheme turns out to be heterogeneous with respect to age and type (anticipating and nonanticipating the catastrophic climate change). Setting taxes that are homogeneous across the individual type leads to an OL Nash solution that produces socially optimal total emissions, but lower individual utilities.

Search Without Looking

We analyze a dynamic game in which agents strategically search for a prize/reward of known value when they cannot observe the search of others. In every period the rivals decide how much to search. The prize goes to the player who finds it first unless there is simultaneous discovery, in which case the reward is destroyed. In the unique symmetric open loop Nash equilibrium all players receive an expected payoff of zero. A third party could however increase welfare and avoid some search duplication by allocating search zones, even if these exclusive search zones are non-binding.

Constructive design of open-loop Nash equilibrium strategies that admit a feedback synthesis in LQ games

Open-loop Nash equilibrium strategies that admit a feedback synthesis in Linear–Quadratic (LQ) games are studied. A characterization alternative to the classic system of coupled (asymmetric) Riccati equations – one for each player – is provided by relying on a fixed-point argument based on the composition of flows of the underlying state/costate dynamics. As a result, it is shown that in competitive games, namely games in which the players influence the shared state via linearly independent input channels, the characterization of Nash equilibrium strategies hinges upon the solution to a single (regardless of the number of players), sign-definite Riccati equation, with coefficients described by polynomial functions of the feedback gains. The structure of the latter equation is computationally appealing since it naturally allows for gradient-descent algorithms on matrix manifolds, thus ensuring (local) guaranteed convergence to the equilibrium strategy. In the case of antagonistic games, namely games in which the players may share linearly dependent input directions, the fixed-point condition above is combined with a geometric requirement involving the largest invariant subspace contained in the kernel of an auxiliary output matrix. Finally, by building on the latter characterization it is shown that closed-form expressions for the equilibrium strategy for a class of dynamic games can be given.