Equilibria In Stationary Strategies Research Articles

Nonzero-Sum Risk-Sensitive Continuous-Time Stochastic Games with Ergodic Costs

We study nonzero-sum stochastic games for continuous time Markov decision processes on a denumerable state space with risk-sensitive ergodic cost criterion. Transition rates and cost rates are allowed to be unbounded. Under a Lyapunov type stability assumption, we show that the corresponding system of coupled HJB equations admits a solution which leads to the existence of a Nash equilibrium in stationary strategies. We establish this using an approach involving principal eigenvalues associated with the HJB equations. Furthermore, exploiting appropriate stochastic representation of principal eigenfunctions, we completely characterize Nash equilibria in the space of stationary Markov strategies.

An Interior-Point Differentiable Path-Following Method to Compute Stationary Equilibria in Stochastic Games

The subgame perfect equilibrium in stationary strategies (SSPE) is the most important solution concept in applications of stochastic games, making it imperative to develop efficient methods to compute an SSPE. For this purpose, this paper develops an interior-point differentiable path-following method (IPM), which establishes a connection between an artificial logarithmic barrier game and the stochastic game of interest by adding a homotopy variable. IPM brings several advantages over the existing methods for stochastic games. On the one hand, IPM provides a bridge between differentiable path-following methods and interior-point methods and remedies several issues of an existing homotopy method called the stochastic linear tracing procedure (SLTP). First, the starting stationary strategy profile can be arbitrarily chosen. Second, IPM does not need switching between different systems of equations. Third, the use of a perturbation term makes IPM applicable to all stochastic games rather than generic games only. Moreover, a well-chosen transformation of variables reduces the number of equations and variables by roughly one half. Numerical results show that the proposed method is more than three times as efficient as SLTP. On the other hand, the stochastic game can be reformulated as a mixed complementarity problem and solved by the PATH solver. We employ the proposed IPM and the PATH solver to compute SSPEs. Numerical results evince that for some stochastic games the PATH solver may fail to find an SSPE, whereas IPM is successful in doing so for all stochastic games, which confirms the reliability and stability of the proposed method. Summary of Contribution: This paper incorporates the interior-point methods into a differentiable path-following method for computing stationary equilibria for stochastic games. This novel method brings excellent computational advantages and remedies several issues with the existing methods for stochastic games. We prove the global convergence of the proposed method and employ this method to solve numerous randomly generated stochastic games with different scales. Numerical results further confirm the high efficiency, stability, and universality of this method for stochastic games.

A differentiable path-following method to compute subgame perfect equilibria in stationary strategies in robust stochastic games and its applications

As an effective paradigm to address uncertainty in payoffs and transition probabilities, robust stochastic games have been formulated in the literature. This paper is concerned with the computation of subgame perfect equilibria in stationary strategies (SSPEs) in robust stochastic games. To tackle this problem, we develop in this paper a globally convergent differentiable path-following method by exploiting the structures of the games. Incorporating a logarithmic-barrier term into each player’s payoff function with an extra variable between zero and one, we constitute a logarithmic-barrier robust stochastic game in which each player solves in each state a convex optimization problem. An application of the optimality conditions to the barrier game together with a fixed-point argument yields a polynomial equilibrium system for the barrier game. As a result of this system, we establish the existence of a smooth path that starts from an arbitrary mixed strategy profile and ends at an SSPE as the extra variable descends from one to zero. As an alternative scheme, we make up a convex-quadratic-penalty robust stochastic game and attain a globally convergent convex-quadratic-penalty differentiable path-following method for SSPEs in robust stochastic games. Numerical comparisons show that the logarithmic-barrier path-following method significantly outperforms the convex-quadratic-penalty path-following method. To further evince the value of the proposed methods, we apply the logarithmic-barrier path-following method to solve a supply chain configuration problem and a market entry problem from medical waste recycling.

Costless delay in negotiations

We study bargaining models in discrete time with a finite number of players, stochastic selection of the proposing player, endogenously determined sets and orders of responders, and a finite set of feasible alternatives. The standard optimality conditions and system of recursive equations may not be sufficient for the existence of a subgame perfect equilibrium in stationary strategies (SSPE) in case of costless delay. We present a characterization of SSPE that is valid for both costly and costless delay. We address the relationship between an SSPE under costless delay and the limit of SSPEs under vanishing costly delay. An SSPE always exists when delay is costly, but not necessarily so under costless delay, even when mixed strategies are allowed for. This is surprising as a quasi SSPE, a solution to the optimality conditions and the system of recursive equations, always exists. The problem is caused by the potential singularity of the system of recursive equations, which is intimately related to the possibility of perpetual disagreement in the bargaining process.

An Arbitrary Starting Tracing Procedure for Computing Subgame Perfect Equilibria

The computation of subgame perfect equilibrium in stationary strategies is an important but challenging problem in applications of stochastic games. In 2004, Herings and Peeters developed a homotopy method called stochastic linear tracing procedure to solve this problem. However, the starting point of their method requires to be explicitly calculated. To remedy this issue, we formulate an arbitrary starting linear tracing procedure in this paper. By introducing a homotopy variable ranging from two to zero, an artificial penalty game is developed, whose solutions construct a differentiable path after a well-chosen transformation of variables. The starting point of the path can be arbitrarily chosen, so that there is no need to employ additional algorithms to obtain it. Following the path, one can readily attain the “starting point” of the stochastic tracing procedure coined by Herings and Peeters. Then, as the homotopy variable changes from one to zero, the path essentially resumes to the stochastic tracing procedure. We prove that our method globally converges to a subgame perfect equilibrium in stationary strategies for the stochastic game of interest. Numerical results further illustrate the effectiveness and efficiency of our method.

An Interior-Point Path-Following Method to Compute Stationary Equilibria in Stochastic Games

Subgame perfect equilibrium in stationary strategies (SSPE) is the most important solution concept used in applications of stochastic games, which makes it imperative to develop efficient numerical methods to compute an SSPE. For this purpose, this paper develops an interior-point path-following method (IPM), which remedies a number of issues with the existing method called stochastic linear tracing procedure (SLTP). The homotopy system of IPM is derived from the optimality conditions of an artificial barrier game, whose objective function is a combination of the original payoff function and a logarithmic term. Unlike SLTP, the starting stationary strategy profile can be arbitrarily chosen and IPM does not need switching between different systems of equations. The use of a perturbation term makes IPM applicable to all stochastic games, whereas SLTP only works for a generic stochastic game. A transformation of variables reduces the number of equations and variables of by roughly one half. Numerical results show that our method is more than three times as efficient as SLTP.

One-Dimensional Bargaining with a General Voting Rule

We study a model of multilateral bargaining over social outcomes represented by points in the unit interval. The acceptance or rejection of a proposal is determined by an acceptance rule as represented by the collection of decisive coalitions. The focus of the paper is on the asymptotic behavior of subgame perfect equilibria in stationary strategies as the discount factor goes to one. We show that, along any sequence of stationary subgame perfect equilibria the social acceptance set collapses to a point. This point, called the limit of bargaining equilibria, is independent of the sequence of equilibria and is uniquely determined by the set of players, the utility functions, the recognition probabilities, and the acceptance rule. The central result of the paper is a characterization of the limit of bargaining equilibria as a unique zero of the characteristic equation.

Bargaining under monotonicity constraints

We study unanimity bargaining on the division of a surplus in the presence of monotonicity constraints. The monotonicity constraints specify a complete order on the players, which has to be respected by the shares in the surplus the players obtain in any bargaining outcome. A player higher in the order should not receive a lower share of the surplus. We analyze the resulting subgame perfect equilibria in stationary strategies and show that they are characterized by the simpler notion of bargaining equilibrium. Bargaining equilibria are shown to be unique and to have the property that players ranked strictly higher obtain strictly higher shares in the surplus. The key question is whether the bargaining advantage of a higher-ranked player persists when the probability of breakdown of bargaining tends to zero. We argue that such is not the case by showing that bargaining equilibria have a unique limit equal to an equal division of the surplus. It then follows that the limit also coincides with the Nash bargaining solution for this problem.

On the convergence to the Nash bargaining solution for action-dependent bargaining protocols

We consider a non-cooperative multilateral bargaining game and study an action-dependent bargaining protocol, that is, the probability with which a player becomes the proposer in a round of bargaining depends on the identity of the player who previously rejected. An important example is the frequently studied rejector-becomes-proposer protocol. We focus on subgame perfect equilibria in stationary strategies which are shown to exist and to be efficient. Equilibrium proposals do not depend on the probability to propose conditional on the rejection by another player. We consider the limit, as the bargaining friction vanishes. In case no player has a positive probability to propose conditional on his rejection, each player receives his utopia payoff conditional on being recognized. Otherwise, equilibrium proposals of all players converge to a weighted Nash bargaining solution, where the weights are determined by the probability to propose conditional on one's own rejection.

A bargaining theory of the firm

Suppose that a firm has several owners and that the future is uncertain in the sense that one out of many different states of nature will realize tomorrow. An owner’s time preference and risk attitude will determine the importance he places on payoffs in the different states. It is a well-known problem in the literature that under incomplete asset markets, a conflict about the firm’s objective function tends to arise among its owners. In this paper, we take a new approach to this problem, which is based on non-cooperative bargaining. The owners of the firm play a bargaining game in order to choose the firm’s production plan and a scheme of transfers which are payable before the uncertainty about the future state of nature is resolved. We analyze the resulting firm decision in the limit of subgame-perfect equilibria in stationary strategies. Given the distribution of bargaining power, we obtain a unique prediction for a production plan and a transfer scheme. When markets are complete, the production plan chosen corresponds to the profit-maximizing production plan as in the Arrow–Debreu model. Contrary to that model, owners typically do use transfers to redistribute profits. When markets are incomplete, the production plan chosen is almost always different from the one in a transfer-free Dreze (pseudo-)equilibrium and again owners use transfers to redistribute profits. Nevertheless, our results do support the Dreze criterion as the appropriate objective function of the firm.

On the Convergence to the Nash Bargaining Solution for Endogenous Bargaining Protocols

We consider non-cooperative multilateral bargaining games with endogenous bargaining protocols. Under an endogenous protocol, the probability with which a player becomes the proposer in a round of bargaining depends on the identity of the player who previously rejected. An important example is the frequently studied rejector-becomes-proposer protocol. We focus on subgame perfect equilibria in stationary strategies which are shown to exist and to be efficient. Equilibrium proposals do not depend on the probability to propose conditional on the rejection by another player, though equilibrium acceptance sets do depend on these probabilities. Next we consider the limit, as the bargaining friction vanishes. In case no player has a positive probability to propose conditional on his rejection, each player receives his utopia payoff conditional on being recognized and equilibrium payoffs are in general Pareto inefficient. Otherwise, equilibrium proposals of all players converge to a weighted Nash Bargaining Solution, where the weights are determined by the probability to propose conditional on a rejection.

Asymmetric Information in Bilateral Trade and in Markets: An Inversion Result

I consider a simple bilateral trading game between a seller and a buyer who have private valuations for an indivisible good. The seller makes a price offer which the buyer can either accept or reject. If the seller can observe the valuation of the buyer (if information is symmetric), then bilateral trade is trivially efficient. If the seller cannot observe the valuation (if information is asymmetric), then bilateral trade is inefficient. This bilateral trading game between a single buyer and a single seller is embedded into a matching market with a continuum of traders. I consider steady-state equilibria in stationary strategies. I show that, on the overall market level, the relation between the informational regime and efficiency is inverted when frictions are small. In particular, if frictions are small and if information is asymmetric, the trading outcome in the market is close to being efficient. If information is symmetric, however, the trading outcome can be very inefficient -- even if frictions vanish.

Theory of the Firm: Bargaining and Competitive Equilibrium

Suppose that a firm has several owners and that the future is uncertain in the sense that one out of many different states of nature will realize tomorrow. An owner's time preference and risk attitude will determine the importance he places on payoffs in the different states. It is a well-known problem in the literature that under incomplete asset markets, a conflict about the firm's objective function tends to arise among its owners. In this paper, we take a new approach to this problem, which is based on non-cooperative bargaining. The owners of the firm play a bargaining game in order to choose the firm's production plan and a scheme of transfers which are payable before the uncertainty about the future state of nature is resolved. We analyze the resulting firm decision in the limit of subgame-perfect equilibria in stationary strategies. Given the distribution of bargaining power, we obtain a unique prediction for a production plan and a transfer scheme. When markets are complete, the production plan chosen corresponds to the profit-maximizing production plan as in the Arrow-Debreu model. Contrary to that model, owners typically do use transfers to redistribute profits. When markets are incomplete, the production plan chosen is almost always different from the standard notion of competitive equilibrium and again owners use transfers to redistribute profits. Nevertheless, our results do support the Dreze criterion as the appropriate objective function of the firm.

On the Asymptotic Uniqueness of Bargaining Equilibria

This note reexamines the connection between the asymmetric Nash bargaining solution and the equilibria of strategic bargaining games. Several papers in the literature obtain the asymmetric Nash bargaining solution as the unique limit of subgame perfect equilibria in stationary strategies when the breakdown probability approaches zero. This note illustrates by means of two examples that this result depends crucially on the differentiability of the boundary of the set of feasible payoffs. In the first example the game has a unique stationary subgame perfect equilibrium that fails to converge to the asymmetric Nash bargaining solution. In the second example the game has two stationary subgame perfect equilibria that converge to two distinct limits as the breakdown probability vanishes. This example demonstrates that without differentiability of the set of feasible payoffs there is not even asymptotic uniqueness of stationary equilibria in the bargaining model.

One-dimensional bargaining

We study a model of multilateral bargaining over social outcomes represented by the points in the unit interval. The acceptance or rejection of a proposal is determined by an acceptance rule represented by the collection of decisive coalitions. The focus of the paper is on the asymptotic behavior of subgame perfect equilibria in stationary strategies as the players become infinitely patient. We show that, along any sequence of stationary subgame perfect equilibria the social acceptance set collapses to a point. This point, called the limit of bargaining equilibria, is independent of the sequence of equilibria and is uniquely determined by the set of players, the utility functions, the recognition probabilities, and the acceptance rule. The central result of the paper is a characterization of the limit of bargaining equilibria as a unique zero of the characteristic equation.

Endogenous Two-Sided Markets with Repeated Transactions

We consider homogeneous two-sided markets, in which connected buyer-seller pairs bargain and trade repeatedly. In this infinite market game with exogenous matching probabilities and a common discount factor, we prove the existence of equilibria in stationary strategies. The equilibrium payoffs are given implicitly as a solution to a system of linear equations. Then, we endogenize the matching mechanism in a link formation stage that precedes the market game. When agents are sufficiently patient and link costs are low, we provide an algorithm to construct minimally connected networks that are pairwise stable with respect to the expected payoffs in the trading stage. The constructed networks are essentially efficient and consist of components with a constant buyer-seller ratio. The latter ratio increases (decreases) for a buyer (seller) that deletes one of her links in a pairwise stable component.

On the Asymptotic Uniqueness of Bargaining Equilibria

This note reexamines the connection between the asymmetric Nash bargaining solution and the equilibria of strategic bargaining games. Several papers in the literature obtain the asymmetric Nash bargaining solution as the unique limit of subgame perfect equilibria in stationary strategies when the breakdown probability approaches zero. This note illustrates by means of two examples that this result depends crucially on the differentiability of the boundary of the set of feasible payoffs. In the first example the game has a unique stationary subgame perfect equilibrium that fails to converge to the asymmetric Nash bargaining solution. In the second example the game has two stationary subgame perfect equilibria that converge to two distinct limits as the breakdown probability vanishes. This example demonstrates that without differentiability of the set of feasible payoffs there is not even asymptotic uniqueness of stationary equilibria in the bargaining model.

One-dimensional bargaining with Markov recognition probabilities

We study a process of bargaining over alternatives represented by points in the unit interval. The paper focuses on the asymptotic behavior of the subgame perfect equilibrium in stationary strategies as the continuation probability approaches one. We give a complete characterization of the limit of the equilibrium proposals as the generalized fixed point of the decumulative distribution of the players' ideal points as induced by the recognition probabilities. In contrast to the existing literature, we find no general relationship between the limit equilibrium proposals and either the Nash bargaining solution or the median voter outcome.

Non-Cooperative Support for the Asymmetric Nash Bargaining Solution

Our work contributes to the game-theoretic analysis of bargaining by providing additional non-cooperative support to the well-known Nash bargaining solution. In particular, in the present paper we study a model of non-cooperative multilateral bargaining with a very general proposer selection protocol and set of feasible payoffs. In each period of the bargaining game, one out of n players is recognized as the proposer according to an irreducible Markov process. The proposer offers a particular element of the convex set of feasible payoffs. If all players accept the offer, it is implemented. If a player rejects the offer, with some probability the negotiations break down and with the remaining probability the next period starts. We show that subgame perfect equilibria in stationary strategies exist and we fuly characterize the set of such equilibria. Our main result is that in the limit, as the exogenous risk of breakdown goes to zero, stationary subgame perfect equilibrium payoffs converge to the weighted Nash bargaining solution with the stationary distribution of the Markov proposer selection process as the weight vector.

Fictitious play in stochastic games

In this paper we examine an extension of the fictitious play process for bimatrix games to stochastic games. We show that the fictitious play process does not necessarily converge, not even in the 2 × 2 × 2 case with a unique equilibrium in stationary strategies. Here 2 × 2 × 2 stands for 2 players, 2 states, 2 actions for each player in each state.