Markov Perfect Nash Equilibrium Research Articles

A Strategic Dynamic Programming Method for Studying Short-Memory Equilibria of Stochastic Games with Uncountable Number of States

We study a class of infinite horizon stochastic games with uncountable number of states. We first characterize the set of all (nonstationary) short-term (Markovian) equilibrium values by developing a new (Abreu et al. in Econometrica 58(5):1041–1063, 1990)-type procedure operating in function spaces. This (among others) proves Markov perfect Nash equilibrium (MPNE) existence. Moreover, we present techniques of MPNE value set approximation by a sequence of sets of discretized functions iterated on our approximated APS-type operator. This method is new and has some advantages as compared to Judd et al. (Econometrica 71(4):1239–1254, 2003), Feng et al. (Int Econ Rev 55(1):83–110, 2014), or Sleet and Yeltekin (Dyn Games Appl doi: 10.1007/s13235-015-0139-1 , 2015). We show applications of our approach to hyperbolic discounting games and dynamic games with strategic complementarities.

A Strategic Dynamic Programming Method for Studying Short Memory Equilibria of Stochastic Games with Uncountable Number of States

We study a class of infinite horizon stochastic games with uncountable number of states. We first characterize the set of all (nonstationary) short-term (Markovian) equilibrium values by developing a new Abreu, Pearce, and Stacchetti (1990) type procedure operating in function spaces. This (among others) proofs Markov Perfect Nash Equilibrium (MPNE) existence. Moreover we present techniques of MPNE value set approximation by a sequence of sets of discretized functions iterated on our approximated APS-type operator. This method is new and has some advantages as compared to Judd, Yeltekin, and Conklin (2003), Feng, Miao, Peralta-Alva, and Santos (2014) or Sleet and Yeltekin (2015). We show applications of our approach to hyperbolic discounting games and dynamic games with strategic complementarities.

Euler–Lagrange equations of stochastic differential games: application to a game of a productive asset

This paper analyzes a noncooperative and symmetric dynamic game where players have free access to a productive asset whose evolution is a diffusion process with Brownian uncertainty. A Euler–Lagrange equation is found and used to provide necessary and sufficient conditions for the existence and uniqueness of a smooth Markov Perfect Nash Equilibrium. The Euler–Lagrange equation also provides a stochastic Keynes–Ramsey rule, which has the form of a forward–backward stochastic differential equation. It is used to study the properties of the equilibrium and to make some comparative statics exercises.

Noncooperative Resource Exploitation by Patient Players

We consider a discrete-time dynamic game in which a finite number of players extract a non-renewable resource and derive consumption solely from the extracted amount (cake-eating game). Markov perfect Nash equilibria consisting of linear strategies can be constructed in this game not only if the players have time-preference factors that are smaller than 1, but also if these factors are equal to or even larger than 1. We demonstrate this result both for the case of identical players and for the case of heterogeneous players. In addition, we study the influence of the model parameters on the equilibrium.

On the effects of mergers on equilibrium outcomes in a common property renewable asset oligopoly

This paper examines a dynamic game of exploitation of a common pool of some renewable asset by agents that sell the result of their exploitation on an oligopolistic market. A Markov Perfect Nash Equilibrium of the game is used to analyze the effects of a merger of a subset of the agents. We study the impact of the merger on the equilibrium production strategies, on the steady states, and on the profitability of the merger for its members. We show that there exists an interval of the asset׳s stock such that any merger is profitable if the stock at the time the merger is formed falls within that interval. That includes mergers that are known to be unprofitable in the corresponding static equilibrium framework.

Common Pool Politics and Inefficient Fishery Management

Fisheries management often fails because total allowable catches (TACs) are set at inefficiently high levels. To study why decision-makers choose such high TACs, we model the annual negotiation on TACs as a dynamic game in discrete time. TACs are fixed by majority decision in a council consisting of decision-makers who are heterogeneous with respect to their discount rates. We show that the optimal feedback strategy for the less patient decision-makers will set inefficiently high TACs in Markov-perfect Nash equilibrium. A binding commitment to a long-term management plan could help solving this problem and lead to a more sustainable fishery management.

Measuring the Impact of Competition on Pricing Behaviors in a Two-Sided Market

The impact of competition on pricing has been studied in the context of counterfactual merger analyses where expected optimal prices in a hypothetical monopoly are compared with observed prices in an oligopolistic market. Such analyses would typically assume static decision making by consumers and firms and thus have been applied mostly to data obtained from consumer packed goods such as cereal and soft drinks. However such static modeling approach is not suitable when decision makers are forward looking. When it comes to the markets for durable products with indirect network effects, consumer purchase decisions and firm pricing decisions are inherently dynamic as they take into account future states when making purchase and pricing decisions. Researchers need to take into account the dynamic aspects of decision making both in the consumer side and in the supplier side for such markets. Firms in a two-sided market typically subsidize one side of the market to exploit the indirect network effect. Such pricing behaviors would be more prevalent in competitive markets where firms would try to win over the battle for standard. While such qualitative expectation on the relationship between pricing behaviors and competitive structures could be easily formed, little empirical studies have measured the extent to which the distinct pricing structure in two-sided markets depends on the competitive structure of the market. This paper develops an empirical model to measure the impact of competition on optimal pricing of durable products under indirect network effects. In order to measure the impact of exogenously determined competition among firms on pricing, we compare the equilibrium prices in the observed oligopoly market to those in a hypothetical monopoly market. In computing the equilibrium prices, we account for the forward looking behaviors of consumers and supplier. We first estimate a demand function that accounts for consumers`` forward-looking behaviors and indirect network effects. And then, for the supply side, the pricing equation is obtained as an outcome of the Markov Perfect Nash Equilibrium in pricing. In doing so, we utilize numerical dynamic programming techniques. We apply our model to a data set obtained from the U.S. video game console market. The video game console market is considered a prototypical case of two-sidedmarkets in which the platform typically subsidizes one side of market to expand the installed base anticipating larger revenues in the other side of market resulting from the expanded installed base. The data consist of monthly observations of price, hardware unit sales and the number of compatible software titles for Sony PlayStation and Nintendo 64 from September 1996 to August 2002. Sony PlayStation was released to the market a year before Nintendo 64 was launched. We compute the expected equilibrium price path for Nintendo 64 and Playstation for both oligopoly and for monopoly. Our analysis reveals that the price level differs significantly between two competition structures. The merged monopoly is expected to set prices higher by 14.8% for Sony PlayStation and 21.8% for Nintendo 64 on average than the independent firms in an oligopoly would do. And such removal of competition would result in a reduction in consumer value by 43.1%. Higher prices are expected for the hypothetical monopoly because the merged firm does not need to engage in the battle for industry standard. This result is attributed to the distinct property of a two-sided market that competing firms tend to set low prices particularly at the initial period to attract consumers at the introductory stage and to reinforce their own networks and eventually finally to dominate the market.

Extinction in common property resource models: an analytically tractable example

We discuss an analytically tractable discrete-time dynamic game in which a finite number of players extract a renewable resource. We characterize a symmetric Markov-perfect Nash equilibrium of this game and derive a necessary and sufficient condition under which the resource does not become extinct in equilibrium. This condition requires that the intrinsic growth rate of the resource exceeds a certain threshold value that depends on the number of players and on their time-preference rates.

On the Effects of Mergers on Equilibrium Outcomes in a Common Property Renewable Asset Oligopoly

This paper examines a dynamic game of exploitation of a common pool of some renewable asset by agents that sell the result of their exploitation on an oligopolistic market. A Markov Perfect Nash Equilibrium of the game is used to analyze the effects of a merger of a subset of the agents. We study the impact of the merger on the equilibrium production strategies, on the steady states, and on the profitability of the merger for its members. We show that there exists an interval of the asset’s stock such that any merger is profitable if the stock at the time the merger is formed falls within that interval. That includes mergers that are known to be unprofitable in the corresponding static equilibrium framework.

An Empirical Model of Drug Detailing: Dynamic Competition and Policy Implications

The practice of detailing in the marketing of prescription drugs is undergoing significant changes. For instance, in September 2013, the Physician Payment Sunshine Act went into full effect. The accompanying transparency requirements have prompted physician practices and hospitals to severely restrict pharmaceutical sales representatives’ direct access to their physicians. Despite all the attention in the popular press, scant scholarly research has investigated how these restrictions on physician access impact physician prescription behavior and competitive detailing to physicians. To analyze the impact of these restrictions, we develop a structural model of how pharmaceutical firms compete dynamically to schedule detailing to physicians. Detailing activities are known to have significant carry-over effects that are captured in a first-stage model of physicians’ demand for prescription drugs. We also specify detailing policy functions that describe each firm’s observed detailing actions. In a second stage, we estimate a model that describes costs of detailing, assuming that the observed detailing levels are consistent with a Markov perfect Nash equilibrium. The estimated structural model is used to examine the implications of restrictions on the amount of detailing via counterfactual simulations. We find that restriction policies would increase the market share of a non-drug-treatment-only option but impact firms asymmetrically; firms that are strong in detailing and/or rely more on detailing would be hurt more. Unexpectedly, a policy that imposes a ceiling on detailing frequency could significantly reduce detailing of all firms in the market, including those firms with average detailing levels below the ceiling, and effectively would soften competition be-tween firms and enhance their profits.

Do Social Status Seeking Behaviors Worsen the Tragedy of the Commons?

The present paper considers the exploitation of a common-property, nonrenewable resource, by individuals concerned with their social status. Assuming that the social status is reflected by the individuals’ relative consumptions, we formalize this motivation by means of a utility function, depending on the individual’s actual consumption and on the consumption level he aspires, the latter being related to the consumptions in his reference group. We compare the benchmark cooperative solution with a noncooperative Markov-perfect Nash equilibrium. We confirm, under more general conditions than in the existing literature, that the individuals’ concern for social status exacerbates the tragedy of the commons. We finally discuss the policy implications and provide a taxation scheme capable of implementing the cooperative solution as a noncooperative Markov-perfect Nash equilibrium.

Strategic exploitation of a common-property resource under uncertainty

We construct a game of noncooperative common-resource exploitation which delivers analytical solutions for its symmetric Markov-perfect Nash equilibrium. We examine how introducing uncertainty to the natural law of resource reproduction affects strategic exploitation. We show that the commons problem is always present in our example and we identify cases in which increases in risk amplify or mitigate the commons problem. For a specific class of games which imply Markov-perfect strategies that are linear in the resource stock (our example belongs to this class), we provide general results on how payoff-function features affect the responsiveness of exploitation strategies to changes in riskiness. These broader characterizations of games which imply linear strategies (appearing in an Online Appendix) can be useful in future work, given the technical difficulties that may arise from the possible nonlinearity of Markov-perfect strategies in more general settings.

Stationary Markovian equilibrium in altruistic stochastic OLG models with limited commitment

We introduce a new class of infinite horizon altruistic stochastic OLG models with capital and labor, but without commitment between the generations. Under mild regularity conditions, for economies with either bounded or unbounded state spaces, continuous monotone Markov perfect Nash equilibrium (henceforth MPNE) are shown to exist, and form an antichain. Further, for each such MPNE, we can also construct a corresponding stationary Markovian equilibrium invariant distribution. We then show for many versions of our economies found in applied work in macroeconomics, unique MPNE exist relative to the space of bounded measurable functions. We also relate all of our results to those obtained by promised utility/continuation methods based upon the work of Abreu et al. (1990). As our results are constructive, we can provide characterizations of numerical methods for approximating MPNE, and we construct error bounds. Finally, we provide a series of examples to show the potential applications and limitations of our results.

The Costs of Environmental Regulation in a Concentrated Industry

The typical cost analysis of an environmental regulation consists of an engineering estimate of the compliance costs. In industries where fixed costs are an important determinant of market structure this static analysis ignores the dynamic eects of the regulation on entry, investment, and market power. I evaluate the welfare costs of the 1990 Amendments to the Clean Air Act on the US Portland cement industry, accounting for these eects through a dynamic model of oligopoly in the tradition of Ericson and Pakes (1995). Using a recently developed two-step estimator, I recover the entire cost structure of the industry, including the distribution of sunk entry costs and adjustment costs of investment. I find that the Amendments have significantly increased the sunk cost of entry. I solve for the Markov perfect Nash equilibrium (MPNE) of the model and simulate the welfare eects of the Amendments. A static analysis misses the welfare penalty on consumers, and may even obtain the wrong sign of the welfare eects on incumbent firms.

A DYNAMIC GAME OF ENVIRONMENTAL EXPLOITATION BETWEEN TWO COUNTRIES WITH SEQUENTIAL MAXIMIN OBJECTIVES

This paper formulates a dynamic game between the governments of two countries that share a common stock of natural capital (such as environmental quality). The objective of each government is to find a sustainable utility path that satisfies the sequential maximin property. A utility path is a sequential maximin if it survives all successive rounds of eliminating Pareto inferior outcomes based on the maximin criterion. It is shown that, under the sequential maximin objective, there exists a Markov perfect Nash equilibrium in which utility is constant over time. This equilibrium turns out to be Pareto efficient, in sharp contrast to the typical inefficiency of Markov perfect Nash equilibria under the usual discounted utilitarian objective. We also find a Stackelberg equilibrium that results in sustainable development, in which the welfare of the leader is lower (and that of the passive follower is higher) than the symmetric welfare level in the Nash equilibrium.

STATUS-SEEKING AND EXHAUSTIBLE RESOURCE UNDER UNCERTAINTY

This paper is to extend Katayama and Long (2010) to examine the effect of uncertainty on the optimal consumption and resource extraction paths when economic agents are status-seeking. We consider an economy endowed with an exhaustible natural resource and an artificially augmentable capital, both of which are accessible by each agent. The utility of an agent is derived not only from his/her own consumption level but from its relative position in consumption scale among the agents. The resource stock is subject to uncertainty and may increase by newly discovered reservoir or may decrease by, say, water immersion. Two types of uncertainty are modeled: continuous change in stock level over time and jumps in discrete interval of time. The introduction of uncertainty may counteract the property under Markov-perfect Nash equilibrium established by Katayama and Long (2010) that a higher degree of status-seeking leads to greater consumption than in the social optimum.

Carbon Tax and OPEC’s Rents Under a Ceiling Constraint*

AbstractWe study the Markov‐perfect Nash equilibrium (MPNE) of a game between oil‐importing countries, who seek to maintain the atmospheric carbon concentration under a given ceiling, and oil‐exporting countries. The oil‐importing countries set a carbon tax and the oil‐exporting countries control the producer price. We obtain implicit feedback rules and explicit non‐linear time paths of extraction, carbon tax, and producer price. Consumers are always able to reap some share of the scarcity and monopoly rents, whereas producers partially pre‐empt the carbon tax only if the marginal damage under the ceiling is small. We compare the MPNE to the efficient, open‐loop, and cartel‐without‐tax equilibria.

Stationary Markovian Equilibrium in Altruistic Stochastic OLG Models with Limited Commitment

We introduce a new class of in nite horizon altruistic stochastic OLG models with capital and labor, but without commitment between the generations. Under mild regularity conditions, for economies with either bounded or unbounded state spaces, continuous monotone Markov perfect Nash equilibrium (henceforth MPNE) are shown to exist, and form an antichain. Further, for each such MPNE, we can also construct a corresponding stationary Markovian equilibrium invariant distribution. We then show for many versions of our economies found in applied work in macroeconomics, unique MPNE exist relative to the space of bounded measurable functions. We also relate all of our results to those obtained by promised utility/continuation methods based upon the work of Abreu et al. (1990). As our results are constructive, we can provide characterizations of numerical methods for approximating MPNE, and we construct error bounds. Finally, we provide a series of examples to show the potential applications and limitations of our results.

Regulating Environmental Externalities Through Public Firms: A Differential Game

We investigate the possibility of using public firms to regulate polluting emissions in a Cournot oligopoly where production takes place at constant returns to scale and entails a negative environmental externality. We model the problem as a differential game and investigate (i) the Cournot-Nash game among profit-seeking firms; (ii) the Markov Perfect Nash equilibrium under social planning, where the industry output is entirely controlled by a benevolent planner aiming at the maximisation of social welfare; and (iii) the Markov Perfect Nash equilibrium in a mixed setup where at least one firm is public, while the others remain profit-seeking agents. Our analysis identifies the conditions whereby having a mixed market as a regulatory instrument suffices to drive the industry to the same output, externality and social welfare as under planning, both along the optimal path and in steady state.

Stochastic Differential Game of Transboundary Pollution and International Environmental Policy

The purpose of this paper is to investigate a stochastic differential game of transboundary pollution and environmental policy. We considered two asymmetric countries. The production process results in the emission of pollutants that are added to the existing stock common to both countries. The pollution accumulation dynamics is affected by an additive shock, the mean and variance are known. In both countries, the government controls emission levels. Uncertainty is incorporated into the model through a shock. The stock of pollution evolves stochastically according to geometric Brownian motion. In order to maximize the expected net present value of social welfare, the two governments should set the emission levels. We formulate stochastic differential games that need to be solved by the two governments and derive unique Markov-perfect Nash and cooperative equilibrium solutions. Subsequently, we propose a mechanism to derive the payoff distribution procedures of subgame consistent solutions in cooperative stochastic differential games for the two governments, and are able to derive a subgame consistent solution based on the Nash bargaining axioms. In conclusion, we show how the governments agree to maximize the sum of their expected payoffs and divide the total cooperative payoff so that the Nash bargaining outcome is maintained at every instant of time. Moreover, we show conditions guaranteeing individual rationality.JFL Classification: F18, L13, Q58