Markov Equilibria Research Articles

Institution Building without Commitment

We propose a theory of gradualism in the implementation of good policies, suitable for environments featuring time consistency. We downplay the role of the initial period by allowing agents both to wait for future agents to start equilibrium play and to restart the equilibrium by ignoring past history. The allocation gradually transits toward one that weighs both short- and long-term concerns, stopping short of the Ramsey outcome but greatly improving upon Markovian equilibria. We use the theory to account for the slow emergence of both climate policies and the reduction of global tariff rates. (JEL C73, E21, E61, F13, H30)

Stationary Markov Equilibrium Strategies in Asynchronous Stochastic Games: Existence and Computation

We study Asynchronous Dynamic games and show that in games with a finite state space and finite action sets, one can obtain the pure strategy Markov perfect equilibrium by using a simple backward induction method when the time period for the game is finite. The equilibrium strategies for games with an infinite horizon are then obtained as the point-wise limit of the equilibrium strategies of a sequence of finite horizon games, where the finite horizon games are truncated versions of the original game with successively longer time periods. We also show that if the game has a fixed K-period cycle, then there is a stationary Markov equilibrium. Using these results, we derive an algorithm to compute the equilibrium strategies. We test the algorithm in three experiments. The first is a two-player asynchronous game with three states and three actions. In the second experiment, we compute the equilibrium of a cybersecurity game in which there are two players, an attacker and a defender. In the third experiment, we compute the stationary equilibrium of a duopoly game with two firms that choose an output in alternate periods.

Markov equilibrium of social security: An analytic solution under CRRA utility and the future of social security

The politico-economic sustainability of pay-as-you-go social security has been studied with the help of two-period overlapping generation models under Markovian equilibrium concepts. The literature has mostly used logarithmic utility for tractability, whereas versions under the more general isoelastic CRRA utility have only been analyzed computationally. This paper demonstrates that a prototypical social security model under CRRA utility has a closed-form solution; it revisits previously obtained computational conclusions and presents new findings and extensions. Quantitatively, the model can explain the observed path of payroll tax rates in the USA and predicts increased tax rates as the equilibrium response to lower future population growth rates. The model predicts a gradual increase in tax rates, from the current 12.4% to about 15% or so in 2060, and then staying at that level. Benefits are expected to decrease slowly, by about 2% (or less) per decade, starting in 2030.

Short Communication: Existence of Markov Equilibrium Control in Discrete Time

Short Communication: Existence of Markov Equilibrium Control in Discrete Time

Accelerating equilibrium spin-glass simulations using quantum annealers via generative deep learning

Adiabatic quantum computers, such as the quantum annealers commercialized by D-Wave Systems Inc., are routinely used to tackle combinatorial optimization problems. In this article, we show how to exploit them to accelerate equilibrium Markov chain Monte Carlo simulations of computationally challenging spin-glass models at low but finite temperatures. This is achieved by training generative neural networks on data produced by a D-Wave quantum annealer, and then using them to generate smart proposals for the Metropolis-Hastings algorithm. In particular, we explore hybrid schemes by combining single spin-flip and neural proposals, as well as D-Wave and classical Monte Carlo training data. The hybrid algorithm outperforms the single spin-flip Metropolis-Hastings algorithm. It is competitive with parallel tempering in terms of correlation times, with the significant benefit of a much shorter equilibration time.

Learning from Manipulable Signals

We study a dynamic stopping game between a principal and an agent. The principal gradually learns about the agent's private type from a noisy performance measure that can be manipulated by the agent via a costly and hidden action. We fully characterize the unique Markov equilibrium of this game. We find that terminations/market crashes are often preceded by a spike in manipulation intensity and (expected) performance. Moreover, due to endogenous signal manipulation, too much transparency can inhibit learning and harm the principal. As the players get arbitrarily patient, the principal elicits no useful information from the observed signal. (JEL C73, D82, D83, G24, M13)

Timing and time inconsistency in search models

This paper studies how the timing assumption matters for the commitment problem and time inconsistency issue in search models with unemployment insurance. We analyze the Markov equilibrium without commitment and the Ramsey policy with commitment under two different timing assumptions. In the first timing, consumption takes place before search within each period; in the second, search takes place before consumption. Time inconsistency occurs mainly through search disincentive under the first timing, but only indirectly under the second timing. We show theoretically and numerically that the magnitude of time inconsistency is stronger under the first timing. • The timing assumption matters for the time inconsistency issue in search models. • A government’s commitment problem is analyzed under two different timings. • Time inconsistency is stronger when consumption takes place before search.

Implementing the commitment solution via discretionary policy-making

AbstractThis paper demonstrates that, in a large class of linear-quadratic models with rational expectations, losses due to time-inconsistency problems can be avoided, as the commitment solution can be implemented by a policy-maker who acts under discretion. We focus on two approaches. First, we show that a non-Markovian, reputational equilibrium that implements the commitment solution always exists. Second, we show how delegation to a policy-maker with an additional objective for the policy instrument can be used to implement the commitment solution via a standard discretionary Markov equilibrium. Implementation is facilitated by the fact that the commitment outcome can be attained irrespective of the weight that the policy-maker assigns to the additional target. Using the standard new Keynesian model as an example, we study the dynamics of the economy under optimal additional output targets as well as optimal interest-rate targets for central banks.

Layered Networks, Equilibrium Dynamics, and Stable Coalitions

An important aspect of network dynamics that has been missing from our understanding of network dynamics in various applied settings is the influence of strategic behavior in determining equilibrium network dynamics. Our main objective hear to say what we can regarding the emergence of stable club networks—and therefore, stable coalition structures—based on the stability properties of strategically determined equilibrium network formation dynamics. Because club networks are layered networks, our work here can be thought of as a first work on the strategic dynamics of layered networks. In addition to constructing a discounted stochastic game model (i.e., a DSG model) of club network formation, (1) we show that our DSG of network formation possesses a stationary Markov perfect equilibrium in players’ membership-action strategies; (2) we identify the assumptions on primitives which ensure that the induced equilibrium Markov process of layered club network formation satisfies the Tweedie Stability Conditions (Tweedie in Stoch Process Appl 92:345–354, 2001) and (3) we show that, as a consequence, the equilibrium Markov network formation process generates a unique decomposition of the set of state-network pairs into a transient set together with finitely many basins of attraction. Moreover, we show that if there is a basin containing a vio set (a visited infinitely often set) of club networks sufficiently close together, then the coalition structures across club networks in the vio set will be the same (i.e., closeness across networks in a vio set leads to invariance in coalition structure across networks in a vio set).

Social security as Markov equilibrium in OLG models: Clarifications and some new insights

This paper studies the politico-economic sustainability of pay-as-you-go social security in OLG models under Markovian strategies as first studied by Forni (2005). Under logarithmic utility, the paper shows that equilibria with social security can only exist if the underlying economy is dynamically inefficient. The paper also derives the exact parametric conditions that allow for the existence of equilibria and shows that among all the admissible (arbitrary) constants that produce a Markov perfect equilibrium, the maximum constant in such set yields the only equilibrium that solves dynamic inefficiency.

Time-consistent equilibria in dynamic models with recursive payoffs and behavioral discounting

We prove existence of time consistent equilibria in a class of dynamic models with recursive payoffs and generalized discounting involving both behavioral and normative applications. Our generalized Bellman equation method identifies and separates both: recursive and strategic aspects of the equilibrium problem and allows to determine the sufficient assumptions on preferences and stochastic transition to establish existence. In particular we show existence of minimal state space stationary Markov equilibrium (a time-consistent equilibrium) in a deterministic model of consumption-saving with beta-delta discounting and its generalized versions involving non-additive payoffs, general form certainty equivalents, as well as stochastic semi-hyperbolic and hyperbolic discounting models (over possibly unbounded state and unbounded above reward space). We also provide an equilibrium approximation method for a hyperbolic discounting model.

Retracted] Balance Harvest from the Forest as a Renewable Resource Using Game Theory

Forests are among the most important renewable natural resources. While having a great range of benefits, the crucial advantage of forests is the production of oxygen and the absorption of pollutants. Thus, the current article attempts to examine forest exploitation as a source of wood to prevent forest overexploitation. Three smugglers, residents, and timber companies exploit this resource. There is a competition for the perception of this common source between these factors. In this article, these interactions are considered a game in order to obtain the most probable solution in the real competition between these factors by examining this game. This answer is the balance of the game. In order to use this answer, the behavior of the players can be predicted and used in management planning. Since the amount of withdrawals of operating companies depends on the current inventory (t), the player’s behavior is determined by the system’s current state and then Markov equilibrium was obtained for them.

Social Security as Markov Equilibrium in OLG Models: Clarifications and New Insights

Social Security as Markov Equilibrium in OLG Models: Clarifications and New Insights

A dynamic theory of spatial externalities

We characterize the shape of spatial externalities in a continuous time and space differential game with transboundary pollution. We posit a realistic spatiotemporal law of motion for pollution (diffusion and advection), and tackle spatiotemporal non-cooperative (and cooperative) differential games. Precisely, we consider a circle partitioned into several states where a local authority decides autonomously about its investment, production and depollution strategies over time knowing that investment/production generates pollution, and pollution is transboundary. The time horizon is infinite. We allow for a rich set of geographic heterogeneities across states. We solve analytically the induced non-cooperative differential game and characterize its long-term spatial distributions. In particular, we prove that there exist a Perfect Markov Equilibrium, unique among the class of the affine feedbacks. We further provide with a full exploration of the free riding problem and the associated border effect.

A Case Study on Stochastic Games on Large Graphs in Mean Field and Sparse Regimes

We study a class of linear-quadratic stochastic differential games in which each player interacts directly only with its nearest neighbors in a given graph. We find a semiexplicit Markovian equilibrium for any transitive graph, in terms of the empirical eigenvalue distribution of the graph’s normalized Laplacian matrix. This facilitates large-population asymptotics for various graph sequences, with several sparse and dense examples discussed in detail. In particular, the mean field game is the correct limit only in the dense graph case, that is, when the degrees diverge in a suitable sense. Although equilibrium strategies are nonlocal, depending on the behavior of all players, we use a correlation decay estimate to prove a propagation of chaos result in both the dense and sparse regimes, with the sparse case owing to the large distances between typical vertices. Without assuming the graphs are transitive, we show also that the mean field game solution can be used to construct decentralized approximate equilibria on any sufficiently dense graph sequence.

Cricket mix optimization using heuristic framework after ensuring Markovian equilibrium

International Cricket Council, in consultation with its member boards, prepares the Future Tours and Programme (FTP) which is an eight year long itinerary covering world championships in three formats of cricket, bilateral series and other tournaments. However, the FTP (2015–2023) had been criticized for its asymmetric itinerary and the point system for World Test championship and the FTP (2023–2031) is being criticized for including eight championships in limited formats and enhanced workload for players. Cricket mix standardization like marketing and product mix can work in homogeneous markets. This study derives three homogeneous markets of four teams each using hierarchical cluster analysis. For each market, it finds out the Markovian equilibrium analyzing cricket mix transition over past years. While the same can be used to derive the number of games per format per country, the study proposes a heuristic approach for fine tuning the same taking care of major stakeholders’ (e.g. Administrators, Players and Spectators) aspirations. Despite scores of criticisms and articles on the issue, there is hardly any scholastic contribution on game scheduling in the extant literature. This study thus is a pioneering effort in helping the policy makers to create a balance between cricket formats within each homogeneous market.

Advertising patterns in a dynamic oligopolistic growing market with decay

A finite-horizon Lanchester model of a (continuous-time) differential game of oligopolistic advertising is considered, and the analytical form of the unique closed-loop Nash equilibrium derived and analyzed. In contrast to previous research, the finite-horizon Lanchester model is modified to include two novel factors. First, a growing market allows us to analyze the competition for a potential market via generic advertising, with the latter giving the otherwise zero-sum-like game a public good dimension. Second, it is assumed that each firm’s market share declines in the absence of advertising efforts. The analysis investigates the Markovian (closed-loop) equilibrium calling for firms operating in a competitive growing market to invest in offensive, defensive or generic advertising in all or part of the decision horizon. In the most novel part of the paper, the non-cooperative outcome is compared to the Pareto-optimal or cooperative solution. The conclusions derived reveal significant differences in the resulting patterns of co-existence of the different dimensions of advertising over time, relative to the existing literature.

Some Results on the Control of Polluting Firms According to Dynamic Nash and Stackelberg Patterns

In this paper we model the conflict between the group of polluting firms in a country and any social planner in the same country who attempts to control the volume of emissions generated during the production process. Both players of the game have their own control policies, i.e., the rate of emissions on behalf of the polluting firms and the rate of pollution control (e.g., pollution abatement or environmental taxation) on behalf of the home country. The common state variable of the model is the number of polluting firms, which aims to be minimized via the country’s control policy, but on the polluters’ side it is beneficial to be maximized. Regarding the game model, its setup belongs to the special class of differential games, which are called ‘state separable differential games’. An important property of these games is that the open-loop Nash equilibrium coincides with the Markovian (closed-loop) equilibrium and, in the case of hierarchical moves, analytical solutions are easily obtained. The game proposed here is analyzed for both types of equilibrium, i.e., Nash and Stackelberg. In the simultaneous move game (i.e., the Nash game) we find the equilibrium’s analytical expressions of the controls for both players, as well as the stationary value of the stock of polluting firms. A sensitivity analysis of the model’s crucial variables takes place. In the hierarchical move game (i.e., the Stackelberg game) we find the equilibrium values of the controls, as well as of the state variable. As a result, a comparison between the two types of equilibrium for the game takes place. The analysis of the comparison reveals that the conflict is more intensive (since both controls have greater values) for the case in which the polluting firms act as the leader in the hierarchical move game.

Reputation and Sovereign Default

This paper presents a continuous‐time model of sovereign debt. In it, a relatively impatient sovereign government's hidden type switches back and forth between a commitment type, which cannot default, and an opportunistic type, which can, and where we assume outside lenders have particular beliefs regarding how a commitment type should borrow for any given level of debt and bond price. In any Markov equilibrium, the opportunistic type mimics the commitment type when borrowing, revealing its type only by defaulting on its debt at random times. The equilibrium features a “graduation date”: a finite amount of time since the last default, after which time reputation reaches its highest level and is unaffected by not defaulting. Before such date, not defaulting always increases the country's reputation. For countries that have recently defaulted, bond prices and the total amount of debt are increasing functions of the amount of time since the country's last default. For countries that have not recently defaulted (i.e., those that have graduated), bond prices are constant.

A characterization of Markov equilibrium in stochastic overlapping generations models

This paper characterizes the Markov equilibrium in stochastic overlapping generations models with both pure exchange and production using the theory of linear iterated function systems. We provide conditions for the linearized Markov equilibrium to have an absolutely continuous invariant measure or a singular invariant measure. We also examine relationships between the linear equilibrium system and the model parameters. This analysis identifies the set of economies for each type of invariant measure. We show that the attractor of the singular invariant measure exhibits fractal patterns, which supports the proposition that self-affinity patterns in financial data can be explained via rational expectations equilibrium as an economic mechanism.