Stationary Markov Perfect Equilibria Research Articles

Optimal investor life cycle decisions with time-inconsistent preferences

We examine the lifetime decisions of an investor with time-inconsistent preferences, obtaining optimal time-consistent strategies for investment, consumption, and life insurance premiums explicitly under a stationary Markov perfect equilibrium framework. We find that the investor increases consumption to seek immediate gratification, and simultaneously increases life insurance purchasing to fulfill a legacy need. However, at a later stage in the life cycle, the investor confronts slower wealth accumulation and reduces consumption and life insurance purchasing accordingly. We also investigate the effects of naivete, degree of time inconsistency, risk preferences, legacy weight, insurance cost, death duty, and labor income on the investor's decision making. Our study reveals the balance of consumption, bequest, and terminal wealth motives in life cycle decisions featuring microfounded bequest utility and stochastic labor income.

Layered Networks, Equilibrium Dynamics, and Stable Coalitions

An important aspect of network dynamics that has been missing from our understanding of network dynamics in various applied settings is the influence of strategic behavior in determining equilibrium network dynamics. Our main objective hear to say what we can regarding the emergence of stable club networks—and therefore, stable coalition structures—based on the stability properties of strategically determined equilibrium network formation dynamics. Because club networks are layered networks, our work here can be thought of as a first work on the strategic dynamics of layered networks. In addition to constructing a discounted stochastic game model (i.e., a DSG model) of club network formation, (1) we show that our DSG of network formation possesses a stationary Markov perfect equilibrium in players’ membership-action strategies; (2) we identify the assumptions on primitives which ensure that the induced equilibrium Markov process of layered club network formation satisfies the Tweedie Stability Conditions (Tweedie in Stoch Process Appl 92:345–354, 2001) and (3) we show that, as a consequence, the equilibrium Markov network formation process generates a unique decomposition of the set of state-network pairs into a transient set together with finitely many basins of attraction. Moreover, we show that if there is a basin containing a vio set (a visited infinitely often set) of club networks sufficiently close together, then the coalition structures across club networks in the vio set will be the same (i.e., closeness across networks in a vio set leads to invariance in coalition structure across networks in a vio set).

Equilibrium in a Symmetric Game of Resource Extraction with Coalitional Structure

The game of resource extraction / capital accumulation is a stochastic nonzero-sum infinite horizon game, obtained as an extension of the well-known optimal growth model to m strategically competing players, who jointly posses a renewable resource. The existence of a Nash equilibrium in different, often symmetric, frameworks of the game received a significant attention in the scientific literature on the topic. The focus of this paper is to introduce the coalitional component to the symmetric problem. Specifically, we examine whether the game with a fixed coalitional structure admits stability against profitable coalitional deviations.It is assumed that the set of all players is partitioned into coalitions which do not intersect and remain consistent throughout the game. The members of each coalition are able to coordinate their actions and perform joint deviations in a cooperative manner. Such setting incorporates a natural concept of established social ties, which may reflect a potential context appearing in practical applications. The corresponding notion of equilibrium in the paper is expressed as a position, from which none of the set coalitions can deviate in a manner to increase a total reward of its members. Its existence is studied in the context of a certain symmetric resource extraction game model with unbounded utilities of the players. This model was studied in [12; 13], concluding a Stationary Markov Perfect Equilibrium existence in both symmetric and non-symmetric game structure. The first feature of the model is that the preferences of the players are considered to be isoelastic in the form of strictly concave power functions. Furthermore, the law of motion between states is set to follow a geometric random walk in relation to players' joint investments. We prove that the game within the formulated settings admits stability against profitable coalitional deviations for any partition on the set of agents. The method provides an algorithm for building the corresponding stationary strategies, which can be useful for practical purposes. Finally, we use two examples with different numerical configurations to illustrate possible patterns of how the individual rewards of the players vary depending on a coalitional structure, which is set at the beginning of the game.

K-Correspondences, USCOs, and fixed point problems arising in discounted stochastic games

We establish a fixed point theorem for the composition of nonconvex, measurable selection valued correspondences with Banach space valued selections. We show that if the underlying probability space of states is nonatomic and if the selection correspondences in the composition are K-correspondences (meaning correspondences having graphs that contain their Komlos limits), then the induced measurable selection valued composition correspondence takes contractible values and therefore has fixed points. As an application we use our fixed point result to show that all nonatomic uncountable-compact discounted stochastic games have stationary Markov perfect equilibria – thus resolving a long-standing open question in game theory.

Representative Voting Games

We propose the stationary Markov perfect equilibria of representative voting games as a benchmark to evaluate the outcomes of dynamic elections, in which the evolution of voters’ political power is endogenous. We show that the equilibria of dynamic elections can achieve this benchmark if politicians are sufficiently office motivated. For arbitrary equilibria of the electoral model, we characterize the faithfulness of politicians’ choices to the policy objectives of representative voters through a delegated best-response property. Finally, we provide conditions under which general dynamic electoral environments admit representative voters in each state.

A Discounted Stochastic Multiplayer Game Approach for Vehicle-to-Grid Voltage Regulation

Along with the diversification of electricity market, the voltage regulation (VR) service is opened up to various qualified providers to meet the enormous demand, among which, the electric vehicle (EV) aggregators (AGGs) can integrate the scattered EVs and play the role of VR sources with a high response speed and low cost. In order to coordinate the VR service providers to satisfy the total VR requirement, auction mechanisms are widely used to select VR sources to achieve better performance. Yet, it is challenging to optimize the strategy of each EV AGG in the auction process due to the stochastic EV mobility, various distribution network topology, and the competition mechanism. To address these challenges, we proposed a discounted stochastic multiplayer game (DSMG) approach to analyze the competition among EV AGGs. Due to the constraint of distribution network topology, the efficiency of the VR sources at different locations can be different. Thus, the impact of distribution network topology on the VR efficiency is investigated by DSO when evaluating the capacity of AGGs. The randomness of EV numbers is considered when predicting the AGGs’ available VR capacity so that the tendency for the AGGs to follow the optimal strategies can be modeled accurately. Accordingly, a linear power flow analysis approach and a battery pool model are developed to address the distribution network topology and EV mobility, respectively. Then, the DSMG approach is used in the VR auction process to optimize the AGGs’ strategies. The existence proof of the stationary Markov perfect equilibrium is presented, and the corresponding algorithms to obtain the equilibrium is proposed. The performance of the proposed DSMG approach is evaluated and compared with other approaches based on the IEEE 33-bus test feeder, IEEE 123-bus test feeder, and the real-world generation and load data from PVWatts Calculator and Market Analysis and Information System, respectively.

Simple Markovian equilibria in dynamic spatial legislative bargaining

The paper proves, by construction, the existence of Markovian equilibria in a dynamic spatial legislative bargaining model. Players bargain over policies in an infinite horizon. In each period, a sequential protocol of proposal-making and voting, with random proposer recognitions and a simple majority, produces a policy that becomes the next period's status-quo; the status-quo is endogenous. The construction relies on simple strategies determined by strategic bliss points computed by the algorithm we provide. A strategic bliss point, the dynamic utility ideal, is a moderate policy relative to a bliss point, the static utility ideal. Moderation is strategic and germane to the dynamic environment; players moderate in order to constrain the future proposals of opponents. Moderation is a strategic substitute; when a player's opponents do moderate, she does not, and when they do not moderate, she does. We provide conditions under which the simple strategies induced by the strategic bliss points computed by the algorithm deliver a Stationary Markov Perfect equilibrium, and we prove its existence in generic games with impatient players and in symmetric games. Because the algorithm constructs all equilibria in simple strategies, we provide their general characterization, and we show their generic uniqueness.

Existence of Nash equilibria in stochastic games of resource extraction with risk-sensitive players

We consider a two-person stochastic game of resource extraction. It is assumed that players have identical preferences. A novelty relies on the fact that each player is equipped with the same risk coefficient and calculates his discounted utility in the infinite time horizon in a recursive way by applying the entropic risk measure parametrized by this risk coefficient. Under two alternative sets of assumptions, we prove the existence of a symmetric stationary Markov perfect equilibrium.

Markov perfect equilibria in a dynamic decision model with quasi-hyperbolic discounting

We study a discrete-time non-stationary decision model in which the preferences of the decision maker change over time and are described by quasi-hyperbolic discounting. A time-consistent optimal solution in this model corresponds with a Markov perfect equilibrium in a stochastic game with uncountable state space played by countably many short-lived players. We show that Markov perfect equilibria may be constructed using a generalized policy iteration algorithm. This method is in part inspired by the fundamental works of Mertens and Parthasarathy (in: Raghavan, Ferguson, Parthasarathy, Vrieze (eds) Stochastic games and related topics, Kluwer Academic Publishers, Dordrecht, 1991; in: Neyman, Sorin (eds) Stochastic games and applications, Academic Publishers, Dordrecht, 2003) devoted to subgame perfect equilibria in standard n-person discounted stochastic games. If the one-period utilities and transition probabilities are independent of time, we obtain on new existence results on stationary Markov perfect equilibria in the models with unbounded from above utilities.

On symmetric stochastic games of resource extraction with weakly continuous transitions

We study stochastic games of resource extraction, in which the players have identical preferences. The transition probability is either non-atomic or a convex combination of transition probabilities depending on the investment with coefficients also dependent on the investment. Our approach covers the unbounded utility case, which was not examined in this class of games beforehand. We prove the existence of a stationary Markov perfect equilibrium in a non-randomised class of strategies.

Stationary Almost Markov Perfect Equilibria in Discounted Stochastic Games

In this paper, we study discounted stochastic games with Borel state and compact action spaces depending on the state variable. The primitives of our model satisfy standard continuity and measurability conditions. The transition probability is a convex combination of finitely many probability measures depending on states, and it is dominated by some finite measure on the state space. The coefficients of the combination depend on both states and action profiles. This class of models contains stochastic games with Borel state spaces and finite, state-dependent action sets. Our main result establishes the existence of subgame perfect equilibria, which are stationary in the sense that the equilibrium strategy for each player is determined by a single function of the current and previous states of the game. This dependence is called almost Markov. Our result enhances both the theorem of Mertens and Parthasarathy established in 1991 for games with finite, state-independent action sets, where the equilibrium strategies were also depended on the calendar time, and their result on stationary equilibria proved under an additional condition that the transition probabilities are atomless. A counterexample given very recently by Levy shows that stationary Markov perfect equilibria may not exist in the class of games considered in this paper. The presented results in this work are illustrated by the Cournot dynamic games, which were already considered in the literature under much stronger assumptions.

Stochastic bequest games

In this paper, we prove the existence of a stationary Markov perfect equilibrium for a stochastic version of the bequest game. A novel feature in our approach is the fact that the transition probability need not be non-atomic and therefore, the deterministic production function is not excluded from consideration. Moreover, in addition to the common expected utility we also deal with a utility that takes into account an attitude of the generation towards risk.

Bargaining over an endogenous agenda

We present a model of bargaining in which a committee searches over the policy space, successively amending the default by voting over proposals. Bargaining ends when proposers are unable or unwilling to amend the existing default, which is then implemented. Our main goal is to study the policies that can be implemented from any initial default in a pure-strategy stationary Markov perfect equilibrium for an interesting class of environments including multidimensional and infinite policy spaces. It is convenient to start by characterizing the set of immovable policies that are implemented, once reached as default. These policies form a weakly stable set and, conversely, any weakly stable set is supported by some equilibrium. Using these results, we show that minimum-winning coalitions may not form and that a player who does not propose may nevertheless earn all of the surplus from agreement. We then consider how equilibrium outcomes change as we vary the order in which players propose, the identity of proposers, and the set of winning coalitions. First, if the policy space is well ordered, then the committee implements the ideal policy of the last proposer in a subset of a weakly stable set, but this result does not generalize to other cases. We also show, surprisingly, that a player may prefer not to be given the opportunity to propose and that the set of immovable policies may shrink as the quota increases. Finally, we derive conditions under which immovable policies in semi-Markovian equilibria form a consistent choice set.

Stationary Markov perfect equilibria in risk sensitive stochastic overlapping generations models

In this paper, we study intergenerational stochastic games that can be viewed as a special class of overlapping generations models under uncertainty. Making use of the theorem of Dvoretzky, Wald and Wolfowitz [27] from the statistical decision theory, we obtain new results on stationary Markov perfect equilibria for the aforementioned games, with a general state space, satisfying rather mild continuity and compactness conditions. A novel feature of our approach is the fact that we consider risk averse generations in the sense that they aggregate partial utilities using an exponential function. As a byproduct, we also provide a new existence theorem for intergenerational stochastic game within the standard framework where the aggregator is linear. Our assumptions imposed on the transition probability and utility functions allow to embrace a pretty large class of intergenerational stochastic games analysed recently in macroeconomics. Finally, we formulate a set of assumptions under which the stochastic process induced by the stationary Markov perfect equilibrium possesses an invariant distribution.

Simple Markovian Equilibria in Dynamic Spatial Legislative Bargaining

The paper proves, by construction, the existence of Markovian equilibria in a model of dynamic spatial legislative bargaining. Players bargain over policies in an infinite horizon. In each period, a majority vote takes place between the proposal of a randomly selected player and the status-quo, the policy last enacted. This determines the policy outcome that carries over as the status-quo in the following period; the status-quo is endogenous. Proposer recognition probabilities are constant and discount factors are homogeneous. The construction relies on simple strategies determined by strategic bliss points computed by the algorithm we provide. A strategic bliss point is the policy maximizing the dynamic utility of a player with ample bargaining power. Relative to a bliss point, the static utility ideal, a strategic bliss point is a moderate policy. Moderation is strategic and germane to the dynamic environment; players moderate in order to constrain the future proposals of opponents. Moderation is a strategic substitute; when a player's opponents do moderate, she does not, and when they do not moderate, she does. We prove that the simple strategies induced by the strategic bliss points computed by the algorithm deliver a Stationary Markov Perfect equilibrium. Thus we prove its existence in a large class of symmetric games with more than three players and (possibly with slight adjustment) in any three-player game. Because the algorithm constructs all equilibria in simple strategies, we provide their general characterization, and we show their generic uniqueness. Finally, we analyse how the degree of moderation changes with changes in the model parameters, and we discuss the dynamics of the equilibrium policies.

Two-sided intergenerational transfer policy and economic development: A politico-economic approach

We consider an overlapping generations model with public education and social security financed by labor income taxation, in which the overall size of these policies is determined in a repeated majority voting game. We investigate the interaction between these policies and economic development in stationary Markov perfect equilibria. In the politico-economic equilibrium, the labor income tax rate is represented as a linear increasing function of the ratio of the decisive voter's human capital and the average human capital level. A high level of initial income inequality reduces the size of public policies and retards economic growth.

Noisy Stochastic Games

This paper establishes existence of a stationary Markov perfect equilibrium in general stochastic games with noise—a component of the state that is nonatomically distributed and not directly affected by the previous period's state and actions. Noise may be simply a payoff-irrelevant public randomization device, delivering known results on the existence of correlated equilibrium as a special case. More generally, noise can take the form of shocks that enter into players' stage payoffs and the transition probability on states. The existence result is applied to a model of industry dynamics and to a model of dynamic electoral competition.

Preemptive Capacity Investment Under Uncertainty

The incentive to \overinvest in capital may be eroded in dynamic, competitive settings if rms face uncertainty and irreversibility. In this paper, we derive the stationary Markov perfect equilibrium for a dynamic, innite-horizon capacity investment game formulated in continuous time in which reduced-form prots are subject to industry shocks. We show that the theory of marginal Tobin’s

Markov Perfect Equilibrium Advertising Strategies of Lanchester Duopoly Model: A Technical Note

We propose a numerical approach to compute stationary Markov perfect Nash equilibrium advertising strategies of the Lanchester model. The algorithm can be implemented using a standard mathematical package, and, importantly, it does not require that the players discount their future earnings at a zero rate, an assumption that has been made in the literature.

Consistent planning under quasi-geometric discounting

We study properties of stationary Markov-perfect equilibria in a general model of intertemporal choice under quasi-geometric discounting. The dynamics generated by stationary Markov-perfect equilibria can be very complicated, even if the model satisfies strict convexity and smoothness properties and the decision maker is arbitrarily patient. If there exist multiple stationary Markov-perfect equilibria, then it is in general possible to construct infinitely many non-degenerate stationary sunspot equilibria as well.