Infinite-horizon Stochastic Games Research Articles

Cost-aware defense for parallel server systems against reliability and security failures

Parallel server systems in transportation, manufacturing, and computing heavily rely on dynamic routing using connected cyber components for computation and communication. Yet, these components remain vulnerable to random malfunctions and malicious attacks, motivating the need for resilient dynamic routing that is both traffic-stabilizing and cost-efficient. In this paper, we consider a parallel server system with dynamic routing subject to reliability and stability failures. For the reliability setting, we consider an infinite-horizon Markov decision process where the system operator strategically activates a protection mechanism upon each job arrival based on traffic state observations. We prove that an optimal deterministic threshold protecting policy exists based on the dynamic programming recursion of the Hamilton–Jacobi–Bellman equation. For the security setting, we extend the model to an infinite-horizon stochastic game where the attacker strategically manipulates routing assignments. We show that both players follow a threshold strategy at every Markov perfect equilibrium. For both failure settings, we also analyze the stability of the traffic queues. Finally, we develop approximate dynamic programming algorithms to compute the optimal/equilibrium policies and present numerical examples/experiments for validation and illustration.

Relational Contracting, Repeated Negotiations, and Hold-Up

We propose a unified framework to study relational contracting and hold-up problems in infinite horizon stochastic games. We first illustrate that with respect to long run decisions, the common formulation of relational contracts as Pareto-optimal public perfect equilibria is in stark contrast to fundamental assumptions of hold-up models. We develop a model in which relational contracts are repeatedly newly negotiated during relationships. Negotiations take place with positive probability and cause bygones to be bygones. Traditional relational contracting and hold-up formulations are nested as opposite corner cases. Allowing for intermediate cases yields very intuitive results and sheds light on many plausible trade-offs that do not arise in these corner cases. We establish a general existence result and a tractable characterization for stochastic games in which money can be transferred.

We Can Cooperate Even When the Monitoring Structure Will Never Be Known

This paper considers infinite-horizon stochastic games with hidden states and hidden actions. The state changes over time, players observe only a noisy public signal about the state each period, and actions are private information. In this model, uncertainty about the monitoring structure does not disappear. We show how to construct an approximately efficient equilibrium in a repeated Cournot game. Then we extend it to a general case and obtain the folk theorem using ex-post equilibria under a mild condition.

Stochastic Games with Hidden States, Second Version

This paper studies infinite-horizon stochastic games in which players observe payoffs and noisy public information about a hidden state each period. Public randomization is available. We find that very generally, the feasible and individually rational payoff set is invariant to the initial prior about the state in the limit as the discount factor goes to one. We also provide a re-cursive characterization of the equilibrium payoff set and establish the folk theorem.

A Strategic Dynamic Programming Method for Studying Short-Memory Equilibria of Stochastic Games with Uncountable Number of States

We study a class of infinite horizon stochastic games with uncountable number of states. We first characterize the set of all (nonstationary) short-term (Markovian) equilibrium values by developing a new (Abreu et al. in Econometrica 58(5):1041–1063, 1990)-type procedure operating in function spaces. This (among others) proves Markov perfect Nash equilibrium (MPNE) existence. Moreover, we present techniques of MPNE value set approximation by a sequence of sets of discretized functions iterated on our approximated APS-type operator. This method is new and has some advantages as compared to Judd et al. (Econometrica 71(4):1239–1254, 2003), Feng et al. (Int Econ Rev 55(1):83–110, 2014), or Sleet and Yeltekin (Dyn Games Appl doi: 10.1007/s13235-015-0139-1 , 2015). We show applications of our approach to hyperbolic discounting games and dynamic games with strategic complementarities.

A Strategic Dynamic Programming Method for Studying Short Memory Equilibria of Stochastic Games with Uncountable Number of States

We study a class of infinite horizon stochastic games with uncountable number of states. We first characterize the set of all (nonstationary) short-term (Markovian) equilibrium values by developing a new Abreu, Pearce, and Stacchetti (1990) type procedure operating in function spaces. This (among others) proofs Markov Perfect Nash Equilibrium (MPNE) existence. Moreover we present techniques of MPNE value set approximation by a sequence of sets of discretized functions iterated on our approximated APS-type operator. This method is new and has some advantages as compared to Judd, Yeltekin, and Conklin (2003), Feng, Miao, Peralta-Alva, and Santos (2014) or Sleet and Yeltekin (2015). We show applications of our approach to hyperbolic discounting games and dynamic games with strategic complementarities.

Stochastic Games with Hidden States

This paper studies infinite-horizon stochastic games in which players observe noisy public information about a hidden state each period. We find that if the game is connected, the limit feasible payoff set exists and is invariant to the initial prior about the state. Building on this invariance result, we provide a recursive characterization of the equilibrium payoff set and establish the folk theorem. We also show that connectedness can be replaced with an even weaker condition, called asymptotic connectedness. Asymptotic connectedness is satisfied for generic signal distributions, if the state evolution is irreducible.

Relational Contracting, Repeated Negotiations, and Hold-Up

We propose a unified framework to study relational contracting and hold-up problems in infinite horizon stochastic games. We first illustrate that with respect to long run decisions, the common formulation of relational contracts as Pareto-optimal public perfect equilibria is in stark contrast to fundamental assumptions of hold-up models. We develop a model in which relational contracts are repeatedly newly negotiated during relationships. Negotiations take place with positive probability and cause bygones to be bygones. Traditional relational contracting and hold-up formulations are nested as opposite corner cases. Allowing for intermediate cases yields very intuitive results and sheds light on many plausible trade-offs that do not arise in these corner cases. We establish a general existence result and a tractable characterization for stochastic games in which money can be transferred.

Markov Stationary Equilibria in Stochastic Supermodular Games with Imperfect Private and Public Information

We study a class of discounted, infinite horizon stochastic games with public and private signals and strategic complementarities. Using monotone operators defined on the function space of values and strategies (equipped with a product order), we prove existence of a stationary Markov–Nash equilibrium via constructive methods. In addition, we provide monotone comparative statics results for ordered perturbations of our space of games. We present examples from industrial organization literature and discuss possible extensions of our techniques for studying principal-agent models.

Markov Stationary Equilibria in Stochastic Supermodular Games with Imperfect Private and Public Information

We study a class of discounted, infinite horizon stochastic games with public and private signals and strategic complementarities. Using monotone operators defined on the function space of values and strategies (equipped with a product order), we prove existence of a Stationary Markov Nash Equilibrium via constructive methods. In addition, we provide monotone comparative statics results for ordered perturbations of our space of stochastic games. We present examples from industrial organization literature and discuss possible extensions of our techniques for studying principal-agent models.

Pure strategy Markov equilibrium in stochastic games with a continuum of players

We discuss stochastic games which are played by a continuum of players. The player space is given by the atomless measure space of a unit interval with the Lebesgue measure. The state space in period t is given by ( S, Σ, μ t ), where S is a complete, separable metric space, Σ is the Borel sigma algebra and μ t is a fixed probability measure. There is a common action space K which is either a finite set or the K-ball of an Euclidean space R n . The transition probabilities in period t are product measurable in the realizations of the state and average responses and norm continuous in the current average response of the players and absolutely continuous with respect to the measure μ t . The payoffs of the players in these infinite horizon stochastic games are the discounted sum of the payoffs received every period. We show that these stochastic games have pure strategy Markov equilibrium points. In the process we are able to provide a characterization of Markov equilibrium strategy as the equilibrium strategies of a sequence of single-period constituent games.

Risk-Sensitive Dynamic Market Share Attraction Games

We present partial results showing that risk-sensitive oligopolists would spend less on advertising than would their risk-neutral counterparts. The model is an infinite-horizon stochastic game in which each firm's “goodwill” is a random function of both its own and its competitors' current and past advertising expenditures. Single-period firm profits have a market share attraction form. Each firm seeks to maximize its expected exponential utility of the sum of discounted profits. We analyze the impact that risk sensitivity and other parameters have on equilibrium advertising strategies by exploiting the special structure of the stochastic game model.Journal of Economic LiteratureClassification Number: C73.