Continuous-time Stochastic Games Research Articles

Nonzero-Sum Risk-Sensitive Continuous-Time Stochastic Games with Ergodic Costs

We study nonzero-sum stochastic games for continuous time Markov decision processes on a denumerable state space with risk-sensitive ergodic cost criterion. Transition rates and cost rates are allowed to be unbounded. Under a Lyapunov type stability assumption, we show that the corresponding system of coupled HJB equations admits a solution which leads to the existence of a Nash equilibrium in stationary strategies. We establish this using an approach involving principal eigenvalues associated with the HJB equations. Furthermore, exploiting appropriate stochastic representation of principal eigenfunctions, we completely characterize Nash equilibria in the space of stationary Markov strategies.

Approximation of value function of differential game with minimal cost

The paper is concerned with the approximation of the value function of the zero-sum differential game with the minimal cost, i.e., the differential game with the payoff functional determined by the minimization of some quantity along the trajectory by the solutions of continuous-time stochastic games with the stopping governed by one player. Notice that the value function of the auxiliary continuous-time stochastic game is described by the Isaacs–Bellman equation with additional inequality constraints. The Isaacs–Bellman equation is a parabolic PDE for the case of stochastic differential game and it takes a form of system of ODEs for the case of continuous-time Markov game. The approximation developed in the paper is based on the concept of the stochastic guide first proposed by Krasovskii and Kotelnikova.

Predatory Pricing and the Value of Corporate Cash Holdings

We analyze the interaction between firms' payout policies and their decisions in product markets in a continuous-time stochastic game between two firms. One of these is financially constrained, whereas the other is not. Contrary to the standard literature we allow firms to choose production and payout strategies, and focus on the effect of predation incentives on both. We find that predation induces fewer dividend payouts. Furthermore, the liquidity position of the constrained firm has an economically significant effect on the production choices of both firms and, thus, on the evolution of profits, cash holdings and stock returns.

An Update on Continuous-Time Stochastic Games of Fixed Duration

This paper shows that continuous-time stochastic games of fixed duration need not possess equilibria in Markov strategies. The example requires payoffs and transitions to depend on time in a continuous but irregular (almost nowhere almost differentiable) way. This example offers a correction to the erroneous construction presented previously in Levy (Dyn Games Appl 3(2):279–312, 2013. https://doi.org/10.1007/s13235-012-0067-2).

Nonzero-sum risk-sensitive finite-horizon continuous-time stochastic games

This paper concerns the nonzero-sum games for continuous-time jump processes with unbounded transition rates under the risk-sensitive finite-horizon cost criterion. The state space is a countable set and the costs are allowed to be unbounded in the game model. Under the suitable optimality conditions, by introducing an appropriate topology for the set of all randomized Markov multi-strategies and employing the risk-sensitive finite-horizon optimality equations of the players, we prove the existence of a randomized Markov Nash equilibrium in the class of all randomized history-dependent multi-strategies.

A uniformization-based algorithm for continuous-time stochastic games model checking

Continuous-time Stochastic Game (CTSG) can be seen as a proper model to analyze probabilistic, non-deterministic, and competitive behaviors as found in Stochastic Multi-Player Game (SMG). The difference is that in an SMG the system under design is analyzed in the discrete-time setting. In comparison, in a CTSG, the system transitions occur in the continuous-time setting with exponentially distributed delays. This paper focuses on the model checking of CTSGs and presents a uniformization-based algorithm, called TPA, to approximate the time-bounded reachability probabilities with an arbitrary error bound. We concentrate on CTSGs with bounded transition rates. To illustrate the strength points of our algorithm, we have implemented it in Java, developed some large examples, and compared our results with the ones from an existing algorithm which is based on discretization. In order to express properties specifying these types of probabilities in the continuous-time setting, we adjusted the existing temporal logic, PATL. Thus, properties like the following can be stated and checked over a CTSG: “A coalition of players has a strategy to guarantee to reach the target states within time boundt∈Q⩾0with a probability equal to or greater than threshold P, regardless of the other players' behaviors.”

Using Persistence to Generate Incentives in a Dynamic Moral Hazard Problem

I study how the persistence of past choices can be used to create incentives in a continuous time stochastic game in which a large player, such as a i¬ rm, interacts with a sequence of short-run players, such as customers. The long-run player faces moral hazard and her past actions are imperfectly observed – they are distorted by a Brownian motion. Persistence refers to the fact that actions impact a payoi¬€relevant state variable, e.g. the quality of a product depends on both current and past investment choices. I obtain a characterization of actions and payoi¬€s in Markov Perfect Equilibria (MPE), for a i¬ xed discount rate. I show that the perfect public equilibrium (PPE) payoi¬€ set is the convex hull of the MPE payoi¬€ set. Finally, I derive sui¬ƒcient conditions for a MPE to be the unique PPE. Persistence creates ei¬€ective intertemporal incentives to overcome moral hazard in settings where traditional channels fail. Several applications illustrate how the structure of persistence impacts the strength of these incentives.

Mean Field Games of Timing and Models for Bank Runs

The goal of the paper is to introduce a set of problems which we call mean field games of timing. We motivate the formulation by a dynamic model of bank run in a continuous-time setting. We briefly review the economic and game theoretic contributions at the root of our effort, and we develop a mathematical theory for continuous-time stochastic games where the strategic decisions of the players are merely choices of times at which they leave the game, and the interaction between the strategic players is of a mean field nature.

Continuous-time stochastic games

We study continuous-time stochastic games, with a focus on the existence of their equilibria that are insensitive to a small imprecision in the specification of players' evaluations of streams of payoffs.We show that the stationary, namely, time-independent, discounting game has a stationary equilibrium and that the discounting game and the more general game with time-separable payoffs have an epsilon equilibrium that is an epsilon equilibrium in all games with a sufficiently small perturbation of the players' valuations.A limit point of discounting valuations need not be a discounting valuation as some of the “mass” may be pushed to infinity; it is represented by an average of a discounting valuation and a mass at infinity.We show that for every such limit point there is a strategy profile that is an epsilon equilibrium of all the discounting games with discounting valuations that are sufficiently close to the limit point.

Using Persistence to Generate Incentives in a Dynamic Moral Hazard Problem

This paper studies how persistence can be used to create incentives in a continuous-time stochastic game in which a long-run player interacts with a sequence of short-run players. Observation of the long-run player's actions are distorted by a Brownian motion and the actions of both players impact future payoffs through a state variable. For example, a firm or worker provides customers with a product, and the quality of this product depends on both current and past investment choices by the rm. I derive general conditions under which a Markov equilibrium emerges as the unique perfect public equilibrium, and characterize the equilibrium payoff and actions in this equilibrium, for any discount rate. I develop an application of persistent product quality to illustrate how persistence creates effective intertemporal incentives in a setting where traditional channels fail, and explore how the structure of persistence impacts equilibrium behavior. This demonstrates the power of the continuous-time setting to deliver sharp insights and a tractable equilibrium characterization for a rich class of dynamic games.

Approximate Solutions of Continuous-Time Stochastic Games

The paper is concerned with a zero-sum continuous-time stochastic differential game with a dynamics controlled by a Markov process and a terminal payoff. The value function of the original game is estimated using the value function of a model game. The dynamics of the model game differs from the original one. The general result applied to differential games yields the approximation of the value function of the differential game by the solution of countable system of ODEs.

Stochastic Games in Continuous Time: Persistent Actions in Long-Run Relationships, Second Version

This paper studies a class of continuous-time stochastic games in which the actions of a long-run player have a persistent effect on payoffs. For example, the quality of a firm's product depends on past as well as current effort, or the level of a policy instrument depends on a government's past choices. The long-run player faces a population of small players, and its actions are imperfectly observed. I establish the existence of Markov equilibria, characterize the Perfect Public Equilibria (PPE) pay-offset as the convex hull of the Markov Equilibria payoff set, and identify conditions for the uniqueness of a Markov equilibrium in the class of all PPE. The existence proof is constructive: it characterizes the explicit form of Markov equilibria payoffs and actions, for any discount rate. Action persistence creates a crucial new channel to generate intertemporal incentives in a setting where traditional channels fail, and can provide permanent non-trivial incentives in many settings. These results offer a novel framework for thinking about reputational dynamics of firms, governments, and other long-run agents.

Continuous-time stochastic games with time-bounded reachability

We study continuous-time stochastic games with time-bounded reachability objectives and time-abstract strategies. We show that each vertex in such a game has a value (i.e., an equilibrium probability), and we classify the conditions under which optimal strategies exist. Further, we show how to compute ε-optimal strategies in finite games and provide detailed complexity estimations. Moreover, we show how to compute ε-optimal strategies in infinite games with finite branching and bounded rates where the bound as well as the successors of a given state are effectively computable. Finally, we show how to compute optimal strategies in finite uniform games.

Continuous-Time Stochastic Games of Fixed Duration

We study nonzero-sum continuous-time stochastic games, also known as continuous-time Markov games, of fixed duration. We concentrate on Markovian strategies. We show by way of example that equilibria need not exist in Markovian strategies, but they always exist in Markovian public-signal correlated strategies. To do so, we develop criteria for a strategy profile to be an equilibrium via differential inclusions, both directly and also by modeling continuous-time stochastic as differential games and using the Hamilton–Jacobi–Bellman equations. We also give an interpretation of equilibria in mixed strategies in continuous time and show that approximate equilibria always exist.

Avoiding the Curse of Dimensionality in Dynamic Stochastic Games

Discrete-time stochastic games with a finite number of states have been widely applied to study the strategic interactions among forward-looking players in dynamic environments. However, these games suffer from a curse of dimensionality since the cost of computing players' expectations over all possible future states increases exponentially in the number of state variables. We explore the alternative of continuous-time stochastic games with a finite number of states, and show that continuous time has substantial computational and conceptual advantages. Most important, continuous time avoids the curse of dimensionality, thereby speeding up the computations by orders of magnitude in games with more than a few state variables. Overall, the continuous-time approach opens the way to analyze more complex and realistic stochastic games than currently feasible.

Avoiding the curse of dimensionality in dynamic stochastic games

Discrete-time stochastic games with a flnite number of states have been widely applied to study the strategic interactions among forward-looking players in dynamic environments. However, these games sufier from a \curse of dimensionality since the cost of computing players’ expectations over all possible future states increases exponentially in the number of state variables. We explore the alternative of continuous-time stochastic games with a flnite number of states, and show that continuous time has substantial computational and conceptual advantages. Most important, continuous time avoids the curse of dimensionality, thereby speeding up the computations by orders of magnitude in games with more than a few state variables. Overall, the continuous-time approach opens the way to analyze more complex and realistic stochastic games than currently feasible.

Semi-Markov and Jump Markov Controlled Models: Average Cost Criterion

Semi-Markov and Jump Markov Controlled Models: Average Cost Criterion