Two-person Stochastic Games Research Articles

Search for an immobile hider on a stochastic network

Harry hides on an edge of a graph and does not move from there. Sally, starting from a known origin, tries to find him as soon as she can. Harry’s goal is to be found as late as possible. At any given time, each edge of the graph is either active or inactive, independently of the other edges, with a known probability of being active. This situation can be modeled as a zero-sum two-person stochastic game. We show that the game has a value and we provide upper and lower bounds for this value. Finally, by generalizing optimal strategies of the deterministic case, we provide more refined results for trees and Eulerian graphs.

Existence of Nash equilibria in stochastic games of resource extraction with risk-sensitive players

We consider a two-person stochastic game of resource extraction. It is assumed that players have identical preferences. A novelty relies on the fact that each player is equipped with the same risk coefficient and calculates his discounted utility in the infinite time horizon in a recursive way by applying the entropic risk measure parametrized by this risk coefficient. Under two alternative sets of assumptions, we prove the existence of a symmetric stationary Markov perfect equilibrium.

Stationary, completely mixed and symmetric optimal and equilibrium strategies in stochastic games

In this paper, we address various types of two-person stochastic games—both zero-sum and nonzero-sum, discounted and undiscounted. In particular, we address different aspects of stochastic games, namely: (1) When is a two-person stochastic game completely mixed? (2) Can we identify classes of undiscounted zero-sum stochastic games that have stationary optimal strategies? (3) When does a two-person stochastic game possess symmetric optimal/equilibrium strategies? Firstly, we provide some necessary and some sufficient conditions under which certain classes of discounted and undiscounted stochastic games are completely mixed. In particular, we show that, if a discounted zero-sum switching control stochastic game with symmetric payoff matrices has a completely mixed stationary optimal strategy, then the stochastic game is completely mixed if and only if the matrix games restricted to states are all completely mixed. Secondly, we identify certain classes of undiscounted zero-sum stochastic games that have stationary optima under specific conditions for individual payoff matrices and transition probabilities. Thirdly, we provide sufficient conditions for discounted as well as certain classes of undiscounted stochastic games to have symmetric optimal/equilibrium strategies—namely, transitions are symmetric and the payoff matrices of one player are the transpose of those of the other. We also provide a sufficient condition for the stochastic game to have a symmetric pure strategy equilibrium. We also provide examples to show the sharpness of our results.

A policy-improvement type algorithm for solving zero-sum two-person stochastic games of perfect information

We give a policy-improvement type algorithm to locate an optimal pure stationary strategy for discounted stochastic games with perfect information. A graph theoretic motivation for our algorithm is presented as well.

EQUILIBRIUM STRATEGIES IN STOCHASTIC GAMES WITH ADDITIVE COST AND TRANSITION STRUCTURE

Two-person stochastic games with additive transition and cost structure and the criterion of expected total costs are treated. State space and action spaces are standard Borel, and unbounded costs are allowed. For the zero-sum case, it is shown that there are stationary deterministic εη-optimal strategies for every ε>0 and a certain weight function η if some semi-continuity and compactness conditions are fulfilled. Using these results, the existence of so-called quasi-stationary deterministic εη-equilibrium strategy pairs under corresponding conditions is proven.

Stochastic Games with Average Payoff Criterion

We study two-person stochastic games on a Polish state and compact action spaces and with average payoff criterion under a certain ergodicity condition. For the zero-sum game we establish the existence of a value and stationary optimal strategies for both players. For the nonzero-sum case the existence of Nash equilibrium in stationary strategies is established under certain separability conditions.

An operator solution of stochastic games

A class of zero-sum, two-person stochastic games is shown to have a value which can be calculated by transfinite iteration of an operator. The games considered have a countable state space, finite action spaces for each player, and a payoff sufficiently general to include classical stochastic games as well as Blackwell’s infiniteG δ games of imperfect information.

On stochastic games with additive reward and transition structure

In this paper, we introduce a new class of two-person stochastic games with nice properties. For games in this class, the payoffs as well as the transitions in each state consist of a part which depends only on the action of the first player and a part dependent only on the action of the second player. For the zero-sum games in this class, we prove that the orderfield property holds in the infinite-horizon case and that there exist optimal pure stationary strategies for the discounted as well as the undiscounted payoff criterion. For both criteria also, finite algorithms are given to solve the game. An example shows that, for nonzero sum games in this class, there are not necessarily pure stationary equilibria. But, if such a game possesses a stationary equilibrium point, then there also exists a stationary equilibrium point which uses in each state at most two pure actions for each player.

An orderfield property for stochastic games when one player controls transition probabilities

When the transition probabilities of a two-person stochastic game do not depend on the actions of a fixed player at all states, the value exists in stationary strategies. Further, the data of the stochastic game, the values at each state, and the components of a pair of optimal stationary strategies all lie in the same Archimedean ordered field. This orderfield property holds also for the nonzero sum case in Nash equilibrium stationary strategies. A finite-step algorithm for the discounted case is given via linear programming.

A birth–death model of advertising and pricing

This paper employs the methods currently used to solve many queuing control models in order to investigate the behavior of a firm's optimal advertising and pricing strategies over time. Given that a firm's market position expands or deteriorates in a probabilistic way which depends upon the current position, the rate of advertising, and the price the firm charges, we present conditions which ensure that the optimal level of advertising is a monotonic function of the firm's market position, and we discuss the economic meaning of these conditions. Furthermore, although the primary focus is upon a non-competitive environment, we develop the above model as a non-zero sum, two-person stochastic game and show that an equilibrium strategy exists which is simple to compute.

A birth–death model of advertising and pricing

This paper employs the methods currently used to solve many queuing control models in order to investigate the behavior of a firm's optimal advertising and pricing strategies over time. Given that a firm's market position expands or deteriorates in a probabilistic way which depends upon the current position, the rate of advertising, and the price the firm charges, we present conditions which ensure that the optimal level of advertising is a monotonic function of the firm's market position, and we discuss the economic meaning of these conditions. Furthermore, although the primary focus is upon a non-competitive environment, we develop the above model as a non-zero sum, two-person stochastic game and show that an equilibrium strategy exists which is simple to compute.