Positive Zero-Sum Stochastic Games with Countable State and Action Spaces

János Flesch,Arkadi Predtetchinski,William Sudderth

doi:10.1007/s00245-018-9536-3

János Flesch, Arkadi Predtetchinski + Show 1 more

Open Access

https://doi.org/10.1007/s00245-018-9536-3

Copy DOI

Abstract

A positive zero-sum stochastic game with countable state and action spaces is shown to have a value if, at every state, at least one player has a finite action space. The proof uses transfinite algorithms to calculate the upper and lower values of the game. We also investigate the existence of (epsilon -)optimal strategies in the classes of stationary and Markov strategies.

Highlights

A positive zero-sum stochastic game is a two-person dynamic game played in stages t = 0, 1, . . . At each stage t of the game the players simultaneously select actions from their action sets at the current state
A typical example is the following: There is a state with action set N = {0, 1, . . .} for each player in which the reward is 1 if player 1’s action is greater than player 2’s action and is 0 otherwise
If α(s) = β(s), this quantity is called the value of the game for initial state s and it is denoted by v(s)

Summary

Introduction

If at every state at least one player has a finite action set, the algorithms for the upper and lower values give the same answer. In this case, the game has a value. If the action set for player 2 is finite at every state, the algorithm for the value simplifies and becomes the limit of the sequence of values of the n-stage games. In this simpler case, player 2 has an optimal stationary strategy (cf Theorem 12).

The Model

One-Shot Games

One-Shot Operators and Their Fixed Points

The Operator A for the Lower Value

The Operator B for the Upper Value

The Value and Optimal Strategies