Optimal Stationary Strategies Research Articles

Controlled Markov Chains and Stochastic Networks

Controlled Markov chains with average cost criterion and with special cost and transition structures are studied. Existence of optimal stationary strategies is established for the average cost criterion. Corresponding dynamic programming equations are derived. A stochastic network problem that includes interconnected queues as a special case is described and studied within this framework.

The existence of optimal strategy in the exploitation of limited biocenoses

Optimal strategy in the control of multicomponent biocenoses that are described adequately by an autonomous discrete model has been investigated. It is shown that for any such biocenosis there exists at least one stationary optimal strategy in terms of discounted reward. The optimal strategy is described for communities of competing species, together with the method of its derivation.

A two-person zero-sum Markov game with a stopped set

We consider a two-person zero-sum Markov game with continuous time up to the time that the game process goes into a fixed subset of a countable state space, this subset is called a stopped set of the game. We show that such a game with a discount factor has optimal value function and both players will have their optimal stationary strategies. The same result is proved for the case of a nondiscounted Markov game under some additional conditions, that is a reward rate function is nonnegative and the first time τ (entrance time) of the game process going to the stopped set is finite with probability one (i.e., p( τ < ∞) = 1). It is remarkable that in the case of a nondiscounted Markov game, if the expectation of the entrance time is bounded, and the reward rate function need not be nonnegative, then the same result holds.

Existence of Value and Randomized Strategies in Zero-Sum Discrete-Time Stochastic Dynamic Games

Two players with conflicting objectives are simultaneously controlling a discrete-time stochastic system. The goal of this paper is to analyze such zero-sum, discrete-time, stochastic systems when the two players are allowed to use randomized strategies. Previous results have been restricted to systems with finite or compact state spaces. Such restrictions are usually untenable from the point of view of applications, since many applications frequently use either the integers or $\mathbb{R}^n $ as a state space. Our results are proved for complete, separable, metric spaces which are very useful for applications. All previously known results emerge as special cases of our results. In addition, a variety of conjectures and open problems are resolved regarding the existence of a value function, its properties such as Borel measurability or continuity, and the existence for either or both players of optimal or $\varepsilon $-optimal stationary strategies.

Ordered field property for stochastic games when the player who controls transitions changes from state to state

In this paper, we consider a zero-sum stochastic game with finitely many states restricted by the assumption that the probability transitions from a given state are functions of the actions of only one of the players. However, the player who thus controls the transitions in the given state will not be the same in every state. Further, we assume that all payoffs and all transition probabilities specifying the law of motion are rational numbers. We then show that the values of both a β-discounted game, for rational β, and of a Cesaro-average game are in the field of rational numbers. In addition, both games possess optimal stationary strategies which have only rational components. Our results and their proofs form an extension of the results and techniques which were recently developed by Parthasarathy and Raghavan (Ref. 1).

An orderfield property for stochastic games when one player controls transition probabilities

When the transition probabilities of a two-person stochastic game do not depend on the actions of a fixed player at all states, the value exists in stationary strategies. Further, the data of the stochastic game, the values at each state, and the components of a pair of optimal stationary strategies all lie in the same Archimedean ordered field. This orderfield property holds also for the nonzero sum case in Nash equilibrium stationary strategies. A finite-step algorithm for the discounted case is given via linear programming.

A semi-markovian game of economic survival

Abstract This paper is devoted to a semi-markovian generalization of a well-known game of economic survival: a firm has an initial capital x, the profits resulting from the successive time intervals (n, n + 1) are random variables; at each instant n the firm may pay dividends to the shareholders, his only goal being to maximize the expected discounted value of all dividends payed before the ruin. The existence of an optimal stationary strategy is proved; the only “impatient” optimal stationary strategy is a “band-strategy”. In the last part of the paper we construct an algorithm producing the optimal band-strategy after a finite number of iterations, and we show how to calculate the ruin-probabilities associated with this strategy.

Zur bestimmung optimaler strategien für unvollständig beobaehtbare stationäre stopp-steuer-probleme

Specifying results given in [4] the existence of stationary optimal strategies is proved. The numerical calculation of optimal strategies is investigated.

Relations between game parameters, value, and optimal strategy spaces in stochastic games and construction of games with given solution

Results of Bohnenblust, Karlin, and Shapley and results of Shapley and Snow, concerning solutions of matrix games, are extended to the class of discounted stochastic games. Prior to these extensions, relations between the game parameters, value, and optimal stationary strategy spaces are established. Then, the inverse problem of constructing stochastic games, given the solution, is considered.

Discounted Stochastic Ratio Games

In a recent work, the authors considered a finite state Markov ratio decision process in which the objective was to maximize the ratio of total discounted rewards. In this paper, discounted Markov ratio decision processes are generalized to discounted stochastic ratio games. These may also be viewed as generalizations of ratio games to a stochastic context where the payoff is the ratio of the two total discounted rewards. We show that in the discounted stochastic ratio game the players have stationary optimal strategies with a unique value. The solution may depend on the initial probability distribution. We also provide a convergent algorithm.

Successive Approximation Methods in Undiscounted Stochastic Games

This paper considers undiscounted two-person, zero-sum sequential games with finite state and action spaces. Under conditions that guarantee the existence of stationary optimal strategies, we present two successive approximation methods for finding the optimal gain rate, a solution to the optimality equation, and for any ϵ &gt; 0, ϵ-optimal policies for both players.

Stochastic games with metric state space

In this paper the stochastic two-person zero-sum game of Shapley is considered, with metric state space and compact action spaces. It is proved that both players have stationary optimal strategies, under conditions which are weaker than those ofMaitra/Parthasarathy (a.o. no compactness of the state space). This is done in the following way: we show the existence of optimal strategies first for the one-period game with general terminal reward, then for then-period games (n=1,2,...); further we prove that the game over the infinite horizon has a valuev, which is the limit of then-period game values. Finally the stationary optimal strategies are found as optimal strategies in the one-period game with terminal rewardv.

Successive approximations for average reward Markov games

This paper considers two-person zero-sum Markov games with finitely many states and actions with the criterion of average reward per unit time. Two special situations are treated and it is shown that in both cases the method of successive approximations yields ane-band for the value of the game as well as stationarye-optimal strategies. In the first case all underlying Markov chains of pure stationary optimal strategies are assumed to be unichained. In the second case it is assumed that the functional equation Uv=v+ge has a solution.

Repeated matrix games

A matrix game is played repeatedly, with the actions taken at each stage determining both a reward paid to Player I and the probability of continuing to the next stage. An infinite history of play determines a sequence (Rn ) of such rewards, to which we assign the payoff lim supn (R 1 + · ·· + Rn ). Using the concept of playable strategies, we slightly generalize the definition of the value of a game. Then we find sufficient conditions for the existence of a value and for the existence of stationary optimal strategies (Theorems 8 and 9). An example shows that the game need not have a value (Example 4).

Repeated matrix games

A matrix game is played repeatedly, with the actions taken at each stage determining both a reward paid to Player I and the probability of continuing to the next stage. An infinite history of play determines a sequence (Rn) of such rewards, to which we assign the payoff lim supn (R1 + · ·· + Rn). Using the concept of playable strategies, we slightly generalize the definition of the value of a game. Then we find sufficient conditions for the existence of a value and for the existence of stationary optimal strategies (Theorems 8 and 9). An example shows that the game need not have a value (Example 4).

On Stochastic Games with Stationary Optimal Strategies

We study two-person zero-sum stochastic games in which the state and action spaces are finite. We give both necessary and sufficient conditions for the players to have stationary optimal strategies in the infinite-stage game.

On an optimization problem for a Wiener process

For a Wiener-type production process with an observation cost, the optimal stationary strategy is shown to exist. Existence and uniqueness of the optimal strategy is shown by showing that a corresponding contraction mapping has a unique fixed point.

可附番状態空間をもつ連続時間マルコフゲームについて

This paper is a continuation of our papers [6] and [7] and is concerned with a continuous time Markov game in which the state space is countable and the action spaces of player I and player II are compact metric spaces. In the game, the players continuously observe the state of the system and then choose actions. As a result, the reward is paid to player I from player II and the system moves to a new state by the known transition rates. Then we consider the optimization problem to maximize the total expected discounted gain for player I and, at the same time, to minimize the total expected loss for player II as the game proceeds to the infinite future. We show that such a two-person zero-sum game is strictly determined and both players have optimal stationary strategies.

Discounted Markov games; successive approximation and stopping times

This paper presents a number of successive approximation algorithms for the repeated two-person zero-sum game called Markov game using the criterion of total expected discounted rewards. AsWessels [1977] did for Markov decision processes stopping times are introduced in order to simplify the proofs. It is shown that each algorithm provides upper and lower bounds for the value of the game and nearly optimal stationary strategies for both players.

Existence of p-Equilibrium and Optimal Stationary Strategies in Stochastic Games

Existence of p-Equilibrium and Optimal Stationary Strategies in Stochastic Games