Average Payoff Criterion Research Articles

An Approximate Dynamic Programming Approach to Repeated Games with Vector Losses

A seminal study showed repeated games with vector payoffs, which has had wide-ranging applications to decision making under uncertainty. Although this study characterized the achievable guarantees in such games under the long-run average payoff criterion, the characterization and computation of these guarantees under the discounted payoff criterion have remained a significant gap in the literature. In “An Approximate Dynamic Programming Approach to Repeated Games with Vector Losses,” V. Kamble, P. Loiseau, and J. Walrand develop a set-valued dynamic programming approach along with an approximation framework to fully address this problem. This theory results in new algorithms that beat the state of the art for applications such as exact regret minimization in the well-known problem of prediction under expert advice.

Average stochastic games for continuous-time jump processes

We consider nonzero-sum games for continuous-time jump processes with unbounded transition rates under expected average payoff criterion. The state and action spaces are Borel spaces and reward rates are unbounded. We introduce an approximating sequence of stochastic game models with extended state space, for which the uniform exponential ergodicity is obtained. Moreover, we prove the existence of a stationary almost Markov Nash equilibrium by introducing auxiliary static game models. Finally, a cash flow model is employed to illustrate the results.

Discrete-time constrained stochastic games with the expected average payoff criteria

ABSTRACT In this paper we consider nonzero-sum discrete-time constrained stochastic games under the expected average payoff criteria. The state space is a countable set, the action spaces of the players are Borel spaces and the cost functions can be possibly unbounded. Under reasonable conditions, we first construct an approximating sequence of the auxiliary constrained stochastic game models and obtain the ergodicity of the approximating transition laws. Then basing on the properties of the invariant probability measures, we introduce a suitable multifunction and show the existence of constrained Nash equilibria for these approximating game models by a fixed point approach. Moreover, we prove the existence of a stationary constrained Nash equilibrium for the original game model via an approximation technique. Furthermore, we use a controlled population system to illustrate our results.

Constrained Expected Average Stochastic Games for Continuous-Time Jump Processes

In this paper we study the nonzero-sum constrained stochastic games for continuous-time jump processes with denumerable states and possibly unbounded transition rates. The optimality criterion under consideration is the expected average payoff criterion and the payoff functions of the players are allowed to be unbounded. Under the reasonable conditions, we introduce an approximating sequence of the auxiliary game models and show the existence of stationary constrained Nash equilibria for these approximating game models via employing the average occupation measures and constructing a suitable multifunction. Moreover, we obtain that any limit point of the stationary constrained Nash equilibria for the approximating sequence of the game models is a constrained Nash equilibrium for the original game model. Furthermore, we use a controlled birth and death system to illustrate our main results.

Nonzero-sum Games for Continuous-Time Jump Processes Under the Expected Average Payoff Criterion

In this paper we investigate nonzero-sum games for continuous-time jump processes with Borel state spaces. The optimality criterion to be considered is the expected average payoff criterion. The action spaces for the players are Borel spaces, and the reward and transition rates can be possibly unbounded. Under suitable conditions, we introduce the auxiliary static games and prove the existence of a stationary Nash equilibrium in the class of all randomized history-dependent strategy profiles via a technique of discounted approximation. Moreover, an example is given to illustrate the optimality conditions.

Nonzero-Sum Expected Average Discrete-Time Stochastic Games: The Case of Uncountable Spaces

In this paper we study nonzero-sum discrete-time stochastic games with an uncountable state space and Borel action spaces under the expected average payoff criterion. The reward functions can be possibly unbounded and the transition law is a convex combination of finitely many probability measures dependent on the state variable and dominated by some probability measure on the state space. We introduce several auxiliary static game models and obtain their properties. Moreover, by a technique of extending the space state, we introduce auxiliary stochastic game models and derive the uniform geometric ergodicity of Markov chains taking values in the extended state space. Furthermore, we show the existence of a stationary almost Markov Nash equilibrium via an approximation method. Finally, we use a resource extraction model to illustrate the main results.

Risk-Sensitive Average Equilibria for Discrete-Time Stochastic Games

In this paper, we study the risk-sensitive average payoff criterion for the nonzero-sum discrete-time stochastic games with a denumerable state space. The risk-sensitivity coefficient can take positive values and negative values. Under the suitable conditions, we show the existence of a solution to the coupled equations by a technique of the discounted approximation, and obtain the existence of a stationary Nash equilibrium. Moreover, we present some verifiable sufficient conditions imposed on the primitive data of the model for the verification of our assumption and use an example to illustrate that our conditions are weaker than those in the existing literature.

Discount-sensitive equilibria in zero-sum stochastic differential games

We consider infinite-horizon zero-sum stochastic differential games with average payoff criteria,discount -sensitive criteria and, infinite-horizon undiscounted reward criteria which are sensitive to the growth rate of finite-horizon payoffs. These criteria include, average reward optimality, strong 0-discount optimality, strong -1-discount optimality, 0-discount optimality, bias optimality, F-strong average optimality and overtaking optimality. The main objective is to give conditions under which these criteria are interrelated.

Discount-sensitive equilibria in zero-sum stochastic differential games

We consider infinite-horizon zero-sum stochastic differential games with average payoff criteria, discount -sensitive criteria and, infinite-horizon undiscounted reward criteria which are sensitive to the growth rate of finite-horizon payoffs. These criteria include, average reward optimality, strong 0-discount optimality, strong -1-discount optimality, 0-discount optimality, bias optimality, F-strong average optimality and overtaking optimality. The main objective is to give conditions under which these criteria are interrelated.

The Lagrange and the vanishing discount techniques to controlled diffusions with cost constraints

In this paper we introduce two useful methods to compute optimal control policies for either the discounted or the average payoff criterion with cost constraints when the dynamic system evolves as a n-dimensional diffusion processes. As for the attribute “cost constraints” we mean the coexistence of a given cost function that in general is dominated above by another function (in particular by a constant) playing the role of an extra constraint in the control model. To deduce optimality results for the discounted case, we employ the Lagrange multipliers technique along with dynamic programming arguments. Then, the vanishing discount method is applied to easily obtain average optimal policies. We support our theory by providing an example on pollution accumulation problem.

Constrained stochastic games with the average payoff criteria

In this paper we study two-person nonzero-sum constrained stochastic games under the average payoff criteria. The state space is denumerable and the action spaces of the players are Borel spaces. Under the suitable conditions, we prove the existence of a constrained stationary Nash equilibrium via the vanishing discount approach. Furthermore, we use an example to illustrate our conditions.

Approximation of two-person zero-sum continuous-time Markov games with average payoff criterion

We consider a two-person zero-sum continuous-time Markov game G with denumerable state space, Borel action spaces, unbounded payoff and transition rates, under the long-run expected average payoff criterion. To approximate numerically the value of G we construct finite state and actions game models Gn whose value functions converge to the value of G. Rates of convergence are given. We propose a policy iteration algorithm for the finite state and actions games Gn. We show an application to a population system.

Policy Iteration Algorithms for Zero-Sum Stochastic Differential Games with Long-Run Average Payoff Criteria

This paper studies the policy iteration algorithm (PIA) for zero-sum stochastic differential games with the basic long-run average criterion, as well as with its more selective version, the so-called bias criterion. The system is assumed to be a nondegenerate diffusion. We use Lyapunov-like stability conditions that ensure the existence and boundedness of the solution to certain Poisson equation. We also ensure the convergence of a sequence of such solutions, of the corresponding sequence of policies, and, ultimately, of the PIA.

ORDERED FIELD PROPERTY IN A SUBCLASS OF FINITE SER-SIT SEMI-MARKOV GAMES

In this paper, we deal with a subclass of two-person finite SeR-SIT (Separable Reward-State Independent Transition) semi-Markov games which can be solved by solving a single matrix/bimatrix game under discounted as well as limiting average (undiscounted) payoff criteria. A SeR-SIT semi-Markov game does not satisfy the so-called (Archimedean) ordered field property in general. Besides, the ordered field property does not hold even for a SeR-SIT-PT (Separable Reward-State-Independent Transition Probability and Time) semi-Markov game, which is a natural version of a SeR-SIT stochastic (Markov) game. However by using an additional condition, we have shown that a subclass of finite SeR-SIT-PT semi-Markov games have the ordered field property for both discounted and undiscounted semi-Markov games with both players having state-independent stationary optimals. The ordered field property also holds for the nonzero-sum case under the same assumptions. We find a relation between the values of the discounted and the undiscounted zero-sum semi-Markov games for this modified subclass. We propose a more realistic pollution tax model for this subclass of SeR-SIT semi-Markov games than pollution tax model for SeR-SIT stochastic game. Finite step algorithms are given for the discounted and for the zero-sum undiscounted cases.

Zero-Sum Stochastic Games with Partial Information and Average Payoff

We consider a discrete time partially observable zero-sum stochastic game with average payoff criterion. We study the game using an equivalent completely observable game. We show that the game has a value and also we present a pair of optimal strategies for both the players.

A New Condition and Approach for Zero-Sum Stochastic Games with Average Payoffs

This article deals with discrete-time two-person zero-sum stochastic games with Borel state and action spaces. The optimality criterion to be studied is the long-run expected average payoff criterion, and the (immediate) payoff function may have neither upper nor lower bounds. We first replace the optimality equation widely used in the previous literature with two so-called optimality inequalities, and give a new set of conditions for the existence of solutions to the optimality inequalities. Then, from the optimality inequalities we ensure the existence of a pair of average optimal stationary strategies. Our new condition is slightly weaker than those in the previous literature, and as a byproduct some interesting results such as the convergence of a value iteration scheme to the value of the discounted payoff game is obtained. Finally, we first apply the main results in this article to generalized inventory systems, and then further provide an example of controlled population processes for which all of our conditions are satisfied, while some of conditions in some of previous literature fail to hold.

Remarks on sensitive equilibria in stochastic games with additive reward and transition structure

A class of stochastic games with additive reward and transition structure is studied. For zero-sum games under some ergodicity assumptions 1-equilibria are shown to exist. They correspond to so-called sensitive optimal policies in dynamic programming. For a class of nonzero-sum stochastic games with nonatomic transitions nonrandomized Nash equilibrium points with respect to the average payoff criterion are also obtained. Included examples show that the results of this paper can not be extented to more general payoff or transition structure.

A nonzero-sum stochastic differential game in the orthant

We study a nonzero-sum stochastic differential game where the state is a controlled reflecting diffusion in the nonnegative orthant. Under certain conditions, we establish the existence of Nash equilibria in stationary strategies for both discounted and average payoff criteria.

Zero-sum games for continuous-time Markov chains with unbounded transition and average payoff rates

This paper is a first study of two-person zero-sum games for denumerable continuous-time Markov chains determined by given transition rates, with an average payoff criterion. The transition rates are allowed to be unbounded, and the payoff rates may have neither upper nor lower bounds. In the spirit of the ‘drift and monotonicity’ conditions for continuous-time Markov processes, we give conditions on the controlled system's primitive data under which the existence of the value of the game and a pair of strong optimal stationary strategies is ensured by using the Shapley equations. Also, we present a ‘martingale characterization’ of a pair of strong optimal stationary strategies. Our results are illustrated with a birth-and-death game.

Zero-sum games for continuous-time Markov chains with unbounded transition and average payoff rates

This paper is a first study of two-person zero-sum games for denumerable continuous-time Markov chains determined by given transition rates, with an average payoff criterion. The transition rates are allowed to be unbounded, and the payoff rates may have neither upper nor lower bounds. In the spirit of the ‘drift and monotonicity’ conditions for continuous-time Markov processes, we give conditions on the controlled system's primitive data under which the existence of the value of the game and a pair of strong optimal stationary strategies is ensured by using the Shapley equations. Also, we present a ‘martingale characterization’ of a pair of strong optimal stationary strategies. Our results are illustrated with a birth-and-death game.