Payoff Criteria Research Articles

Multiple-Population Discrete-Time Mean Field Games with Discounted and Total Payoffs: The Existence of Equilibria

In the paper we present a model of discrete-time mean-field game with several populations of players. Mean-field games with multiple populations of the players have only been studied in the literature in the continuous-time setting. The main results of this article are the first stationary and Markov mean-field equilibrium existence theorems for discrete-time mean-field games of this type. We consider two payoff criteria: β\\documentclass[12pt]{minimal} \\usepackage{amsmath} \\usepackage{wasysym} \\usepackage{amsfonts} \\usepackage{amssymb} \\usepackage{amsbsy} \\usepackage{mathrsfs} \\usepackage{upgreek} \\setlength{\\oddsidemargin}{-69pt} \\begin{document}$$\\beta $$\\end{document}-discounted payoff and total payoff. The results are provided under some rather general assumptions on one-step reward functions and individual transition kernels of the players. In addition, the results for total payoff case, when applied to a single population, extend the theory of mean-field games also by relaxing some strong assumptions used in the existing literature.

Multiply Accelerated Value Iteration for NonSymmetric Affine Fixed Point Problems and Application to Markov Decision Processes

We analyze a modified version of the Nesterov accelerated gradient algorithm, which applies to affine fixed point problems with non-self-adjoint matrices, such as the ones appearing in the theory of Markov decision processes with discounted or mean payoff criteria. We characterize the spectra of matrices for which this algorithm does converge with an accelerated asymptotic rate. We also introduce a $d$th-order algorithm and show that it yields a multiply accelerated rate under more demanding conditions on the spectrum. We subsequently apply these methods to develop accelerated schemes for nonlinear fixed point problems arising from Markov decision processes. This is illustrated by numerical experiments.

Discrete-time constrained stochastic games with the expected average payoff criteria

ABSTRACT In this paper we consider nonzero-sum discrete-time constrained stochastic games under the expected average payoff criteria. The state space is a countable set, the action spaces of the players are Borel spaces and the cost functions can be possibly unbounded. Under reasonable conditions, we first construct an approximating sequence of the auxiliary constrained stochastic game models and obtain the ergodicity of the approximating transition laws. Then basing on the properties of the invariant probability measures, we introduce a suitable multifunction and show the existence of constrained Nash equilibria for these approximating game models by a fixed point approach. Moreover, we prove the existence of a stationary constrained Nash equilibrium for the original game model via an approximation technique. Furthermore, we use a controlled population system to illustrate our results.

A Game Theoretical Approach to Homothetic Robust Forward Investment Performance Processes in Stochastic Factor Models

This paper studies an optimal forward investment problem in an incomplete market with model uncertainty, in which the dynamics of the underlying stocks depends on the correlated stochastic factors. The uncertainty stems from the probability measure chosen by an investor to evaluate the performance. We obtain directly the representation of the power robust forward performance process in factor-form by combining the zero-sum stochastic differential game and ergodic BSDE approach. We also establish the connections with the risk-sensitive zero-sum stochastic differential games over an infinite horizon with ergodic payoff criteria, as well as with the classical power robust expected utility for long time horizons.

Discount-sensitive equilibria in zero-sum stochastic differential games

We consider infinite-horizon zero-sum stochastic differential games with average payoff criteria,discount -sensitive criteria and, infinite-horizon undiscounted reward criteria which are sensitive to the growth rate of finite-horizon payoffs. These criteria include, average reward optimality, strong 0-discount optimality, strong -1-discount optimality, 0-discount optimality, bias optimality, F-strong average optimality and overtaking optimality. The main objective is to give conditions under which these criteria are interrelated.

Discount-sensitive equilibria in zero-sum stochastic differential games

We consider infinite-horizon zero-sum stochastic differential games with average payoff criteria, discount -sensitive criteria and, infinite-horizon undiscounted reward criteria which are sensitive to the growth rate of finite-horizon payoffs. These criteria include, average reward optimality, strong 0-discount optimality, strong -1-discount optimality, 0-discount optimality, bias optimality, F-strong average optimality and overtaking optimality. The main objective is to give conditions under which these criteria are interrelated.

Constrained stochastic games with the average payoff criteria

In this paper we study two-person nonzero-sum constrained stochastic games under the average payoff criteria. The state space is denumerable and the action spaces of the players are Borel spaces. Under the suitable conditions, we prove the existence of a constrained stationary Nash equilibrium via the vanishing discount approach. Furthermore, we use an example to illustrate our conditions.

Policy Iteration Algorithms for Zero-Sum Stochastic Differential Games with Long-Run Average Payoff Criteria

This paper studies the policy iteration algorithm (PIA) for zero-sum stochastic differential games with the basic long-run average criterion, as well as with its more selective version, the so-called bias criterion. The system is assumed to be a nondegenerate diffusion. We use Lyapunov-like stability conditions that ensure the existence and boundedness of the solution to certain Poisson equation. We also ensure the convergence of a sequence of such solutions, of the corresponding sequence of policies, and, ultimately, of the PIA.

ORDERED FIELD PROPERTY IN A SUBCLASS OF FINITE SER-SIT SEMI-MARKOV GAMES

In this paper, we deal with a subclass of two-person finite SeR-SIT (Separable Reward-State Independent Transition) semi-Markov games which can be solved by solving a single matrix/bimatrix game under discounted as well as limiting average (undiscounted) payoff criteria. A SeR-SIT semi-Markov game does not satisfy the so-called (Archimedean) ordered field property in general. Besides, the ordered field property does not hold even for a SeR-SIT-PT (Separable Reward-State-Independent Transition Probability and Time) semi-Markov game, which is a natural version of a SeR-SIT stochastic (Markov) game. However by using an additional condition, we have shown that a subclass of finite SeR-SIT-PT semi-Markov games have the ordered field property for both discounted and undiscounted semi-Markov games with both players having state-independent stationary optimals. The ordered field property also holds for the nonzero-sum case under the same assumptions. We find a relation between the values of the discounted and the undiscounted zero-sum semi-Markov games for this modified subclass. We propose a more realistic pollution tax model for this subclass of SeR-SIT semi-Markov games than pollution tax model for SeR-SIT stochastic game. Finite step algorithms are given for the discounted and for the zero-sum undiscounted cases.

Zero-Sum Risk-Sensitive Stochastic Differential Games

We study zero-sum risk-sensitive stochastic differential games on the infinite horizon with discounted and ergodic payoff criteria. Under certain assumptions, we establish the existence of values and saddle-point equilibria. We obtain our results by studying the corresponding Hamilton–Jacobi–Isaacs equations. Finally, we show that the value of the ergodic payoff criterion is a constant multiple of the maximal eigenvalue of the generators of the associated nonlinear semigroups.

Zero-Sum Ergodic Semi-Markov Games with Weakly Continuous Transition Probabilities

Zero-sum ergodic semi-Markov games with weakly continuous transition probabilities and lower semicontinuous, possibly unbounded, payoff functions are studied. Two payoff criteria are considered: the ratio average and the time average. The main result concerns the existence of a lower semicontinuous solution to the optimality equation and its proof is based on a fixed-point argument. Moreover, it is shown that the ratio average as well as the time average payoff stochastic games have the same value. In addition, one player possesses an e-optimal stationary strategy (e>0), whereas the other has an optimal stationary strategy.

Nonzero Sum Stochastic Differential Games with Discounted Payoff Criterion: An Approximating Markov Chain Approach

We develop a new constructive method for proving the existence of Nash equilibrium for a class of nonzero sum stochastic differential games. Under certain usual assumptions, we prove the existence of Nash equilibrium for discounted payoff criteria. A novel feature of our method is that it allows us to compute Nash equilibrium for a large class of stochastic differential games.

THE JLR DECISION MODEL FOR CNOs: PAYOFF CRITERIA EXTENSION

Collaborative, networked organizations/enterprises enable powerful competitiveness in today's uncertain market conditions. The aim of this article is to present an extension of the JLR decision model, defined and analyzed by Chituc and Nof, (2005, 2007), by explicitly defining the payoff criteria in a collaborative-competitive economic environment. Specific aspects related to the influence of information sharing, and the strategic approaches adopted to payoff are also discussed. PERA Model, an enterprise model for supporting organizations' alignment and performance assessment in a collaborative-competitive economic environment is then applied to illustrate the value of this JLR payoff criteria extension.

Nonzero-sum games for continuous-time Markov chains with unbounded discounted payoffs

In this paper, we study two-person nonzero-sum games for continuous-time Markov chains with discounted payoff criteria and Borel action spaces. The transition rates are possibly unbounded, and the payoff functions might have neither upper nor lower bounds. We give conditions that ensure the existence of Nash equilibria in stationary strategies. For the zero-sum case, we prove the existence of the value of the game, and also provide arecursiveway to compute it, or at least to approximate it. Our results are applied to a controlled queueing system. We also show that if the transition rates areuniformly bounded, then a continuous-time game is equivalent, in a suitable sense, to a discrete-time Markov game.

Nonzero-sum games for continuous-time Markov chains with unbounded discounted payoffs

In this paper, we study two-person nonzero-sum games for continuous-time Markov chains with discounted payoff criteria and Borel action spaces. The transition rates are possibly unbounded, and the payoff functions might have neither upper nor lower bounds. We give conditions that ensure the existence of Nash equilibria in stationary strategies. For the zero-sum case, we prove the existence of the value of the game, and also provide arecursiveway to compute it, or at least to approximate it. Our results are applied to a controlled queueing system. We also show that if the transition rates areuniformly bounded, then a continuous-time game is equivalent, in a suitable sense, to a discrete-time Markov game.

A nonzero-sum stochastic differential game in the orthant

We study a nonzero-sum stochastic differential game where the state is a controlled reflecting diffusion in the nonnegative orthant. Under certain conditions, we establish the existence of Nash equilibria in stationary strategies for both discounted and average payoff criteria.

Achieving Target State-Action Frequencies in Multichain Average-Reward Markov Decision Processes

In this paper we address a basic problem that arises naturally in average-reward Markov decision processes with constraints and/or nonstandard payoff criteria: Given a feasible state-action frequency vector (“the target”), construct a policy whose state-action frequencies match those of the target vector. While it is well known that the solution to this problem cannot, in general, be found in the space of stationary randomized policies, we construct a solution that has “ultimately stationary” structure: It consists of two stationary policies where the first one is used initially, and then the switch to the second one is made at a certain random switching time. The computational effort required to construct this solution is minimal. We also show that our problem can always be solved by a stationary policy if the original MDP is “extended” by adding certain states and actions. The solution in the original MDP is obtained by mapping the solution in the extended MDP back to the original process.

A Stochastic Differential Game in the Orthrant

We study a zero-sum stochastic differential game in the nonnegative orthrant. The state of the system is governed by controlled reflecting diffusions in the nonnegative orthrant. We consider discounted and average payoff evaluation criteria. We prove the existence of values and optimal strategies for both payoff criteria.

Zero-Sum Stochastic Games in Borel Spaces: Average Payoff Criteria

This paper is concerned with two-person zero-sum dynamic stochastic games in Borel spaces, with possibly unbounded payoff function, and several average (or ergodic) payoff criteria. We give conditions under which the long-run expected average payoff criterion, the sample-path average criterion, the existence of solutions to the average payoff Shapley equations, and a certain "martingale condition" are all equivalent.

Stochastic differential games: Occupation measure based approach

A new approach based on occupation measures is introduced for studying stochastic differential games. For two-person zero-sum games, the existence of values and optimal strategies for both players is established for various payoff criteria. ForN-person games, the existence of equilibria in Markov strategies is established for various cases.