Mean-payoff Games Research Articles

On the Existence of Stationary Nash Equilibria for Mean Payoff Games on Graphs

In this paper we extend the classical concept of positional strategies for a mean payoff game to a general mixed stationary strategy approach, and prove the existence of mixed stationary Nash equilibria for an arbitrary m-player mean payoff game on graphs. Traditionally, a positional strategy represents a pure stationary strategy in a classical mean payoff game, where a Nash equilibrium in pure stationary strategies in general may not exist. Based on a constructive proof of the existence of specific equilibria for an m-player mean payoff game we propose a new approach for determining the optimal mixed stationary strategies. Additionally we characterize and extend the general problem of the existence of pure stationary Nash equilibria for some special classes of mean payoff games.

Subgame-perfect Equilibria in Mean-payoff Games (journal version)

In this paper, we provide an effective characterization of all the subgame-perfect equilibria in infinite duration games played on finite graphs with mean-payoff objectives. To this end, we introduce the notion of requirement, and the notion of negotiation function. We establish that the plays that are supported by SPEs are exactly those that are consistent with a fixed point of the negotiation function. Finally, we use that characterization to prove that the SPE threshold problem, who status was left open in the literature, is decidable.

Tropical Linear Regression and Mean Payoff Games: Or, How to Measure the Distance to Equilibria

Tropical Linear Regression and Mean Payoff Games: Or, How to Measure the Distance to Equilibria

The Theory of Universal Graphs for Infinite Duration Games

We introduce the notion of universal graphs as a tool for constructing algorithms solving games of infinite duration such as parity games and mean payoff games. In the first part we develop the theory of universal graphs, with two goals: showing an equivalence and normalisation result between different recently introduced related models, and constructing generic value iteration algorithms for any positionally determined objective. In the second part we give four applications: to parity games, to mean payoff games, to a disjunction between a parity and a mean payoff objective, and to disjunctions of several mean payoff objectives. For each of these four cases we construct algorithms achieving or improving over the best known time and space complexity.

Tropical pseudolinear and pseudoquadratic optimization as parametric mean-payoff games

We apply an approach based on parametric mean-payoff games to develop bisection and Newton schemes for solving problems of tropical pseudolinear and pseudoquadratic optimization with general two-sided constraints.

Mean-Payoff Games with ω-Regular Specifications

Multi-player mean-payoff games are a natural formalism for modelling the behaviour of concurrent and multi-agent systems with self-interested players. Players in such a game traverse a graph, while attempting to maximise a (mean-)payoff function that depends on the play generated. As with all games, the equilibria that could arise may have undesirable properties. However, as system designers, we typically wish to ensure that equilibria in such systems correspond to desirable system behaviours, for example, satisfying certain safety or liveness properties. One natural way to do this would be to specify such desirable properties using temporal logic. Unfortunately, the use of temporal logic specifications causes game theoretic verification problems to have very high computational complexity. To address this issue, we consider ω-regular specifications. These offer a concise and intuitive way of specifying system behaviours with a comparatively low computational overhead. The main results of this work are characterisation and complexity bounds for the problem of determining if there are equilibria that satisfy a given ω-regular specification in a multi-player mean-payoff game in a number of computationally relevant game-theoretic settings.

Solving Mean-Payoff Games Via Quasi Dominions

We propose a novel algorithm for the solution of mean-payoff games that merges together two seemingly unrelated concepts introduced in the context of parity games, small progress measures and quasi dominions. We show that the integration of the two notions can be highly beneficial and significantly speeds up convergence to the problem solution. Experiments show that the resulting algorithm performs orders of magnitude better than the asymptotically-best solution algorithm currently known, without sacrificing on the worst-case complexity.

Infinite-duration Bidding Games

<?tight?>Two-player games on graphs are widely studied in formal methods, as they model the interaction between a system and its environment. The game is played by moving a token throughout a graph to produce an infinite path. There are several common modes to determine how the players move the token through the graph; e.g., in turn-based games the players alternate turns in moving the token. We study the bidding mode of moving the token, which, to the best of our knowledge, has never been studied in infinite-duration games. The following bidding rule was previously defined and called Richman bidding. Both players have separate budgets , which sum up to 1. In each turn, a bidding takes place: Both players submit bids simultaneously, where a bid is legal if it does not exceed the available budget, and the higher bidder pays his bid to the other player and moves the token. The central question studied in bidding games is a necessary and sufficient initial budget for winning the game: a threshold budget in a vertex is a value t ∈ [0, 1] such that if Player 1’s budget exceeds t , he can win the game; and if Player 2’s budget exceeds 1 − t , he can win the game. Threshold budgets were previously shown to exist in every vertex of a reachability game, which have an interesting connection with random-turn games—a sub-class of simple stochastic games in which the player who moves is chosen randomly. We show the existence of threshold budgets for a qualitative class of infinite-duration games, namely parity games, and a quantitative class, namely mean-payoff games. The key component of the proof is a quantitative solution to strongly connected mean-payoff bidding games in which we extend the connection with random-turn games to these games, and construct explicit optimal strategies for both players.

Enforcing opacity by insertion functions under multiple energy constraints

This paper investigates the enforcement of opacity by insertion functions subject to multiple quantitative constraints capturing resource or energy limitations. There is a malicious intruder attempting to infer secrets of the system from its observations. To prevent the disclosure of secrets, the insertion function inserts fictitious events to the output of the system to obfuscate the intruder. The system is initialized with several types of resources, referred to as energy. The energy is consumed or replenished with event occurrences while always consumed with event insertions. The insertion function must enforce opacity while ensuring that each type of resource is never depleted. This problem is then reduced to a two-player game between the insertion function and the system (environment), with properly defined objectives. A game structure called the Energy Insertion Structure, denoted by EIS is proposed, which provably contains solutions to the energy constrained opacity enforcement problem. Then we further study the bounded cost rate insertion problem on the insertion function’s winning region of EIS, which requires that the long run average rate of insertion cost be bounded. This problem is formulated as a multidimensional mean payoff game and a special method called hyperplane separation technique is applied to efficiently solve it.

The Operator Approach to Entropy Games

Entropy games and matrix multiplication games have been recently introduced by Asarin et al. They model the situation in which one player (Despot) wishes to minimize the growth rate of a matrix product, whereas the other player (Tribune) wishes to maximize it. We develop an operator approach to entropy games. This allows us to show that entropy games can be cast as stochastic mean payoff games in which some action spaces are simplices and payments are given by a relative entropy (Kullback-Leibler divergence). In this way, we show that entropy games with a fixed number of states belonging to Despot can be solved in polynomial time. This approach also allows us to solve these games by a policy iteration algorithm, which we compare with the spectral simplex algorithm developed by Protasov.

Log-Sum-Exp Neural Networks and Posynomial Models for Convex and Log-Log-Convex Data.

In this paper, we show that a one-layer feedforward neural network with exponential activation functions in the inner layer and logarithmic activation in the output neuron is a universal approximator of convex functions. Such a network represents a family of scaled log-sum exponential functions, here named log-sum-exp ( LSET ). Under a suitable exponential transformation, the class of LSET functions maps to a family of generalized posynomials GPOST , which we similarly show to be universal approximators for log-log-convex functions. A key feature of an LSET network is that, once it is trained on data, the resulting model is convex in the variables, which makes it readily amenable to efficient design based on convex optimization. Similarly, once a GPOST model is trained on data, it yields a posynomial model that can be efficiently optimized with respect to its variables by using geometric programming (GP). The proposed methodology is illustrated by two numerical examples, in which, first, models are constructed from simulation data of the two physical processes (namely, the level of vibration in a vehicle suspension system, and the peak power generated by the combustion of propane), and then optimization-based design is performed on these models.

On Solving Mean Payoff Games Using Pivoting Algorithms

Two classical pivoting algorithms, due to Lemke and Cottle–Dantzig, are studied on linear complementarity problems (LCPs) and their generalizations that arise from infinite duration two-person mean payoff games (MPGs) under zero-mean partition problem. Lemke’s algorithm was studied in solving MPGs via reduction to discounted payoff games or to simple stochastic games. We provide an alternative and efficient approach for studying the LCPs arising from the MPGs without any reduction. A binary MPG can easily be formulated as an LCP which has always terminated in a complementary solution in numerical experiments, but has not yet been proven either the processability of MPG’s by Lemke’s algorithm or a counter example that it will not terminate with a solution. Till now, the processability of MPG’s by Lemke’s algorithm remains open. A general MPG (with arbitrary outgoing arcs) naturally reduces to a generalized linear complementarity problem (GLCP) involving a rectangular matrix where the vertices are represented by the columns and the outgoing arcs from each vertex are represented by rows in a particular block. The noteworthy result in this paper is that the GLCP obtained from an MPG is processable by Cottle–Dantzig principal pivoting algorithm which terminates with a solution. Several properties of the matrix which arise in this context are also discussed.

On two-sided Max-Linear equations

This paper is a contribution to the study of finite, two-sided max-linear systems Ax=Bx over the max-plus algebra. These systems are known to be equivalent to mean-payoff games and the problem of deciding whether or not a non-trivial solution exists is in NP∩co−NP. Yet, no polynomial solution method seems to be known to date.We study two special types of these systems with square matrices A and B. The first type, called minimally active, is defined by the requirement that for every non-trivial solution x, the maximum on each side of every equation is attained exactly once. For the second type, called essential systems, we require that every component of any non-trivial solution is active on at least one side of at least one equation. Minimally active systems are shown to be a special case of essential systems and it is shown that all essential systems can be reduced to minimally active ones. Essential systems are equivalently defined as those square finite systems for which all non-trivial solutions are finite.We prove that in every solvable two-sided max-linear system of minimally active or essential type, all positions in C≔A⊕B active in any optimal permutation for the assignment problem for C, are also active for some non-trivial solution (any non-trivial solution in the minimally active case) of the two-sided system. This enables us to deduce conditions on a solution x for which it is possible in some cases to find x in polynomial time. It is also proved that any essential system can be transformed to a minimally active system in polynomial time.

Looking at mean payoff through foggy windows

Mean-payoff games (MPGs) are infinite duration two-player zero-sum games played on weighted graphs. Under the hypothesis of perfect information, they admit memoryless optimal strategies for both players and can be solved in Open image in new window . MPGs are suitable quantitative models for open reactive systems. However, in this context the assumption of perfect information is not always realistic. For the partial-observation case, the problem that asks if the first player has an observation-based winning strategy that enforces a given threshold on the mean-payoff, is undecidable. In this paper, we study the window mean-payoff objectives introduced recently as an alternative to the classical mean-payoff objectives. We show that, in sharp contrast to the classical mean-payoff objectives, some of the window mean-payoff objectives are decidable in games with partial-observation.

Checking dynamic consistency of conditional hyper temporal networks via mean payoff games: Hardness and (pseudo) singly-exponential time algorithm

Conditional Simple Temporal Network (CSTN ) is a constraint-based graph formalism for conditional temporal planning, which may be viewed as an extension of Simple Temporal Networks. Recently, STN s have been generalized into Hyper Temporal Networks (HyTN s), by considering weighted directed hypergraphs where each hyperarc models a disjunctive temporal constraint. We introduce the Conditional Hyper Temporal Network (CHyTN) model, a natural extension and generalization of both CSTN s and HyTN s, obtained by blending them together. We show that deciding whether a given CSTN is dynamically-consistent is coNP-hard, and that deciding whether a given CHyTN is dynamically-consistent is PSPACE-hard. Next, we offer the first deterministic (pseudo) singly-exponential time algorithm for checking DC in CHyTNs and CSTN s. To analyze the computational complexity of the proposed algorithm, we introduce a refined notion of DC, named ϵ-DC, presenting a sharp lower bounding analysis on the critical value of the reaction time where a conditional temporal network transits from being, to not being, dynamically-consistent.

Generic uniqueness of the bias vector of finite zero-sum stochastic games with perfect information

Mean-payoff zero-sum stochastic games can be studied by means of a nonlinear spectral problem. When the state space is finite, the latter consists in finding an eigenpair (u,λ) solution of T(u)=λe+u, where T:Rn→Rn is the Shapley (or dynamic programming) operator, λ is a scalar, e is the unit vector, and u∈Rn. The scalar λ yields the mean payoff per time unit, and the vector u, called bias, allows one to determine optimal stationary strategies in the mean-payoff game. The existence of the eigenpair (u,λ) is generally related to ergodicity conditions. A basic issue is to understand for which classes of games the bias vector is unique (up to an additive constant). In this paper, we consider perfect-information zero-sum stochastic games with finite state and action spaces, thinking of the transition payments as variable parameters, transition probabilities being fixed. We show that the bias vector, thought of as a function of the transition payments, is generically unique (up to an additive constant). The proof uses techniques of nonlinear Perron–Frobenius theory. As an application of our results, we obtain an explicit perturbation scheme allowing one to solve degenerate instances of stochastic games by policy iteration.

Hyperplane separation technique for multidimensional mean-payoff games

We consider finite-state and recursive game graphs with multidimensional mean-payoff objectives. In recursive games two types of strategies are relevant: global strategies and modular strategies. Our contributions are: (1) We show that finite-state multidimensional mean-payoff games can be solved in polynomial time if the number of dimensions and the maximal absolute value of weights are fixed; whereas for arbitrary dimensions the problem is coNP-complete. (2) We show that one-player recursive games with multidimensional mean-payoff objectives can be solved in polynomial time. Both above algorithms are based on hyperplane separation technique. (3) For recursive games we show that under modular strategies the multidimensional problem is undecidable. We show that if the number of modules, exits, and the maximal absolute value of the weights are fixed, then one-dimensional recursive mean-payoff games under modular strategies can be solved in polynomial time, whereas for unbounded number of exits or modules the problem is NP-hard.

Tropically Convex Constraint Satisfaction

A semilinear relation $S \subseteq {\mathbb {Q}}^{n}$ is max-closed if it is preserved by taking the componentwise maximum. The constraint satisfaction problem for max-closed semilinear constraints is at least as hard as determining the winner in Mean Payoff Games, a notorious problem of open computational complexity. Mean Payoff Games are known to be in N P ∩ c o − N P, which is not known for max-closed semilinear constraints. Semilinear relations that are max-closed and additionally closed under translations have been called tropically convex in the literature. One of our main results is a new duality for open tropically convex relations, which puts the CSP for tropically convex semilinear constraints in general into N P ∩ c o − N P. This extends the corresponding complexity result for scheduling under and-or precedence constraints, or equivalently the max-atoms problem. To this end, we present a characterization of max-closed semilinear relations in terms of syntactically restricted first-order logic, and another characterization in terms of a finite set of relations L that allow primitive positive definitions of all other relations in the class. We also present a subclass of max-closed constraints where the CSP is in P; this class generalizes the class of max-closed constraints over finite domains, and the feasibility problem for max-closed linear inequalities. Finally, we show that the class of max-closed semilinear constraints is maximal in the sense that as soon as a single relation that is not max-closed is added to L, the CSP becomes N P-hard.

Mean-payoff games with partial observation

Mean-payoff games are important quantitative models for open reactive systems. They have been widely studied as games of full observation. In this paper we investigate the algorithmic properties of several sub-classes of mean-payoff games where the players have asymmetric information about the state of the game. These games are in general undecidable and not determined according to the classical definition. We show that such games are determined under a more general notion of winning strategy. We also consider mean-payoff games where the winner can be determined by the winner of a finite cycle-forming game. This yields several decidable classes of mean-payoff games of asymmetric information that require only finite-memory strategies, including a generalization of full-observation games where positional strategies are sufficient. We give an exponential time algorithm for determining the winner of the latter.

Strategy recovery for stochastic mean payoff games

We prove that to find optimal positional strategies for stochastic mean payoff games when the value of every state of the game is known, in general, is as hard as solving such games tout court. This answers a question posed by Daniel Andersson and Peter Bro Miltersen.