Memoryless Strategies Research Articles

Perspective Games

We introduce and study perspective games , which model multi-agent systems in which agents can view only the parts of the system that they own. As in standard multi-player turn-based games, the vertices of the game graph are partitioned among the players. Starting from an initial vertex, the players jointly generate a computation, with each player deciding the successor vertex whenever the generated computation reaches a vertex she owns. A perspective strategy for a player depends only on the history of visits in her vertices. Thus, unlike observation-based models of partial visibility, where uncertainty is longitudinal—players partially observe all vertices in the history, uncertainty in the perspective model is transverse—players fully observe part of the vertices in the history. We consider deterministic and probabilistic perspective games, with structural (e.g., Büchi or parity) and behavioral (e.g., LTL formulas) winning conditions. For these settings, we study the theoretical properties of the game as well as the decidability and complexity of the problem of deciding whether a player has a winning perspective strategy, in terms of both the game graph and the objectives. We compare perspective strategies with memoryless ones, and study an extension of the temporal logic ATL ⋆ with path quantifiers that capture perspective and memoryless strategies.

Complex objective optimization in fuzzy environments

Multi-objective optimization can be used to address possible conflicting relationships between multiple objectives. However, some objectives have a fuzzy temporal relationship between them, making it difficult to give a common method to portray the fuzzy temporal relationship. To fill this gap, we propose the concept of complex objectives, which can be described by fuzzy temporal logic that includes both temporal and logical operators. Furthermore, we investigated the optimal control problems of complex objectives and developed a fuzzy system called possibilistic decision systems (PDSs) to establish a framework for optimal control. In PDSs, states of fuzzy systems are determined by a family of variables, and transitions induced by actions between fuzzy states of systems are also fuzzy uncertain and determined by a possibility degree. Importantly, we proved that memoryless strategies are sufficient for optimal control of complex objectives. Finally, the theory presented in this paper is illustrated by a mobile robot simulation.

Formal Verification of Bayesian Mechanisms

In this paper, for the first time, we study the formal verification of Bayesian mechanisms through strategic reasoning. We rely on the framework of Probabilistic Strategy Logic (PSL), which is well-suited for representing and verifying multi-agent systems with incomplete information. We take advantage of the recent results on the decidability of PSL model checking under memoryless strategies, and reduce the problem of formally verifying Bayesian mechanisms to PSL model checking. We show how to encode Bayesian-Nash equilibrium and economical properties, and illustrate our approach with different kinds of mechanisms.

Optimal Strategy Synthesis for Stochastic Boolean Games

In this study, we regard a classification of winning strategy and payoff function over infinite plays. The central goal is to examine the values of the game and then to determine the existence of optimal (∈-optimal) strategy. Furthermore, we are interested in the subject of what sort of optimal (∈-optimal) strategy exists. We first review on generalised reachability games and then we introduce a new game, called Boolean games and concentrate on games with a Boolean combination of the reachability games. The primary contribution is on the existence of ∈-optimal finite memory strategy of each player for any ∈ > 0. We also prove every player has no ∈-optimal memoryless strategy for some Boolean game.

H∞ Constrained Pareto Suboptimal Strategy for Stochastic LPV Time-Delay Systems

Not only in control problems, but also in dynamic games, several sources of performance degradation, such as model variation, deterministic and stochastic uncertainties and state delays, need to be considered. In this paper, we present an [Formula: see text] constrained Pareto suboptimal strategy for stochastic linear parameter-varying (LPV) time-delay systems involving multiple decision makers. The goal of developing the [Formula: see text] constrained Pareto suboptimal strategy set is to construct a memoryless state feedback strategy set, so that the closed-loop stochastic LPV system is stochastically mean-square stable. In the paper, the existence condition of the extended bounded real lemma is first established via linear matrix inequalities (LMIs). Then, a quadratic cost bound for cost performance is derived. Based on these preliminary results, sufficient conditions for the existence of such a strategy set under the [Formula: see text] constraint are derived by using cross-coupled bilinear matrix inequalities (BMIs). To determine the strategy set, a viscosity iterative scheme based on the LMIs is established to avoid the processing of BMIs. Finally, two numerical examples are presented to demonstrate the reliability and usefulness of the proposed method.

On the existence of weak subgame perfect equilibria

We study multi-player turn-based games played on (potentially infinite) directed graphs. An outcome is assigned to every play of the game. Each player has a preference relation on the set of outcomes which allows him to compare plays. We focus on the recently introduced notion of weak subgame perfect equilibrium (weak SPE). This is a variant of the classical notion of SPE, where players who deviate can only use strategies deviating from their initial strategy in a finite number of histories. Having an SPE in a game implies having a weak SPE but the contrary is generally false.We propose general conditions on the structure of the game graph and on the preference relations of the players that guarantee the existence of a weak SPE, that additionally is finite-memory. From this general result, we derive two large classes of games for which there always exists a weak SPE: (i) the games with a finite-range outcome function, and (ii) the games with a finite underlying graph and a prefix-independent outcome function. For the second class, we identify conditions on the preference relations that guarantee memoryless strategies for the weak SPE.

Fuzzy Pushdown Termination Games

The computational study on finite/infinite-state systems, probabilistic systems, and finite-state fuzzy systems, has received much attention recently. In contrast, there are very few results for algorithmic analysis of infinite-state fuzzy systems. In this paper, we introduce fuzzy pushdown termination games (FPDTGs), which are an extension of fuzzy pushdown automata with a game feature and can serve as a formal model of infinite-state fuzzy systems. We investigate some computational issues of the games under termination objectives for two players: the goal of player-1 is to maximize the truth value of eventually terminating at some given configurations with the empty stack, while player-2 aims at the opposite. Some interesting results are obtained. For example, we show that both players have optimal memoryless strategies and the same value. The problem of computing the value can be solved in exponential time when the triangular norm is chosen as the minimum one. Furthermore, we present efficient algorithms for computing the values of two special subclasses of FPDTGs. The potential for practical use of our model is demonstrated by a case study on a manufacturing system.

Memoryless determinacy of finite parity games: Another simple proof

Memoryless determinacy of (infinite) parity games is an important result with numerous applications. It was first independently established by Emerson and Jutla [1] and Mostowski [2] but their proofs involve elaborate developments. The elegant and simpler proof of Zielonka [3] still requires a nested induction on the finite number of priorities and on ordinals for sets of vertices. There are other proofs for finite games like the one of Björklund, Sandberg and Vorobyovin [4] that relies on relating infinite and finite duration games. We present here another simple proof that finite parity games are determined with memoryless strategies using induction on the number of relevant states. The closest proof that relies on induction over non absorbing states is the one of Grädel [5]. However instead of focusing on a single appropriate vertex for induction as we do here, he considers two reduced games per vertex, for all the vertices of the game. The idea of reasoning about a single state has been inspired to me by the analysis of finite stochastic priority games by Karelovic and Zielonka [6].

Communication in the Presence of a State-Aware Adversary

We study communication systems over the state-dependent channels in the presence of a malicious state-aware jamming adversary. The channel has a memoryless state with an underlying distribution. The adversary introduces a jamming signal into the channel. The message and the entire state sequence are known non-causally to both the encoder and the adversary. This state-aware adversary may choose an arbitrary jamming vector depending on the message and the state vector. Taking an arbitrarily varying channel (AVC) approach, we consider two setups, namely, the discrete memoryless Gel’fand–Pinsker AVC and the additive white Gaussian dirty paper (DP) AVC. We determine the randomized coding capacity of both the AVCs under a maximum probability of error criterion. Similar to other randomized coding setups, we show that the capacity is the same even under the average probability of error criterion. Though the adversary can choose an arbitrary vector jamming strategy, we prove that the adversary cannot affect the rate any worse than when it employs a memoryless strategy, which depends only on the instantaneous state. Thus, the AVC capacity characterization is given in terms of the capacity of the worst memoryless channels with state, induced by the adversary employing such memoryless jamming strategies. For the DP-AVC, it is further shown that among memoryless jamming strategies, none impact the communication more than a memoryless Gaussian jamming strategy which completely disregards the knowledge of the state. Thus, the capacity of the DP-AVC equals that of a standard additive white Gaussian noise (AWGN) channel with two independent sources of AWGN, i.e., the channel noise and the jamming noise.

Qualitative Determinacy and Decidability of Stochastic Games with Signals

We consider two-person zero-sum stochastic games with signals, a standard model of stochastic games with imperfect information. The only source of information for the players consists of the signals they receive; they cannot directly observe the state of the game, nor the actions played by their opponent, nor their own actions. We are interested in the existence of almost-surely winning or positively winning strategies, under reachability, safety, Büchi, or co-Büchi winning objectives, and the computation of these strategies when the game has finitely many states and actions. We prove two qualitative determinacy results. First, in a reachability game, either player 1 can achieve almost surely the reachability objective, or player 2 can achieve surely the dual safety objective, or both players have positively winning strategies. Second, in a Büchi game, if player 1 cannot achieve almost surely the Büchi objective, then player 2 can ensure positively the dual co-Büchi objective. We prove that players only need strategies with finite memory . The number of memory states needed to win with finite-memory strategies ranges from one (corresponding to memoryless strategies) to doubly exponential, with matching upper and lower bounds. Together with the qualitative determinacy results, we also provide fix-point algorithms for deciding which player has an almost-surely winning or a positively winning strategy and for computing an associated finite-memory strategy. Complexity ranges from EXPTIME to 2EXPTIME, with matching lower bounds. Our fix-point algorithms also enjoy a better complexity in the cases where one of the players is better informed than their opponent. Our results hold even when players do not necessarily observe their own actions. The adequate class of strategies, in this case, is mixed or general strategies (they are equivalent). Behavioral strategies are too restrictive to guarantee determinacy: it may happen that one of the players has a winning general strategy but none of them has a winning behavioral strategy. On the other hand, if a player can observe their actions, then general, mixed, and behavioral strategies are equivalent. Finite-memory strategies are sufficient for determinacy to hold, provided that randomized memory updates are allowed.

A Backward-traversal-based Approach for Symbolic Model Checking of Uniform Strategies for Constrained Reachability

Since the introduction of Alternating-time Temporal Logic (ATL), many logics have been proposed to reason about different strategic capabilities of the agents of a system. In particular, some logics have been designed to reason about the uniform memoryless strategies of such agents. These strategies are the ones the agents can effectively play by only looking at what they observe from the current state. ATL_ir can be seen as the core logic to reason about such uniform strategies. Nevertheless, its model-checking problem is difficult (it requires a polynomial number of calls to an NP oracle), and practical algorithms to solve it appeared only recently. This paper proposes a technique for model checking uniform memoryless strategies. Existing techniques build the strategies from the states of interest, such as the initial states, through a forward traversal of the system. On the other hand, the proposed approach builds the winning strategies from the target states through a backward traversal, making sure that only uniform strategies are explored. Nevertheless, building the strategies from the ground up limits its applicability to constrained reachability objectives only. This paper describes the approach in details and compares it experimentally with existing approaches implemented into a BDD-based framework. These experiments show that the technique is competitive on the cases it can handle.

Reachability in Fuzzy Game Graphs

Two-player turn-based games on graphs (or game graphs for short) and their probabilistic versions have received increasing attention in computer science, especially in the formal verification of reactive systems. However, in the fuzzy setting, game graphs are yet to be addressed, although some practical applications, such as modeling fuzzy systems that interact with their environments, appeal to such models. To fill the gap, in this paper, we propose a fuzzy version of game graphs and focus on the fuzzy game graphs with reachability objectives, which we will refer to as fuzzy reachability games (FRGs). In an FRG, the goal of one player is to maximize her truth value of reaching a given target set, while the other player aims at the opposite. In this framework, we show that FRGs are determined in the sense that for every state, both of the two players have the same value, and there exist optimal memoryless strategies for both players. Moreover, we design algorithms, which achieve polynomial time complexity in the size of the FRG, to compute the values of all states and the optimal memoryless strategies for the players. For a special class of FRGs, we provide an improved algorithm that achieves linear-logarithmic running time to compute the values of states. In addition, several examples are given to illustrate our motivation and the theoretical development.

Design of H2/H∞ RMPC for Boiler Superheated Steam Temperature Based on Memoryless Feedback Multistep Strategy

The collection of superheated steam temperature models of a thermal power plant under different loads can be approximated to “multimodel” linear uncertain systems. After transformation, the tracking system was obtained from “multimodel” linear uncertain systems. For this tracking uncertain system, a mixed H2/H∞ robust model predictive control (HRMPC) based on a memoryless feedback multistep strategy is proposed. A multistep control strategy combines the advantages of predictive control rolling optimization with memoryless feedback control thoughts. It could effectively decrease the controller optimization parameter and ensure closed-loop system stability, and, at the same time, it also achieved acceptable control performance. Successful application to the superheated steam temperature system of a 300 MW thermal power plant verified the study of the HRMPC-P cascade controller design scheme in terms of feasibility and effectiveness.

Average-energy games

Two-player quantitative zero-sum games provide a natural framework to synthesize controllers with performance guarantees for reactive systems within an uncontrollable environment. Classical settings include mean-payoff games, where the objective is to optimize the long-run average gain per action, and energy games, where the system has to avoid running out of energy. We study average-energy games, where the goal is to optimize the long-run average of the accumulated energy. We show that this objective arises naturally in several applications, and that it yields interesting connections with previous concepts in the literature. We prove that deciding the winner in such games is in $$\mathsf{NP}\cap \mathsf{coNP}$$ and at least as hard as solving mean-payoff games, and we establish that memoryless strategies suffice to win. We also consider the case where the system has to minimize the average-energy while maintaining the accumulated energy within predefined bounds at all times: this corresponds to operating with a finite-capacity storage for energy. We give results for one-player and two-player games, and establish complexity bounds and memory requirements.

Efficient approximation of optimal control for continuous-time Markov games

We study the time-bounded reachability problem for continuous-time Markov decision processes (CTMDPs) and games (CTMGs). Existing techniques for this problem use discretisation techniques to partition time into discrete intervals of size ε, and optimal control is approximated for each interval separately. Current techniques provide an accuracy of O(ε2) on each interval, which leads to an infeasibly large number of intervals. We propose a sequence of approximations that achieve accuracies of O(ε3), O(ε4), and O(ε5), that allow us to drastically reduce the number of intervals that are considered. For CTMDPs, the performance of the resulting algorithms is comparable to the heuristic approach given by Buchholz and Schulz, while also being theoretically justified. All of our results generalise to CTMGs, where our results yield the first practically implementable algorithms for this problem. We also provide memoryless strategies for both players that achieve similar error bounds.

Average-energy games

Two-player quantitative zero-sum games provide a natural framework to synthesize controllers with performance guarantees for reactive systems within an uncontrollable environment. Classical settings include mean-payoff games, where the objective is to optimize the long-run average gain per action, and energy games, where the system has to avoid running out of energy. We study average-energy games, where the goal is to optimize the long-run average of the accumulated energy. We show that this objective arises naturally in several applications, and that it yields interesting connections with previous concepts in the literature. We prove that deciding the winner in such games is in NP inter coNP and at least as hard as solving mean-payoff games, and we establish that memoryless strategies suffice to win. We also consider the case where the system has to minimize the average-energy while maintaining the accumulated energy within predefined bounds at all times: this corresponds to operating with a finite-capacity storage for energy. We give results for one-player and two-player games, and establish complexity bounds and memory requirements.

Augmenting [formula omitted] with strategy contexts

We study the extension of the alternating-time temporal logic (ATL) with strategy contexts: contrary to the original semantics, in this semantics the strategy quantifiers do not reset the previously selected strategies.We show that our extension ATLsc is very expressive, but that its decision problems are quite hard: model checking is k-EXPTIME-complete when the formula has k nested strategy quantifiers; satisfiability is undecidable, but we prove that it is decidable when restricting to turn-based games. Our algorithms are obtained through a very convenient translation to QCTL (the computation-tree logic CTL extended with atomic quantification), which we show also applies to Strategy Logic, as well as when strategy quantification ranges over memoryless strategies.

Reasoning about memoryless strategies under partial observability and unconditional fairness constraints

Alternating-time Temporal Logic is a logic to reason about strategies that agents can adopt to achieve a specified collective goal.A number of extensions for this logic exist; some of them combine strategies and partial observability, some others include fairness constraints, but to the best of our knowledge no work provides a unified framework for strategies, partial observability and fairness constraints. Integration of these three concepts is important when reasoning about the capabilities of agents without full knowledge of a system, for instance when the agents can assume that the environment behaves in a fair way.We present ATLKirF, a logic combining strategies under partial observability in a system with fairness constraints on states. We introduce a model-checking algorithm for ATLKirF by extending the algorithm for a full-observability variant of the logic and we investigate its complexity. We validate our proposal with an experimental evaluation.

The complexity of multi-mean-payoff and multi-energy games

In mean-payoff games, the objective of the protagonist is to ensure that the limit average of an infinite sequence of numeric weights is nonnegative. In energy games, the objective is to ensure that the running sum of weights is always nonnegative. Multi-mean-payoff and multi-energy games replace individual weights by tuples, and the limit average (resp., running sum) of each coordinate must be (resp., remain) nonnegative. We prove finite-memory determinacy of multi-energy games and show inter-reducibility of multi-mean-payoff and multi-energy games for finite-memory strategies. We improve the computational complexity for solving both classes with finite-memory strategies: we prove coNP-completeness improving the previous known EXPSPACE bound. For memoryless strategies, we show that deciding the existence of a winning strategy for the protagonist is NP-complete. We present the first solution of multi-mean-payoff games with infinite-memory strategies: we show that mean-payoff-sup objectives can be decided in NP∩coNP, whereas mean-payoff-inf objectives are coNP-complete.

Measuring and Synthesizing Systems in Probabilistic Environments

The traditional synthesis question given a specification asks for the automatic construction of a system that satisfies the specification, whereas often there exists a preference order among the different systems that satisfy the given specification. Under a probabilistic assumption about the possible inputs, such a preference order is naturally expressed by a weighted automaton, which assigns to each word a value, such that a system is preferred if it generates a higher expected value. We solve the following optimal synthesis problem: given an omega-regular specification, a Markov chain that describes the distribution of inputs, and a weighted automaton that measures how well a system satisfies the given specification under the input assumption, synthesize a system that optimizes the measured value. For safety specifications and quantitative measures that are defined by mean-payoff automata, the optimal synthesis problem reduces to finding a strategy in a Markov decision process (MDP) that is optimal for a long-run average reward objective, which can be achieved in polynomial time. For general omega-regular specifications along with mean-payoff automata, the solution rests on a new, polynomial-time algorithm for computing optimal strategies in MDPs with mean-payoff parity objectives. Our algorithm constructs optimal strategies that consist of two memoryless strategies and a counter. The counter is in general not bounded. To obtain a finite-state system, we show how to construct an ϵ-optimal strategy with a bounded counter, for all ϵ > 0. Furthermore, we show how to decide in polynomial time if it is possible to construct an optimal finite-state system (i.e., a system without a counter) for a given specification. We have implemented our approach and the underlying algorithms in a tool that takes qualitative and quantitative specifications and automatically constructs a system that satisfies the qualitative specification and optimizes the quantitative specification, if such a system exists. We present some experimental results showing optimal systems that were automatically generated in this way.