One-player Game Research Articles

Game characterizations for the number of quantifiers

AbstractA game that characterizes equivalence of structures with respect to all first-order sentences containing a given number of quantifiers was introduced by Immerman in 1981. We define three other games and prove that they are all equivalent to the Immerman game, and hence also give a characterization for the number of quantifiers needed for separating structures. In the Immerman game, Duplicator has a canonical optimal strategy, and hence Duplicator can be completely removed from the game by replacing her moves with default moves given by this optimal strategy. On the other hand, in the last two of our games there is no such optimal strategy for Duplicator. Thus, the Immerman game can be regarded as a one-player game, but two of our games are genuine two-player games.

Submixing and shift-invariant stochastic games

We study optimal strategies in two-player stochastic games that are played on a finite graph, equipped with a general payoff function. The existence of optimal strategies that do not make use of memory and randomisation is a desirable property that vastly simplifies the algorithmic analysis of such games. Our main theorem gives a sufficient condition for the maximizer to possess such a simple optimal strategy. The condition is imposed on the payoff function, saying the payoff does not depend on any finite prefix (shift-invariant) and combining two trajectories does not give higher payoff than the payoff of the parts (submixing). The core technical property that enables the proof of the main theorem is that of the existence of ϵ\\documentclass[12pt]{minimal} \\usepackage{amsmath} \\usepackage{wasysym} \\usepackage{amsfonts} \\usepackage{amssymb} \\usepackage{amsbsy} \\usepackage{mathrsfs} \\usepackage{upgreek} \\setlength{\\oddsidemargin}{-69pt} \\begin{document}$$\\epsilon$$\\end{document}-subgame-perfect strategies when the payoff function is shift-invariant. Furthermore, the same techniques can be used to prove a finite-memory transfer-type theorem: namely that for shift-invariant and submixing payoff functions, the existence of optimal finite-memory strategies in one-player games for the minimizer implies the existence of the same in two-player games. We show that numerous classical payoff functions are submixing and shift-invariant.

Stochastic games with lexicographic objectives

AbstractWe study turn-based stochastic zero-sum games with lexicographic preferences over objectives. Stochastic games are standard models in control, verification, and synthesis of stochastic reactive systems that exhibit both randomness as well as controllable and adversarial non-determinism. Lexicographic order allows one to consider multiple objectives with a strict preference order. To the best of our knowledge, stochastic games with lexicographic objectives have not been studied before. For a mixture of reachability and safety objectives, we show that deterministic lexicographically optimal strategies exist and memory is only required to remember the already satisfied and violated objectives. For a constant number of objectives, we show that the relevant decision problem is in $$\textsf{NP}\cap \textsf{coNP}$$ NP ∩ coNP , matching the current known bound for single objectives; and in general the decision problem is $$\textsf{PSPACE}$$ PSPACE -hard and can be solved in $$\textsf{NEXPTIME}\cap \textsf{coNEXPTIME}$$ NEXPTIME ∩ coNEXPTIME . We present an algorithm that computes the lexicographically optimal strategies via a reduction to the computation of optimal strategies in a sequence of single-objectives games. For omega-regular objectives, we restrict our analysis to one-player games, also known as Markov decision processes. We show that lexicographically optimal strategies exist and need either randomization or finite memory. We present an algorithm that solves the relevant decision problem in polynomial time. We have implemented our algorithms and report experimental results on various case studies.

Characterizing Omega-Regularity through Finite-Memory Determinacy of Games on Infinite Graphs

We consider zero-sum games on infinite graphs, with objectives specified as sets of infinite words over some alphabet of colors. A well-studied class of objectives is the one of $\omega$-regular objectives, due to its relation to many natural problems in theoretical computer science. We focus on the strategy complexity question: given an objective, how much memory does each player require to play as well as possible? A classical result is that finite-memory strategies suffice for both players when the objective is $\omega$-regular. We show a reciprocal of that statement: when both players can play optimally with a chromatic finite-memory structure (i.e., whose updates can only observe colors) in all infinite game graphs, then the objective must be $\omega$-regular. This provides a game-theoretic characterization of $\omega$-regular objectives, and this characterization can help in obtaining memory bounds. Moreover, a by-product of our characterization is a new one-to-two-player lift: to show that chromatic finite-memory structures suffice to play optimally in two-player games on infinite graphs, it suffices to show it in the simpler case of one-player games on infinite graphs. We illustrate our results with the family of discounted-sum objectives, for which $\omega$-regularity depends on the value of some parameters.

Games Where You Can Play Optimally with Arena-Independent Finite Memory

For decades, two-player (antagonistic) games on graphs have been a framework of choice for many important problems in theoretical computer science. A notorious one is controller synthesis, which can be rephrased through the game-theoretic metaphor as the quest for a winning strategy of the system in a game against its antagonistic environment. Depending on the specification, optimal strategies might be simple or quite complex, for example having to use (possibly infinite) memory. Hence, research strives to understand which settings allow for simple strategies. In 2005, Gimbert and Zielonka provided a complete characterization of preference relations (a formal framework to model specifications and game objectives) that admit memoryless optimal strategies for both players. In the last fifteen years however, practical applications have driven the community toward games with complex or multiple objectives, where memory -- finite or infinite -- is almost always required. Despite much effort, the exact frontiers of the class of preference relations that admit finite-memory optimal strategies still elude us. In this work, we establish a complete characterization of preference relations that admit optimal strategies using arena-independent finite memory, generalizing the work of Gimbert and Zielonka to the finite-memory case. We also prove an equivalent to their celebrated corollary of great practical interest: if both players have optimal (arena-independent-)finite-memory strategies in all one-player games, then it is also the case in all two-player games. Finally, we pinpoint the boundaries of our results with regard to the literature: our work completely covers the case of arena-independent memory (e.g., multiple parity objectives, lower- and upper-bounded energy objectives), and paves the way to the arena-dependent case (e.g., multiple lower-bounded energy objectives).

Choosing Less over More Money:The Love of Praiseworthiness and the Dread of Blameworthiness in One-Player Games

Why choose less money over more when no one is watching? A central tenet of economics is that this behaviour can be explained by intrinsic motivation. But what does intrinsic motivation entail? What encourages it? This paper answers these questions through a Smithian lens; moral motivation includes not only a naturally strong love of praise and dread of blame but also a natural, and stronger, love of being worthy of praise and dread of being worthy of blame, even if neither is necessarily given. I rely on quantitative and qualitative data from economic experiments to illustrate this claim. While the current scholarship on Smith has applied his theory to situations in which our actions either evoke reactions from others or have monetary consequences for them, I extend his insights to the receiver (Tjotta, 2019) and dice-rolling (Fischbacher & Follmi-Heusi, 2013) games aimed at eliciting self-regarding concerns, actions affecting the interests of only ourselves. I argue that these games accentuate the strength of the love of praiseworthiness in guiding behaviour, emphasising its immediate reference to others and foundation in intentions along with outcomes.

Externalities, stakes, and lying

We study an observed cheating game in which senders can earn money by lying to the experimenter, knowing the experimenter will later be able to observe whether they told the truth or lied. We modify the game by matching each sender with a receiver, whose earnings negatively correlate with the earnings of the sender. Our results show that senders lie less when matched with a receiver who loses money if they lie. However, once we increase the stakes by a factor of five, participants lie as much in the two-player game as in the one-player game. That is, an externality reduces lying but only as long as the stakes are low.

Playing with ghosts in a Dynkin game

We study a class of two-player optimal stopping games (Dynkin games) of preemption type, with uncertainty about the existence of competitors. The set-up is well-suited to model, for example, real options in the context of investors who do not want to publicly reveal their interest in a certain business opportunity. We show that if the underlying process is a Rd-valued, continuous, strong Markov process, and the stopping payoff is a continuous function (with mild integrability properties) there exists a Nash equilibrium in randomised stopping times for the game. Moreover, the equilibrium strategies and the expected payoffs of the two players are computed explicitly in terms of the corresponding one-player game. To the best of our knowledge this is the first paper to address this version of Dynkin games.

Learning Retrosynthetic Planning through Simulated Experience.

The problem of retrosynthetic planning can be framed as a one-player game, in which the chemist (or a computer program) works backward from a molecular target to simpler starting materials through a series of choices regarding which reactions to perform. This game is challenging as the combinatorial space of possible choices is astronomical, and the value of each choice remains uncertain until the synthesis plan is completed and its cost evaluated. Here, we address this search problem using deep reinforcement learning to identify policies that make (near) optimal reaction choices during each step of retrosynthetic planning according to a user-defined cost metric. Using a simulated experience, we train a neural network to estimate the expected synthesis cost or value of any given molecule based on a representation of its molecular structure. We show that learned policies based on this value network can outperform a heuristic approach that favors symmetric disconnections when synthesizing unfamiliar molecules from available starting materials using the fewest number of reactions. We discuss how the learned policies described here can be incorporated into existing synthesis planning tools and how they can be adapted to changes in the synthesis cost objective or material availability.

A game theory appraisal of the insurance hypothesis: Specific polymorphisms in the energy homeostasis network as imprints of a successful minimax strategy.

The existence of specific polymorphisms in genes of key hormones of the energy homeostasis network that have been shown to predispose to obesity and the so-called metabolic syndrome provides further biological support for the proposed insurance hypothesis. In a broader sense, such polymorphisms can be understood as biological imprints of an evolutionarily successful minimax strategy employed by ancient Homo sapiens subpopulations in a one-player game against nature.

Industry 4.0 / Industrial Internet of Things - Related Technologies and Requirements for a Successful Digital Transformation: An Investigation of Manufacturing Businesses Worldwide

The focus of this research study is in the field of future manufacturing, which is characterized by machine-to-machine communications and virtualized processes along the supply and value chain, in what is also termed digital manufacturing. This form of future manufacturing is called Industry 4.0, or the Industrial Internet of Things, and it presents both enormous business opportunities and challenges. Businesses have to radically change their current processes, procedures, products, and services in order to benefit from this development. The aim of this study is to investigate this current development by considering related technologies, business requirements, and strategic approaches that need to be applied by manufacturing companies to perform this transformation successfully. The research study includes a review of relevant literature, and reports on extensive practical research (based on an inductive qualitative research approach), whereby 40 reports from leading consulting companies were analysed, and semi-structured interviews with 11 industry experts were conducted. According to the research findings, this development is widely seen as an industrial revolution with two groups of identified technology driver: the main driver includes Cyber-Physical Systems, Big Data & Analytics, Cloud Computing, and IT-Security Systems, all of which offer tremendous opportunities for manufacturing businesses; and, the second group comprises technologies including Autonomous Robots, Additive Manufacturing, Augmented Reality, and Simulation (all of which are related to this development in a wider sense). From the findings, 10 generalized business requirements have been identified, the most important of which are a ‘Digital Organisational Mind-Set’, ‘Digital Infrastructure’, ‘Future Workforce’, and ‘Collaboration in the Ecosystem’. The other requirements included ‘Data and Cyber Security’, ‘Smart Innovation Processes’, ‘Digital Value and Supply Chain’, ‘Digital Product and Service Portfolio’, Capitalizing on the Value of Data’, and ‘New Types of Business Models’; these are more specific, and depend on individual company circumstances. Based on this study’s findings, a Digital Transformation Model has been proposed, which builds upon the theoretical framework (which comprises all research findings), and which can be applied by the manufacturing industry in their business practices. The main conclusions drawn from this study are that Industry 4.0 (I40) resp. Industrial Internet of Things (IIoT) present significant opportunities and challenges, not only for individual manufacturing businesses, but also for the entire manufacturing industry as well as the global economy. The means of preparing and transforming businesses to meet the demands of I40 resp. IIoT will inevitably take the form of joint projects, and will not be a ‘one-player game’; collaboration and cooperation amongst businesses and other partners will be fundamental. Manufacturing businesses are recommended to start this transformation immediately, regardless of their size and objectives. Companies should follow a systematic and strategic approach that allows them to exploit the short-term and especially long-term potential benefits of I40 resp. IIoT, whilst still being flexible enough to react quickly and embark on new developments.

Bandit-based Monte-Carlo structure learning of probabilistic logic programs

Probabilistic logic programming can be used to model domains with complex and uncertain relationships among entities. While the problem of learning the parameters of such programs has been considered by various authors, the problem of learning the structure is yet to be explored in depth. In this work we present an approximate search method based on a one-player game approach, called LEMUR. It sees the problem of learning the structure of a probabilistic logic program as a multi-armed bandit problem, relying on the Monte-Carlo tree search UCT algorithm that combines the precision of tree search with the generality of random sampling. LEMUR works by modifying the UCT algorithm in a fashion similar to FUSE, that considers a finite unknown horizon and deals with the problem of having a huge branching factor. The proposed system has been tested on various real-world datasets and has shown good performance with respect to other state of the art statistical relational learning approaches in terms of classification abilities.

Ordered Field Property for Semi-Markov Games when One Player Controls Transition Probabilities and Transition Times

Two-person finite semi-Markov games (SMGs) are studied when the transition probabilities and the transition times are controlled by one player at all states. For the discounted games in this class, we prove that the ordered field property holds and there exist optimal/Nash equilibrium stationary strategies for the players. We illustrate that the zero-sum SMGs where only transition probabilities are controlled by one player, do not necessarily satisfy the ordered field property. An algorithm along with a numerical example for the discounted one player control zero-sum SMGs is given via linear programming. For the undiscounted version of such games, we exhibit with an example that if the game ceases to be unichain, an optimal stationary or Markov strategy need not exist, (though in this example of a one-player game we exhibit a semi-stationary optimal strategy/policy). Lastly, we prove that if such games are unichain, then they possess the ordered field property for the undiscounted case as well.

Towards an algorithmic guide to Spiral Galaxies

We study the one-player game Spiral Galaxies from an algorithmic viewpoint. Spiral Galaxies has been shown to be NP-hard [6] more than a decade ago, but so far it seems that no one has dared exploring its algorithmic universe. We take this trip and visit some of its corners.

Decaying Simulation Strategies

The aim of general game playing (GGP) is to create programs capable of playing a wide range of different games at an expert level, given only the rules of the game. The most successful GGP programs currently employ simulation-based Monte Carlo tree search (MCTS). The performance of MCTS depends heavily on the simulation strategy used. In this paper, we investigate the application of a decay factor for two domain-independent simulation strategies: the N-gram selection technique (NST) and the move-average sampling technique (MAST). Three decay factor methods, called move decay, batch decay, and simulation decay, are applied. Furthermore, a combination of move decay and simulation decay is also tested. The decay variants are implemented in the GGP program CadiaPlayer. Four types of games are used: turn taking, simultaneous move, one player, and multiplayer. Except for one-player games, experiments show that decaying can significantly improve the performance of both NST and MAST simulation strategies.

Computer Theorem Proving and HoTT

Theorem-proving is a one-player game. The history of computer programs being the players goes back to 1956 and the ‘LT’ LOGIC THEORY MACHINE of Newell, Shaw and Simon. In game-playing terms, the ‘initial position’ is the core set of axioms chosen for the particular logic and the ‘moves’ are the rules of inference. Now, the Univalent Foundations Program at IAS Princeton and the resulting ‘HoTT’ book on Homotopy Type Theory have demonstrated the success of a new kind of experimental mathematics using computer theorem proving.

More fires and more fighters

Hartnell’s firefighter game models the containment of the spreading of an undesired property within a network. It is a one-player game played in rounds on a graph G in which a fire breaks out at f vertices of G. In each round the fire spreads to neighboring vertices unless the player defends these. The power of the player is limited in the sense that he can defend at most d additional vertices of G in each round. His objective is to save as many vertices as possible from burning. Most research on this game concerned the case f=d=1, which already leads to hard problems even restricted to trees.We study the game for larger values of f and d. We present useful properties of optimal strategies for the game on trees, efficient approximation algorithms, and bounds on the so-called surviving rate.

On discounted approximations of undiscounted stochastic games and Markov decision processes with limited randomness

It is shown that the discount factor needed to solve an undiscounted mean payoff stochastic game to optimality is exponentially close to 1, even in one-player games with a single random node and polynomially bounded rewards and transition probabilities. For the class of the so-called irreducible games with perfect information and a constant number of random nodes, we obtain a pseudo-polynomial algorithm using discounts.

A survey of partial-observation stochastic parity games

We consider two-player zero-sum stochastic games on graphs with ?-regular winning conditions specified as parity objectives. These games have applications in the design and control of reactive systems. We survey the complexity results for the problem of deciding the winner in such games, and in classes of interest obtained as special cases, based on the information and the power of randomization available to the players, on the class of objectives and on the winning mode. On the basis of information, these games can be classified as follows: (a) partial-observation (both players have partial view of the game); (b) one-sided partial-observation (one player has partial-observation and the other player has complete-observation); and (c) complete-observation (both players have complete view of the game). The one-sided partial-observation games have two important subclasses: the one-player games, known as partial-observation Markov decision processes (POMDPs), and the blind one-player games, known as probabilistic automata. On the basis of randomization, (a) the players may not be allowed to use randomization (pure strategies), or (b) they may choose a probability distribution over actions but the actual random choice is external and not visible to the player (actions invisible), or (c) they may use full randomization. Finally, various classes of games are obtained by restricting the parity objective to a reachability, safety, Buchi, or coBuchi condition. We also consider several winning modes, such as sure-winning (i.e., all outcomes of a strategy have to satisfy the winning condition), almost-sure winning (i.e., winning with probability 1), limit-sure winning (i.e., winning with probability arbitrarily close to 1), and value-threshold winning (i.e., winning with probability at least ?, where ? is a given rational).

Probabilistic One-Player Ramsey Games via Deterministic Two-Player Games

Consider the following probabilistic one-player game: The board is a graph with $n$ vertices, which initially contains no edges. In each step, a new edge is drawn uniformly at random from all nonedges and is presented to the player, henceforth called Painter. Painter must assign one of $r$ available colors to each edge immediately, where $r\geq 2$ is a fixed integer. The game is over as soon as a monochromatic copy of some fixed graph $F$ has been created, and Painter's goal is to “survive” for as many steps as possible before this happens. We present a new technique for deriving upper bounds on the threshold of this game, i.e., on the typical number of steps Painter will survive with an optimal strategy. More specifically, we consider a deterministic two-player variant of the game where the edges are chosen not randomly, but by a second player Builder. However, Builder has to adhere to the restriction that, for some real number $d$, the ratio of edges to vertices in all subgraphs of the evolving board never exceeds $d$. We show that the existence of a winning strategy for Builder in this deterministic game implies an upper bound of $n^{2-1/d}$ for the threshold of the original probabilistic game. Moreover, we show that the best bound that can be derived in this way is indeed the threshold of the game if $F$ is a forest. We illustrate these general results with several examples. The technique proposed here has been used by Balogh and Butterfield [Discrete Math., 310 (2010), pp. 3653--3657] to derive the first nontrivial upper bounds for the threshold of the game where $F$ is a triangle and more than two colors are available.