Infinite Action Research Articles

Strategy Complexity of Reachability in Countable Stochastic 2-Player Games

AbstractWe study countably infinite stochastic 2-player games with reachability objectives. Our results provide a complete picture of the memory requirements of $$\varepsilon $$ ε -optimal (resp. optimal) strategies. These results depend on the size of the players’ action sets and on whether one requires strategies that are uniform (i.e., independent of the start state). Our main result is that $$\varepsilon $$ ε -optimal (resp. optimal) Maximizer strategies requires infinite memory if Minimizer is allowed infinite action sets. This lower bound holds even under very strong restrictions. Even in the special case of infinitely branching turn-based reachability games, even if all states allow an almost surely winning Maximizer strategy, strategies with a step counter plus finite private memory are still useless. Regarding uniformity, we show that for Maximizer there need not exist memoryless (i.e., positional) uniformly $$\varepsilon $$ ε -optimal strategies even in the special case of finite action sets or in finitely branching turn-based games. On the other hand, in games with finite action sets, there always exists a uniformly $$\varepsilon $$ ε -optimal Maximizer strategy that uses just one bit of public memory.

Learning the Optimal Control for Evolving Systems with Converging Dynamics

We consider a principle or controller that can pick actions from a fixed action set to control an evolving system with converging dynamics. The converging dynamics means that, if the principle holds the same action, the system will asymptotically converge to a unique stable state determined by this action. In our model, the dynamics of the system are unknown to the principle, and the principle can only receive bandit feedback (maybe noisy) on the impacts of his actions. The principle aims to learn which stable state yields the highest reward while adhering to specific constraints and to immerse the system into this state as quickly as possible. We measure the principle's performance in terms of regret and constraint violation. In cases where the action set is finite, we propose an algorithm Optimistic-Pessimistic Convergence and Confidence Bounds (OP-C2B) that ensures sublinear regret and constraint violation simultaneously. Particularly, OP-C2B achieves logarithmic regret and constraint violation when the system convergence rate is linear or superlinear. Furthermore, we generalize our algorithm OP-C2B to the case of an infinite action set and demonstrate its ability to maintain sublinear regret and constraint violation.

Learning the Optimal Control for Evolving Systems with Converging Dynamics

We consider a principle or controller that can pick actions from a fixed action set to control an evolving system with converging dynamics. The actions are interpreted as different configurations or policies. We consider systems with converging dynamics, i.e., if the principle holds the same action, the system will asymptotically converge (possibly requiring a significant amount of time) to a unique stable state determined by this action. This phenomenon can be observed in diverse domains such as epidemic control, computing systems, and markets. In our model, the dynamics of the system are unknown to the principle, and the principle can only receive bandit feedback (maybe noisy) on the impacts of his actions. The principle aims to learn which stable state yields the highest reward while adhering to specific constraints (i.e., optimal stable state) and to immerse the system into this state as quickly as possible. A unique challenge in our model is that the principle has no prior knowledge about the stable state of each action, but waits for the system to converge to the suboptimal stable states costs valuable time. We measure the principle's performance in terms of regret and constraint violation. In cases where the action set is finite, we propose a novel algorithm, termed Optimistic-Pessimistic Convergence and Confidence Bounds (OP-C2B), that knows to switch an action quickly if it is not worth waiting until the stable state is reached. This is enabled by employing "convergence bounds" to determine how far the system is from the stable states, and choosing actions through maintaining a pessimistic assessment of the set of feasible actions while acting optimistically within this set. We establish that OP-C2B can ensure sublinear regret and constraint violation simultaneously. Particularly, OP-C2B achieves logarithmic regret and constraint violation when the system convergence rate is linear or superlinear. Furthermore, we generalize our algorithm OP-C2B to the case of an infinite action set and demonstrate its ability to maintain sublinear regret and constraint violation. We finally show two game control problems including mobile crowdsensing and resource allocation that our model can address.

Higher-order gravity, finite action, and a safe beginning for the universe

General relativity allows for inhomogeneous and anisotropic universes with finite action. By contrast, in quadratic gravity such solutions obtain infinite action and are thus eliminated. What remains are homogeneous and isotropic solutions undergoing accelerated expansion, thereby automatically inducing an early inflationary phase. In this manner, semi-classical consistency may explain some of the basic, coarse-grained features of the early universe. This includes suitable initial conditions for the second law of thermodynamics, in the spirit of the Weyl curvature hypothesis. We note that quadratic gravity is a renormalisable theory and may admit an asymptotically safe regime at high energies, rendering the theory trustworthy to high energies. We also comment on theories containing curvature terms up to infinite derivative order, and on the contrast with no-boundary initial conditions.

Yang–Mills solutions on Minkowski space via non-compact coset spaces

We find a two-parameter family of solutions of the Yang–Mills equations for gauge group SO(1,3) on Minkowski space by foliating different parts of it with non-compact coset spaces with SO(1,3) isometry. The interior of the lightcone is foliated with hyperbolic space H3≅SO(1,3)/SO(3), while the exterior of the lightcone employs de Sitter space dS≅3SO(1,3)/SO(1,2). The lightcone itself is parametrized by SO(1,3)/ISO(2) in a nilpotent fashion. Equivariant reduction of the SO(1,3) Yang–Mills system on the first two coset spaces yields a mechanical system with inverted double-well potential and the foliation parameter serving as an evolution parameter. Its known analytic solutions are periodic or runaway except for the kink. On the lightcone, only the vacuum solution remains. The constructed Yang–Mills field strength is singular across the lightcone and of infinite action due to the noncompact cosets. Its energy-momentum tensor takes a very simple form, with energy density of opposite signs inside and outside the lightcone.

On the approximate purification of mixed strategies in games with infinite action sets

We consider a game in which the action set of each player is uncountable, and show that, from weak assumptions on the common prior, any mixed strategy has an approximately equivalent pure strategy. The assumption of this result can be further weakened if we consider the purification of a Nash equilibrium. Combined with the existence theorem for a Nash equilibrium, we derive an existence theorem for a pure strategy approximated Nash equilibrium under sufficiently weak assumptions. All of the pure strategies we derive in this paper can take a finite number of possible actions.

On the existence of Pareto undominated mixed-strategy Nash equilibrium in normal-form games with infinite actions

We show that each mixed strategy of a normal-form game can be reformulated as a pure strategy of an induced Bayesian game. Moreover, a normal-form game has a Pareto undominated mixed-strategy Nash equilibrium if and only if its induced Bayesian game has a pure-strategy Bayesian Nash equilibrium. By relying on the existence result of Pareto undominated pure-strategy Nash equilibrium in Bayesian games in Fu and Yu (2015), we also show that every normal-form game has a Pareto undominated mixed-strategy Nash equilibrium.

The Topological Entropy of Stable Sets for Bi-orderable Amenable Groups

We study the topological entropy of stable sets for countable discrete infinite bi-orderable amenable group actions. It is shown that in any positive entropy system, there is a measure-theoretically “large” set such that the topological entropy of stable set of any point from the set is no less than the measure-theoretic entropy of the system, moreover the closure of the stable set of any point from the set contains a weak mixing set.

Self-accelerated Thompson sampling with near-optimal regret upper bound

Thompson sampling utilizes Bayesian heuristic strategy to balance the exploration-exploitation trade-off. It has been applied in a variety of practical domains and achieved great success. Despite being empirically efficient and powerful, Thompson sampling has eluded theoretical analysis. Existing analyses of Thompson sampling only provide regret upper bound of O˜(d3/2T) for linear contextual bandits, which is worse than the information-theoretic lower bound by a factor of d. In this paper, we design and analyze self-accelerated Thompson Sampling algorithm for the stochastic contextual multi-armed bandit problem. Our analysis establishes that the regret upper bound of self-accelerated Thompson sampling is O˜(dT), which is the best upper bound achieved by any efficient contextual bandit algorithm in infinite action space. Our experiment on simulated data and real-world dataset shows that self-accelerated Thompson sampling outperforms standard Thompson sampling in both convergence rate and prediction accuracy.

Weak Closure of Infinite Actions of Rank 1, Joinings, and Spectrum

It is proved that the ergodic self-joining of an infinite transformation of rank 1 is part of the weak limit of shifts of a diagonal measure. A continuous class of nonisomorphic transformations with polynomial closure is proposed. These transformations possess minimal self-joinings and certain unusual spectral properties. Thus, for example, the tensor products of the powers of transformations have both a singular and a Lebesgue spectrum, depending on the choice of the power.

Generalized Backward Induction: Justification for a Folk Algorithm

I introduce axiomatically infinite sequential games that extend Kuhn’s classical framework. Infinite games allow for (a) imperfect information, (b) an infinite horizon, and (c) infinite action sets. A generalized backward induction (GBI) procedure is defined for all such games over the roots of subgames. A strategy profile that survives backward pruning is called a backward induction solution (BIS). The main result of this paper finds that, similar to finite games of perfect information, the sets of BIS and subgame perfect equilibria (SPE) coincide for both pure strategies and for behavioral strategies that satisfy the conditions of finite support and finite crossing. Additionally, I discuss five examples of well-known games and political economy models that can be solved with GBI but not classic backward induction (BI). The contributions of this paper include (a) the axiomatization of a class of infinite games, (b) the extension of backward induction to infinite games, and (c) the proof that BIS and SPEs are identical for infinite games.

Amenable purely infinite actions on the non-compact Cantor set

We prove that any countable non-amenable group $\unicode[STIX]{x1D6E4}$ admits a free, minimal, amenable, purely infinite action on the non-compact Cantor set. This answers a question of Kellerhals, Monod and Rørdam [Non-supramenable groups acting on locally compact spaces. Doc. Math.18 (2013), 1597–1626].

Market Model Benchmark Suite for Machine Learning Techniques

Recent developments in deep-reinforcement learning have yielded promising results in artificial games and test domains. To explore opportunities and evaluate the performance of these machine learning techniques, various benchmark suites are available, such as the Arcade Learning Environment, rllab, OpenAI Gym, and the StarCraft II Learning Environment. This set of benchmark suites is extended with the open business simulation model described here, which helps to promote the use of machine learning techniques as valueadding tools in the context of strategic decision making and economic model calibration and harmonization. The benchmark suite extends the current state-of-the-art problems for deep-reinforcement learning by offering an infinite state and action space for multiple players in a non-zero-sum game environment of imperfect information. It provides a model that can be characterized as both a credit assignment problem and an optimization problem. Experiments with this suite?s deep-reinforcement learning algorithms, which yield remarkable results for various artificial games, highlight that stylized market behavior can be replicated, but the infinite action space, simultaneous decision making, and imperfect information pose a computational challenge. With the directions provided, the benchmark suite can be used to explore new solutions in machine learning for strategic decision making and model calibration.

Yang-Mills solutions on de Sitter space of any dimension

For gauge groups $\mathrm{SO}(n+1)$, $\mathrm{SU}(m+1)$, and $\mathrm{Sp}(\ensuremath{\ell}+1)$, we construct equivariant Yang-Mills solutions on de Sitter space in $n+1$, $2(m+1)$, and $4(\ensuremath{\ell}+1)$ spacetime dimensions. The latter is conformally mapped to a finite cylinder over a coset space realizing an appropriate unit sphere. The equivariance condition reduces the Yang-Mills system to an analog Newtonian particle in one or two dimensions subject to a time-dependent friction and a particular potential. We analyze some properties of the solutions such as their action and energy and display all analytic ones. Beyond ${\mathrm{dS}}_{4}$, all such configurations have finite energy but infinite action.

Minimal self-joinings of infinite mixing actions of rank 1

We prove that measure-preserving actions of rank 1 of the groups ℤ n and ℝ n on a Lebesgue space with a σ-finite measure have minimal self-joinings.

P19: Increased femoral anteversion related biomechanical abnormalities: Gait and hyper mobility

We consider two-player zero-sum games with infinite action spaces and bounded payoff functions. The players' strategies are finitely additive probability measures, called charges. Since a strategy profile does not always induce a unique expected payoff, we distinguish two extreme attitudes of players. A player is viewed as pessimistic if he always evaluates the range of possible expected payoffs by the worst one, and a player is viewed as optimistic if he always evaluates it by the best one. This approach results in a definition of a pessimistic and an optimistic guarantee level for each player. We provide an extensive analysis of the relation between these guarantee levels, and connect them to the classical guarantee levels, and to other known techniques to define expected payoffs, based on computation of double integrals. In addition, we also examine existence of optimal strategies with respect to these guarantee levels.

Infinite tensor product actions of compact quantum groups

Infinite tensor product actions of compact quantum groups

Zero-sum games with charges

We consider two-player zero-sum games with infinite action spaces and bounded payoff functions. The players' strategies are finitely additive probability measures, called charges. Since a strategy profile does not always induce a unique expected payoff, we distinguish two extreme attitudes of players. A player is viewed as pessimistic if he always evaluates the range of possible expected payoffs by the worst one, and a player is viewed as optimistic if he always evaluates it by the best one. This approach results in a definition of a pessimistic and an optimistic guarantee level for each player. We provide an extensive analysis of the relation between these guarantee levels, and connect them to the classical guarantee levels, and to other known techniques to define expected payoffs, based on computation of double integrals. In addition, we also examine existence of optimal strategies with respect to these guarantee levels.

Equilibrium existence for large perfect information games

This paper provides a novel existence theorem for subgame perfect equilibria of potentially large extensive form games with perfect information and continuous preferences, allowing for infinite horizon and infinite action spaces. The approach is based on the properties of the topology on the space of outcomes and differs from all previous approaches in the literature. Furthermore, the existence proof relies on a new algorithm that is independent of the horizon, hence can also be applied to infinite-horizon games.

Learning to Optimize via Posterior Sampling

This paper considers the use of a simple posterior sampling algorithm to balance between exploration and exploitation when learning to optimize actions such as in multiarmed bandit problems. The algorithm, also known as Thompson Sampling and as probability matching, offers significant advantages over the popular upper confidence bound (UCB) approach, and can be applied to problems with finite or infinite action spaces and complicated relationships among action rewards. We make two theoretical contributions. The first establishes a connection between posterior sampling and UCB algorithms. This result lets us convert regret bounds developed for UCB algorithms into Bayesian regret bounds for posterior sampling. Our second theoretical contribution is a Bayesian regret bound for posterior sampling that applies broadly and can be specialized to many model classes. This bound depends on a new notion we refer to as the eluder dimension, which measures the degree of dependence among action rewards. Compared to UCB algorithm Bayesian regret bounds for specific model classes, our general bound matches the best available for linear models and is stronger than the best available for generalized linear models. Further, our analysis provides insight into performance advantages of posterior sampling, which are highlighted through simulation results that demonstrate performance surpassing recently proposed UCB algorithms.