Minimax Games Research Articles

GANs training: A game and stochastic control approach

AbstractTraining generative adversarial networks (GANs) are known to be difficult, especially for financial time series. This paper first analyzes the well‐posedness problem in GANs minimax games and the widely recognized convexity issue in GANs objective functions. It then proposes a stochastic control framework for hyper‐parameters tuning in GANs training. The weak form of dynamic programming principle and the uniqueness and the existence of the value function in the viscosity sense for the corresponding minimax game are established. In particular, explicit forms for the optimal adaptive learning rate and batch size are derived and are shown to depend on the convexity of the objective function, revealing a relation between improper choices of learning rate and explosion in GANs training. Finally, empirical studies demonstrate that training algorithms incorporating this adaptive control approach outperform the standard ADAM method in terms of convergence and robustness. From GANs training perspective, the analysis in this paper provides analytical support for the popular practice of “clipping,” and suggests that the convexity and well‐posedness issues in GANs may be tackled through appropriate choices of hyper‐parameters.

Generative Adversarial Networks are special cases of Artificial Curiosity (1990) and also closely related to Predictability Minimization (1991)

I review unsupervised or self-supervised neural networks playing minimax games in game-theoretic settings: (i) Artificial Curiosity (AC, 1990) is based on two such networks. One network learns to generate a probability distribution over outputs, the other learns to predict effects of the outputs. Each network minimizes the objective function maximized by the other. (ii) Generative Adversarial Networks (GANs, 2010-2014) are an application of AC where the effect of an output is 1 if the output is in a given set, and 0 otherwise. (iii) Predictability Minimization (PM, 1990s) models data distributions through a neural encoder that maximizes the objective function minimized by a neural predictor of the code components. I correct a previously published claim that PM is not based on a minimax game.

Synthesis of Optimal Insertion Functions for Opacity Enforcement

Our prior work has studied the enforcement of opacity security properties using insertion functions. Given a system that is not opaque, the so-called All Insertion Structure (AIS) is a game structure, played by the system and the insertion function, that embeds all valid insertion functions. In this paper, we first propose a more compact AIS that can be constructed with lower computational complexity. We then introduce the maximum total cost and the maximum mean cost, and use them as quantitative objectives to solve for optimal insertion functions. Specifically, we first determine if an insertion function with a finite total cost exists. If such an insertion function exists, we synthesize an optimal total-cost insertion function. Otherwise, we construct an optimal mean-cost insertion function. In either case, we find an optimal insertion strategy on the AIS, with respect to the corresponding cost objective. The algorithmic procedures are adapted from results developed for minimax games and mean payoff games. The resulting optimal strategy is represented as a subgraph of the AIS that consists of all the system actions and the optimal insertion actions. Finally, we use this subgraph to synthesize an optimal insertion function that is encoded as an I/O automaton.

Stochastic Uncertain Systems Subject to Relative Entropy Constraints: Induced Norms and Monotonicity Properties of Minimax Games

Entropy and relative entropy are fundamental concepts on which information theory is founded on, and in general, telecommunication systems design. On the other hand, dissipation inequalities, minimax strategies, and induced norms are the basic concepts on which robustness of uncertain control and estimation of systems are founded on. In this paper, the precise relation between these notions is investigated. In particular, it will be shown that the higher the dissipation the higher the entropy of the system, which has implications in computing the induced norm associated with robustness. These connections are obtained by considering stochastic optimal uncertain control systems, in which uncertainty is described by a relative entropy constraint between the nominal and uncertain measures, while the pay-off is a linear functional of the uncertain measure. This is a minimax game, in which the controller measure seeks to minimize the pay-off, while the disturbance measure aims at maximizing the pay-off. Salient properties of the minimax solution are derived, including a characterization of the optimal sensitivity reduction, computation of the induced norm, monotonicity properties of minimax solution, and relations between dissipation and relative entropy of the system. The theory is developed in an abstract setting and then applied to nonlinear partially observable continuous-time uncertain controlled systems, in which the nominal and uncertain systems are described by conditional distributions. In addition, existence of the optimal control policy among the class of policies known as wide-sense control laws is shown, and an explicit formulae for the worst case conditional measure is derived. The results are applied to linear-quadratic-Gaussian problems

Minimax games for stochastic systems subject to relative entropy uncertainty: applications to SDEs on Hilbert spaces

In this paper, we consider minimax games for stochastic uncertain systems with the pay-off being a nonlinear functional of the uncertain measure where the uncertainty is measured in terms of relative entropy between the uncertain and the nominal measure. The maximizing player is the uncertain measure, while the minimizer is the control which induces a nominal measure. Existence and uniqueness of minimax solutions are derived on suitable spaces of measures. Several examples are presented illustrating the results. Subsequently, the results are also applied to controlled stochastic differential equations on Hilbert spaces. Based on infinite dimensional extension of Girsanov’s measure transformation, martingale solutions are used in establishing existence and uniqueness of minimax strategies. Moreover, some basic properties of the relative entropy of measures on infinite dimensional spaces are presented and then applied to uncertain systems described by a stochastic differential inclusion on Hilbert space. An explicit expression for the worst case measure representing the maximizing player (adversary) is found.

Finite-dimensional nonlinear output feedback dynamic games and bounds for sector nonlinearities

In general, nonlinear output feedback dynamic games are infinite dimensional. The paper treats a class of minimax games when the nonlinearities enter the dynamics of the unobservable states. An information state approach is introduced to recast these games as one of full information in infinite dimensions. Explicit solutions of the first-order partial differential information state equation are derived in terms of a finite-number of sufficient statistics. When the nonlinearities are sector bounded, suboptimal finite-dimensional strategies are derived.

Minimax games, spin glasses, and the polynomial-time hierarchy of complexity classes

We use the negative replica method, which was originally developed for the study of overfrustation in disordered systems, to investigate the statistical behavior of the cost function of minimax games. These games are treated as hierarchical statistical mechanical systems, in which one of the components is at negative temperature.

Best-first fixed-depth minimax algorithms

This article has three main contributions to our understanding of minimax search: First, a new formulation for Stockman's SSS ∗ algorithm, based on Alpha-Beta, is presented. It solves all the perceived drawbacks of SSS ∗, finally transforming it into a practical algorithm. In effect, we show that SSS ∗ = Alpha-Beta + transposition tables. The crucial step is the realization that transposition tables contain so-called solution trees, structures that are used in best-first search algorithms like SSS ∗. Having created a practical version, we present performance measurements with tournament game-playing programs for three different minimax games, yielding results that contradict a number of publications. Second, based on the insights gained in our attempts at understanding SSS ∗, we present a framework that facilitates the construction of several best-first fixed-depth game-tree search algorithms, known and new. The framework is based on depth-first null-window Alpha-Beta search, enhanced with storage to allow for the refining of previous search results. It focuses attention on the essential differences between algorithms. Third, a new instance of this framework is presented. It performs better than algorithms that are currently used in most state-of-the-art game-playing programs. We provide experimental evidence to explain why this new algorithm, MTD( f), performs better than other fixed-depth minimax algorithms.