Stochastic Game Model Research Articles

A Stochastic Differential Game Theoretic Study of Multipath Routing in Heterogeneous Wireless Networks

In heterogeneous wireless networks (HWNs), paths constituting multipath routing are characterized by selfish rationality. Each path's intentions of pursuing individual profits may cause unreasonable competition for limited wireless resources, which leads to the unreliable data transport problem. Therefore, multipath routing optimization is a challenge issue in HWNs. This paper provides a novel approach to study this issue by employing game theory. By taking the utility maximization as the design goal, limited bandwidth resources as the constraint and path reliability as the key metric, a noncooperative stochastic differential game model is constructed for HWNs. With the feedback Nash equilibrium solution, an optimal multipath routing strategy is obtained. Theoretical derivations and simulation results verify the validity of the method present in this paper.

Common Information Based Markov Perfect Equilibria for Stochastic Games With Asymmetric Information: Finite Games

A model of stochastic games where multiple controllers jointly control the evolution of the state of a dynamic system but have access to different information about the state and action processes is considered. The asymmetry of information among the controllers makes it difficult to compute or characterize Nash equilibria. Using the common information among the controllers, the game with asymmetric information is used to construct another game with symmetric information such that the equilibria of the new game can be transformed to equilibria of the original game. Further, under certain conditions, a Markov state is identified for the new symmetric information game and its Markov perfect equilibria are characterized. This characterization provides a backward induction algorithm to find Nash equilibria of the original game with asymmetric information in pure or behavioral strategies. Each step of this algorithm involves finding Bayesian Nash equilibria of a one-stage Bayesian game. The class of Nash equilibria of the original game that can be characterized in this backward manner are named common information based Markov perfect equilibria.

Modeling Attacker-Defender Interaction as a Zero-Sum Stochastic Game

Game-theoretic modeling of computer security views security attack scenarios as an optimization game comprising of multiple players notably the attackers and the defenders (system administrators). This paper first presents theoretically, a two-player zero-sum stochastic game model of the interaction between malicious users and network administrators and secondly introduces a hypothetical network of a typical scenario to show the applicability of our model within that scenario. State games are encoded using a binary scheme in order to properly capture components of the underlying network environment. Our solution involves reducing each state game into a min and max linear programming problems for both the defender and attacker respectively. Game costs, rewards and outcomes are modeled to closely match real world measurements. We propose the use of a combination of the pivotal algorithm and a custom stochastic algorithm to compute the optimal (best-response) strategies for the players at each state. We also describe how the results can be analyzed to show how the optimal strategies can be used by the network administrators to predict adversary's actions, determine vulnerable network assets and suggest optimal defense strategies.

Optimal portfolio choice of gold assets in the differential market and differential game structures

Portfolio choices of gold-related assets for market investors and dealers may not only depend on price differences and the inflation rate, but may also react to the market participants’ strategic behavior and risk attitude. This study develops a two-agent stochastic differential game model to solve the portfolio choice problem of the asset allocations of gold spot, futures, and cash for market participators who are exposed to inflation risks. The equilibrium prices of spot and futures driven by the volatility rate and co-variances that reflect various risk sources are also determined. Specifically, regarding the choice of hedging tools, market participators may prefer gold spot to futures for the purpose of hedging inflation risk. By capturing the stylistic facts of differential market and multiple agent structures, the article can develop a more reasonable and practical model to usefully explain the gold portfolio choices and pricing in the gold markets.

Governance Mechanism for Global Greenhouse Gas Emissions: A Stochastic Differential Game Approach

Today developed and developing countries have to admit the fact that global warming is affecting the earth, but the fundamental problem of how to divide up necessary greenhouse gas reductions between developed and developing countries remains. In this paper, we propose cooperative and noncooperative stochastic differential game models to describe greenhouse gas emissions decision makings of developed and developing countries, calculate their feedback Nash equilibrium and the Pareto optimal solution, characterize parameter spaces that developed and developing countries can cooperate, design cooperative conditions under which participants buy the cooperative payoff, and distribute the cooperative payoff with Nash bargaining solution. Lastly, numerical simulations are employed to illustrate the above results.

A stochastic dynamic game of China's strategic oil stockpiling

China is now building up strategic petroleum reserves (SPRs) to ensure the security of its energy supply. It would be helpful for the decision-makers in China to understand how the non-SPR stockpiles (mainly the commercial oil stockpiles) might behave and how their actions can affect the buildup and drawdown strategies of the government to manage its SPRs. In this paper, a stochastic dynamic game model has been presented to address the potential interactions between the government-owned SPRs and commercial oil stockpiles during the implementation of China’s strategic oil stockpiling. The two players in the game (a public SPR player and a commercial stockpile player) have their own interests and choose individual decisions through the solution of dynamic optimisation problems. Based on the game model, illustrative results for both players’ equilibrium strategies of stockpile buildup and drawdown are presented and they were found to be related to oil market status, disruption probabilities, and regulations on commercial oil stockpiles.

Distributed Frequency Allocation Based on MARL in Dynamic Environment of Cognitive Radio Networks: A Stochastic Game Model

Distributed Frequency Allocation Based on MARL in Dynamic Environment of Cognitive Radio Networks: A Stochastic Game Model

Analysis for Network Attack-Defense Based on Stochastic Game Model

Network ranges can be provided for network attack and defense experiments to simulate real network in a virtual environment,and deduct the process of network threats.This paper presents an overall framework of the network range based on security attack and defense model.In the model,the stochastic models and game model are used,and a series of security attack and defense model algorithm and the security analysis techniques are proposed.Finally,the application of network attack and defense modeling approach on a typical enterprise network attack and defense process of analysis and inference.The results show that the model and analysis approach above proposed is feasible and effective.

Existence of perfect equilibria in a class of multigenerational stochastic games of capital accumulation

In this paper we introduce a model of multigenerational stochastic games of capital accumulation where each generation consists of m different players. The main objective is to prove the existence of a perfect stationary equilibrium in an infinite horizon game. A suitable change in the terminology used in this paper provides (in the case of perfect altruism between generations) a new Nash equilibrium theorem for standard stochastic games with uncountable state space.

A simulation-based approach to study stochastic inventory-planning games

Non-cooperative decision-making problems in a decentralized supply chain can be characterized and studied using a stochastic game model. In an earlier paper, the authors developed a methodology that uses machine learning for finding (near) optimal policies for non-zero sum stochastic games, and applied their methodology on an N-retailer and W-warehouse inventory-planning problem. The focus of this paper is on making the methodology more amenable to practical applications by making it completely simulation-based. It is also demonstrated, through numerical example problems, how this methodology can be used to find (near) equilibrium policies, and evaluate short-term rewards of stochastic games. Short-term rewards of stochastic games could be, in many instances, more critical than equilibrium rewards. To our knowledge, no methodology exists in the open literature that can capture the short-term behaviour of non-zero sum stochastic games as examined in this paper.

The Empirical Bayes Envelope and Regret Minimization in Competitive Markov Decision Processes

This paper proposes an extension of the regret minimizing framework from repeated matrix games to stochastic game models, under appropriate recurrence conditions. A decision maker, P1, who wishes to maximize his long-term average reward is facing a Markovian environment, which may also be affected by arbitrary actions of other agents. The latter are collectively modeled as a second player, P2, whose strategy is arbitrary. Both states and actions are fully observed by both players. While P1 may obviously secure the min-max value of the game, he may wish to improve on that when the opponent is not playing a worst-case strategy. For repeated matrix games, an achievable goal is presented by the Bayes envelope, that traces P1's best-response payoff against the observable frequencies of P2's actions. We propose a generalization to the stochastic game framework, under recurrence conditions that amount to fixed-state reachability. The empirical Bayes envelope (EBE) is defined as P1's best-response payoff against the stationary strategies of P2 that agree with the observed state-action frequencies. Because the EBE may not be attainable in general, we consider its lower convex hull, the convex Bayes envelope (CBE), which is proved to be achievable by P1. The analysis relies on Blackwell's approachability theory. The CBE is lower bounded by the value of the game and for irreducible games turns out to be strictly above the value whenever P2's frequencies deviate from a worst-case strategy. In the special case of single-controller games where P2 alone affects the state transitions, the EBE itself is shown to be attainable.

Existence of Optimal Strategies in Zero-Sum Nonstationary Stochastic Games with Lack of Information on Both Sides

This paper studies a zero-sum discrete-time stochastic game model with Borel state and action spaces. The law of motion of the system in the model is assumed to be nonstationary. Following M. Schal, at each stage of the game every player is assumed to know the sequence of states occurring up to this stage, but has no explicit information about his opponent’s previous decisions. Under certain semicontinuity and compactness conditions, the existence of a value is proved for such a game and the existence of optimal ($\varepsilon $-optimal) universally measurable strategies for the minimizes (maximizes). This essentially improves a result of Schal on this subject.

Dynamic Patent Races with Risky Choices

This paper investigates equilibrium R&D investment strategies of firms endowed with different innovation potentials. To address this issue, this paper permits two stages of innovation and develops a simple stochastic game model involving two firms. It is shown that in equilibrium, a leader in the multiple stage innovation game invests more than a follower; firms compete more vigorously in the later stages of innovation than in the earlier stages; and a follower is more likely to choose a riskier innovation path than one requiring, on average, equivalent effort. It provides an explanation of how the expected benefits, the cost of R&D, and interactions between competing firms combine to determine dynamic R&D strategies over time.

A note on clemhout and Wan's dynamic games of common property resources

Recently, Clemhout and Wan (Ref. 1) provided solutions to a class of stochastic differential game models concerning common property resources. However, they only established closed-loop strategies for the infinite-horizon problem using the Kushner test. In this paper, we provide closed-loop feedback solutions to their game in a finite-time horizon with deterministic evolution dynamics.

A Stochastic Game Model of a Weapons Development Competition

We model a weapons development competition between two countries as a two person zero-sum stochastic game. Conditions are given which ensure that each player’s optimal strategy is monotone in the state, horizon length and discount factor. A non-zero-sum stochastic game model of a weapons development competition is also discussed.

A zero-sum stochastic game model of duopoly

We consider a discrete time zero-sum stochastic game model of duopoly and give a partial characterization of each firm's optimal pricing strategy. An extension to a continuous time model is also discussed.

Stochastic game model of organized strikes

Stochastic game model of organized strikes

Stochastic game models of brand switching

AbstractIn the study of the dynamic nature of brand switching, Markov chain theory has been used extensively. The conflicting interests of the companies involved have, however, not been explicitly incorporated. In this paper, two competing companies, each engaged in marketing a brand product, are considered. Brand switching by a consumer is influenced jointly but probabilistically by the two companies through their respective promotional actions and by the consumer's immediately preceding purchase. A stochastic game model is developed for determining the optimal promotional strategies of the companies, and the unique minimax value of the game in terms of market share. The structure of the model developed is that of a two‐person, zero‐sum, two‐state stochastic game. A method of solution and a few useful variations of the model are présentés. Practical interpretations of the solution are also included.

Contraction Mappings in the Theory Underlying Dynamic Programming

Contraction Mappings in the Theory Underlying Dynamic Programming