Non-cooperative Dynamic Game Research Articles

A fully decentralized demand response and prosumer peer-to-peer trading for secure and efficient energy management of community microgrid

A novel demand response program combines the advantages of price-based demand response under the time-of-use pricing and the incentive-driven demand reduction bidding strategy to manage energy demand, and maximize the net present value of both participants and the community-microgrid. It also facilitates the seller prosumers to optimally sell the surplus power in a fully decentralized peer-to-peer power trading manner. A novel mathematical demand reduction bidding model is proposed to minimize the community-microgrid cost by the prioritized selection of participants for demand reduction. This study proposed a novel decentralized three non-cooperative dynamic games strategy, incorporating Nash equilibrium approaches to achieve the optimum schedules of appliance, buying, and power trading agents to maximize the net present value of both prosumers and the community-microgrid while preserving participants’ privacy and control network congestion. The priority-based optimal selection of seller and buyer prosumers according to their utility and pricing function in the peer-to-peer power trading scheme is accomplished using a proposed color-based segmented decentralized bilateral power trading to maximize the net present value. The numerical and simulation results demonstrate the feasibility and effectiveness of the proposed strategies and increase the net present value of prosumers up to 42.4 % and community-microgrid up to 34.79 %.

Open‐loop equilibrium seeking in two‐agent noncooperative linear‐quadratic dynamic game with asymmetric horizon length

AbstractThis article proposes an iterative‐based algorithm for open‐loop equilibrium seeking in a two‐agent noncooperative dynamic game. The finite‐horizon dynamic games handled in almost all existing articles assume the common prediction horizon length. From the viewpoint of divergence and differences in personal values, it is socially rational to solve the equilibrium of the games with asymmetric prediction horizon length. We thus propose an open‐loop equilibrium‐seeking algorithm without the private knowledge of the other agent's horizon information through a receding‐horizon linear‐quadratic game. Our proposed algorithm is based on iterative optimization. Each agent obtains its own best response control strategy while estimating the other agent's state feedback gain from the state information and guaranteeing the stability of the whole system. We also discuss the performance of the proposed equilibrium‐seeking algorithm through numerical examples.

Alternative Method to Resolve the Principal–Principal Conflict—A New Perspective Based on Contract Theory and Negotiation

Liquidated damages mechanisms have been analyzed from a legal perspective and applied to real-world contracts. Due to the lack of application of LDs to common agency theory, this research explores whether the liquidated damages mechanism can resolve principal–principal conflicts under common agency. We combine the moral hazard model of common agency with non-cooperative dynamic game theory to analyze the influence of the liquidated damages mechanism on the agent and principals under the condition of complete information and incomplete information. We find that liquidated damages are the key factors affecting the optimal contract between the principal and agent. Since the agent does not terminate the current contract, a principal–principal conflict arises when another principal wishes to enter into a new contract with the common agent. We find that the agent terminates the existing contract and signs a new one with another principal. The injured party requires liquidated damages from the breaching party. Therefore, they will negotiate the number of liquidated damages. Liquidated damages cause the bargaining game to generate a unique subgame perfect Nash equilibrium and sequential equilibrium. We prove that only when liquidated damages belong to a specific interval does a mechanism generating the Pareto optimal solutions to solve the principal–principal conflicts under common agency exist. A common way to resolve this conflict is ensuring that the minority is subordinate to the majority. For the first time, we study how the liquidated damages mechanism solves multi-principal conflict. This is another perspective that can be used to solve the conflict. Therefore, our paper expands the method of resolving the conflict and extends the theory of common agency; we first show the delegation process. Our research can be applied to various situations and provide a rational decision-making basis for participants.

A Game Approach for Defending System Security from an Attacker

In this paper, a system security problem is studied based on a two-person non-cooperative dynamic game. Security of the system is regarded in the context of observability. Namely a system is secure if it is not observable. Both the defender and attacker are assumed to have means to modify the dimension of unobservable subspace, which is set as the value function. The strategies and algorithms to minimize or maximize the value function are given respectively. Finally, examples are provided to demonstrate the effectiveness of the proposed algorithms and show some interesting game results, including accumulation mode, loop mode and locked mode.

Non-cooperative game algorithms for computation offloading in mobile edge computing environments

Mobile Edge Computing (MEC) has become a promising technology for 5G networks. Computation offloading is an essential issue of MEC, which enables mobile User Equipment (UE) to enjoy rich wireless resources and huge computing power anywhere. This paper considers the Quality-of-Experience (QoE) of UEs in 5G MEC systems and presents a dynamic non-cooperative game (QCOG-DG) algorithm and a static non-cooperative game (QCOG-SG) algorithm for computation offloading of MEC applications. We establish an MEC computation offloading model by considering the QoE requirements of UEs, and discuss the communication overheads, computation cost, and energy consumption models to minimize the energy consumption and time delay of each UE. Considering that there are multiple UEs who want to offload their computation tasks to a resource-constrained MEC server, and each UE is selfish and competitive, we formulate the problem of computation offloading decision as a non-cooperative game model. We prove the existence of a Nash Equilibrium (NE) solution for the proposed game model. In addition, we propose an algorithm that jointly optimizes energy consumption and time delay under QoE preferences to achieve optimal offloading benefits for each UE. Moreover, we respectively propose a dynamic non-cooperative game (QCOG-DG) algorithm and a static non-cooperative game (QCOG-SG) algorithm to efficiently find the NE solution. Extensive simulation experiments are conducted to verify the effectiveness of the proposed MEC computation offloading model and the QCOG-DG and QCOG-SG algorithms. Simulation results show that the proposed QCOG-DG algorithm can efficiently find the NE solutions in the MEC scenarios with UEs of different sizes.

Adaptive Steering Torque Coupling Framework Considering Conflict Resolution for Human-Machine Shared Driving

Human-machine shared control has become an effective cutting-edge approach to enhance driving safety and assist in the transition from manual to autonomous driving. However, driving authority allocation when conflict occurs between a human driver and a machine still represents an intractable problem. In order to solve this problem, this paper proposes an adaptive steering torque coupling framework to achieve the human-machine shared control with conflict resolution. Additionally, a driver trajectory prediction method considering human-machine interaction is established based on the extended state observer and long short-term memory network. In addition, the human-machine shared steering system is modeled based on a non-cooperative dynamic game by the prediction model method for path-tracking, and the obtained solution is given according to the Nash equilibrium. Particularly, a dynamic load allocation approach is designed to resolve human-machine conflict and relax the driver. In order to verify the strategy formed based on the proposed framework, a driver-in-the-loop experiment is conducted. The experimental results reveal that the proposed strategy can effectively reduce the human-machine conflict torque and ensure a driver has absolute control authority. Furthermore, the proposed strategy can also transfer the driving workload between the driver and machine, provide driving experience, or improve driving comfort when there is no conflict.

Optimization of preventive maintenance schedule of subway train components based on a game model from the perspective of failure risk

With the sustainable development of urban transportation systems, the safety and economy of subway trains have become important criteria. However, in the process of formulating the operation and maintenance strategy of subway trains, there is always a conflict of interest between the operator, who aims to improve the operation reliability, and the maintainer, who aims to reduce the maintenance cost. To resolve this conflict, this paper proposes a preventive maintenance decision-making method for subway train components based on a game model. A failure-rate model based on the memory factor was developed and combined with the dynamic attenuation law of a subway train. The failure risk of subway train components was quantitatively evaluated, and a risk penalty cost model was established. Next, a maintenance cost model based on the maintenance reliability and economy was developed. Then, a non-cooperative dynamic game model with operators and maintainers as players was developed, and the balance was solved using the particle swarm optimization algorithm. Finally, the preventive maintenance of the bogie axle box of a subway train was examined for validation. The validation results indicated that the proposed method can balance the interests of the subway train operation and maintenance departments and improve the operation and maintenance decision-making in the field of sustainable cities. • Multiple failure consequences are considered in preventive maintenance cost model. • The influence of multi-parameter combination on maintenance strategy is applied. • Conflicts of interest among decision-making participants are resolved. • The policy recommendations are provided at both the operational and strategic levels.

Research on Multimicrogrid Transaction Model and Cross-Chain Transaction Mechanism Based on Blockchain

The traditional microgrid electricity transactions face a low security, poor economic benefits, and weak stability. To solve these problems, this paper proposes a multimicrogrid cross-chain transaction model based on quantum blockchain. Specifically, a bidding strategy was developed for the noncooperative dynamic game of aggregator-multimicrogrid alliance, aiming to balance and optimize the benefits of all parties and effectively enhance the consumption rate of electric energy. To improve the transaction efficiency between aggregators, a consensus mechanism was designed for multimicrogrid cross-chain communication, realizing consistent self-adaptation to cross-chain information. In addition, the quantum threshold signature technology was adopted to ensure the reliability of transaction data and create an unconditional secure communication environment. Case analysis shows that our model proposed in this paper not only ensures the security and stability of transactions but also enhances the economic benefits, providing theoretical support and decision support for the optimization of multimicronetwork cross-chain transaction model.

Modeling and simulation: social force model modified by Helbing model in fast pass system

This paper introduced Helbing’s social force model, modified it with game theory. Then how individuals in the space behave in dynamic non-cooperative games was described, different macro grouping characteristics were obtained. Individual behaviours at the micro level were simulated. Setting different parameters and conditions of the model, the macro effects of individual behaviours were observed. The overall behaviour of the system was studied. It could be used to guide the allocation of public resources.

Dynamic Computation Offloading in Ultra-Dense Networks Based on Mean Field Games

In ultra-dense networks, the increasing popularity of computation intensive applications imposes challenges to the resource-constrained smart mobile devices (SMDs), which may be solved by offloading these computation tasks to the nearby mobile edge computing centers. However, when massive SMDs offload computation tasks in a dynamic wireless environment simultaneously, the joint optimization of their offloading decisions becomes prohibitively complex. In this paper, we firstly model the joint optimization problem as a multi-user non-cooperative dynamic stochastic game, then propose a mean field game based algorithm to solve it with a drastically reduced complexity. We derive the two partial differential equations ruling the optimal strategies of the mean field game, namely the Hamilton-Jacobi-Bellman and Fokker-Planck-Kolmogorov equations, which are solved in an iterative manner in our proposed algorithm. Numerical results demonstrate that the proposed mean field game-based offloading algorithm requires a lower cumulated cost than the conventional strategies under the latency constraints of computation tasks, with perfect prediction of future channel states. It also appears that the performance of the mean field game-based offloading strategy depends on the accuracy of the future channel knowledge provided to the system, as the uncertainty may compromise its cumulated cost performance.

Edge Computing Offloading Strategy Based on Dynamic Non-cooperative Games in D-IoT

Edge Computing Offloading Strategy Based on Dynamic Non-cooperative Games in D-IoT

Strategy decision game approach of the combination generation system of wind and thermal power participating in the direct power purchase transaction of large consumer

The participation of thermal power in the direct power purchase transaction (DPPT) of large consumer (LC) reduces the ability of transmission to consume wind power. This article proposes a novel strategy decision game approach framework of the wind-thermal combined generation (W&T-CoGen) participating in the DPPT of LC, to promote the consumption of wind power under the background of Renewables Portfolio Standards (RPS). The game model, with the goal of maximizing the profit of W&T-CoGen and LC, is established based on dynamic non-cooperative game theory. A novel profit distribution method, considering the power generation cost, risk cost and the satisfaction of wind and thermal power generation, is specially designed based on Nash negotiation method. The reverse induction method is adopted to solve the Nash equilibrium of the proposed full-information dynamic game model. The feasibility and effectiveness of this framework is verified by specific example, which shows that the participation of W&T-CoGen in the DPPT of LC can increase profits of all participants, promote the consumption of wind power and improve the energy system flexibility. This novel framework provides a novel way to construct power market optimal model in the power system containing large scale wind power.

Stochastic Dynamic Games in Belief Space

Information gathering while interacting with other agents under sensing and motion uncertainty is critical in domains such as driving, service robots, racing, or surveillance. The interests of agents may be at odds with others, resulting in a stochastic noncooperative dynamic game. Agents must predict others’ future actions without communication, incorporate their actions into these predictions, account for uncertainty and noise in information gathering, and consider what information their actions reveal. Our solution uses local iterative dynamic programming in Gaussian belief space to solve a game-theoretic continuous POMDP. Solving a quadratic game in the backward pass of a game-theoretic belief-space variant of iterative linear-quadratic Gaussian control (iLQG) achieves a runtime polynomial in the number of agents and linear in the planning horizon. Our algorithm yields linear feedback policies for our robot, and predicted feedback policies for other agents. We present three applications: Active surveillance, guiding eyes for a blind agent, and autonomous racing. Agents with game-theoretic belief-space planning win 44% more races than without game theory and 34% more than without belief-space planning.

High-Order Sliding Mode Control for Networked Control System with Dynamic Noncooperative Game Scheduling

Specific to the NCSs where sensor signals can be processed centrally, a collaborative design scheme of dynamic game scheduling and advanced control theory was proposed in the present study. Firstly, by using the Jordan standard state space equation of the research object, the three elements of state noncooperative game were built, and the existence and uniqueness of Nash equilibrium solution were verified. In addition, the iterative equation of the scheduling matrix was derived by complying with the designed utility function. Secondly, refer to the number of restricted states the order of sliding mode was determined. And based on it, the corresponding sliding surface was designed. Subsequently, the quadratic optimization theory was adopted to regulate the control value following the implementation of the scheduling strategy to ensure that the control quality was further enhanced in the limited network service. Lastly, a TrueTime simulation example is established to verify the effectiveness of the proposed scheme.

An optimal subsidy scheduling strategy for electric vehicles in multi-energy systems

With the increasing demand for multi-purpose energy, multi-energy systems (MES) have become the trend of urban development. To coordinate disorder charging among electric vehicles(EVs) in MES, this paper presents an optimal subsidy scheduling strategy for EVs. The strategy can decrease the fluctuation of the load of grids. Firstly, we use Monte Carlo approach based on historical data to simulate users’ charging behaviors. Orderly charging model is established according to simulation results. Then, multi-beneficial model is established based on time of use(TOU) tariff. The multi-beneficial model can be optimized by using Particle Swarm Optimization(PSO). Next, Weber-Fechner law is applied to further enhance users’ satisfaction. Moreover, dynamic non-cooperative game simulation is used to adjust both the users and the grids’ behaviors. By using the method of traverse and PSO, data including the optimal subsidy time and quantity can be obtained. In the end, the results of case study show that multi-beneficial benefit increases by 23.19%, the users benefit increases by 1.33% and the grid benefit increases by 21.86%. This can prove that the strategy is beneficial both to the users and the grid.

Graphon Mean Field Games and Their Equations

The emergence of the graphon theory of large networks and their infinite limits has enabled the formulation of a theory of the centralized control of dynamical systems distributed on asymptotically infinite networks (Gao and Caines, IEEE CDC 2017, 2018). Furthermore, the study of the decentralized control of such systems was initiated in (Caines and Huang, IEEE CDC 2018, 2019), where Graphon Mean Field Games (GMFG) and the GMFG equations were formulated for the analysis of non-cooperative dynamic games on unbounded networks. In that work, existence and uniqueness results were introduced for the GMFG equations, together with an epsilon-Nash theory for GMFG systems which relates infinite population equilibria on infinite networks to finite population equilibria on finite networks. Those results are rigorously established in this paper.

Optimal capital structure and the debtholder-manager conflicts of interests: a management decision model

PurposeDetermining the optimal capital structure becomes more complicated by the presence of an agency problem. The issuance of debt as a corporate governance mechanism introduces the asset substitution problem – the agency cost of debt. Thus, there is a recognized need for models that can resolve the agency problem between the debtholder and the manager who acts on behalf of the shareholder, leading to optimal capital structure choice, and enhanced firm value. The purpose of this paper is to model the debtholder-manager agency problem as a dynamic game, resolve the conflicts of interests and determine the optimal capital structure.Design/methodology/approachAs there is no satisfactory model for dealing with the above issues, this paper uses a differential game framework to analyze the incongruity of interests between the debtholder and the manager as a non-cooperative dynamic game and further resolves the conflicts of interests as a cooperative game via a Pareto-efficient outcome.FindingsThe optimal capital structure required to minimize the marginal cost of the agency problem is a higher use of debt, lower cost of equity and withheld capital distributions. The debtholder is also able to enforce cooperation from the manager by providing a lower and stable cost of debt and a greater debt facility in the overtime framework.Originality/valueThe study develops a new dynamic contract theory model based on the integrated issues of capital structure, corporate governance and agency problems and applies the differential game approach to minimize the agency problem between the debtholder and the manager.

Distributed Model Predictive Control Strategy for Islands Multimicrogrids Based on Noncooperative Game

The multimicrogrids (MMGs) system of the island group is geographically dispersed with different ownership. In this article, a control strategy based on distributed model predictive control is proposed to optimize the economic scheduling of MMGs on an island group. The strategy is designed based on the dynamic noncooperative game theory to regulate the trading behavior among microgrids (MGs) belonging to different owners. The mechanism maximizes the economic benefits of the MGs under the premise of ensuring the closed-loop stability of the single MG system. Only a minimum amount of communication information exchange is needed, which avoids the demands of the central controller and can help the MG to protect its privacy of operating information. The proposed strategy can maximize the benefits of power trading and significantly reduce the operating cost of the system while ensuring the balance between supply and demand. Simulation results are presented to prove the fairness and validity of the proposed control strategy.

Event-Driven Off-Policy Reinforcement Learning for Control of Interconnected Systems.

In this article, we introduce a novel approximate optimal decentralized control scheme for uncertain input-affine nonlinear-interconnected systems. In the proposed scheme, we design a controller and an event-triggering mechanism (ETM) at each subsystem to optimize a local performance index and reduce redundant control updates, respectively. To this end, we formulate a noncooperative dynamic game at every subsystem in which we collectively model the interconnection inputs and the event-triggering error as adversarial players that deteriorate the subsystem performance and model the control policy as the performance optimizer, competing against these adversarial players. To obtain a solution to this game, one has to solve the associated Hamilton-Jacobi-Isaac (HJI) equation, which does not have a closed-form solution even when the subsystem dynamics are accurately known. In this context, we introduce an event-driven off-policy integral reinforcement learning (OIRL) approach to learn an approximate solution to this HJI equation using artificial neural networks (NNs). We then use this NN approximated solution to design the control policy and event-triggering threshold at each subsystem. In the learning framework, we guarantee the Zeno-free behavior of the ETMs at each subsystem using the exploration policies. Finally, we derive sufficient conditions to guarantee uniform ultimate bounded regulation of the controlled system states and demonstrate the efficacy of the proposed framework with numerical examples.

Application of non-cooperative dynamic game theory for groundwater conflict resolution

The opposing objectives of the groundwater stakeholders, ranging from maximum withdrawal for economic gain to minimum withdrawal for aquifer and environmental protection, lead stakeholders to behave non-cooperatively. In this study, dynamic non-cooperative games are applied to model these conflicts. These games incorporate stakeholders' non-cooperative decision-making processes and groundwater exploitation details over time. The two key players considered in this study are the government, which aims to minimize the groundwater drawdown, and local farmers, who pursue maximum economic gain. In order to calculate the players' payoffs over the modeling period, a simulation-optimization model is developed. For evaluation, the proposed methodology is implemented in the Bad-Khaledabad study area in Iran. Therefore, two dynamic non-cooperative games are developed, considering weak governmental power and players' interaction. It is concluded that creating a transactional relationship, involving the cropping patterns changes and the irrigation efficiency enhancement, will increase farmers’ profits by 28% and decrease withdrawal from the aquifer by 21%.