Non-cooperative Game Research Articles

Hierarchical Robust Generalized Nash Equilibrium Seeking of High-Order Uncertain Nonlinear Systems.

This article investigates the distributed generalized Nash equilibrium (GNE) seeking problem of noncooperative games (NGs) for high-order strict-feedback nonlinear multiagent systems (MASs). In particular, the feasible action set of each agent is not only subject to local set and inequality constraints but also coupled through an equality constraint with other agents. This constraint structure is more general and covers most of the constraints in the GNE seeking literature. To accomplish the concerned GNE seeking objective, we propose a novel hierarchical GNE seeking approach in this article to decouple the distributed GNE seeking algorithm design into two layers. First, we construct a distributed primary-dual GNE estimator to generate virtual reference signals that converge to the GNE. Then, with the output of the estimator as the reference signal, we develop an adaptive tracking controller to solve the resultant tracking problems under output constraints. To overcome the negative effects of the disturbances, novel compensating terms associated with smooth functions and positive integrable time-varying functions are incorporated in the controller design, which thereby realizes the exact GNE seeking in the presence of nonvanishing mismatched disturbances. At last, an example is given to support the theoretical analysis of the proposed algorithms.

Guaranteed cost solution for discrete-time uncertain/nonlinear dynamic games

Motivated by an example of fiscal and monetary policy interaction of a national economy, the problem of uncertain/nonlinear two players discrete-time noncooperative games is investigated. Since the models of the systems are uncertain, the notion of Nash equilibrium solution is not suitable, instead, new Nash guaranteeing strategies and Nash guaranteed costs are defined. The system’s uncertainties and/or nonlinearities are assumed to be of quadratically bounded type. First, conditions of the Nash guaranteeing strategies are derived for general uncertain nonlinear systems. These results are specified for systems that have linear nominal part and quadratic cost functions. Approximate solutions are obtained by tractable quadratic matrix inequalities. To illustrate the application of the proposed method, two numerical examples are given.

Strategic behavior in TSO-DSO coordinated flexibility markets: A Nash equilibrium and efficiency analysis

This paper investigates the way in which the design of a TSO-DSO coordinated flexibility market can enable strategic behavior by flexibility service providers (FSPs). Multiple flexibility market models are considered for the procurement of flexibility services by transmission and distribution system operators, namely: a common (joint) market, a fragmented market, and a sequential multi-level market. Considering these market models, three non-cooperative games are introduced to investigate the strategic bidding and interaction between FSPs therein. Detailed conclusions are then drawn on the existence and uniqueness of Nash Equilibria (NEs) in the developed games, including derivations of closed-form expressions of the resulting NEs and corresponding price-of-anarchy, capturing the FSPs’ strategic bidding impact on the markets’ efficiency. The analysis considers – first in a duopoly setting, then with multiple players – three different use cases representing when: (1) a sufficient flexible capacity exists (sufficient flexibility offered from the FSPs and adequate interconnection/grid capacity between systems); (2) participants have a scarce flexibility capacity; and (3) a restrictive interface capacity exists between the systems. A case study considering an interconnected transmission–distribution system and multiple FSPs corroborates the analytical findings. The obtained results show that market participants have incentives to set bid prices greater than their marginal costs, thus decreasing the markets’ efficiency. This aspect is shown to be more pronounced when the available flexible capacity is limited, a restrictive line limit is present, or when the market is fragmented, thus supporting the need for additional network investments and the creation of joint flexibility market formats.

Reinforcement learning with predefined and inferred reward machines in stochastic games

This paper focuses on Multi-Agent Reinforcement Learning (MARL) in non-cooperative stochastic games, particularly addressing the challenge of task completion characterized by non-Markovian reward functions. We employ Reward Machines (RMs) to incorporate high-level task knowledge. Firstly, we introduce Q-learning with Reward Machines for Stochastic Games (QRM-SG), where RMs are predefined and available to agents. QRM-SG learns each agent’s best-response policy at Nash equilibrium by defining the Q-function in augmented state space that integrates the stochastic game and RM states. The Lemke-Howson method is utilized to compute the best-response policies for the stage game defined by the current Q-functions at each time step. Subsequently, we explore a more challenging scenario where RMs are unavailable and propose Multi-Agent Reinforcement learning with Concurrent High-level knowledge inference (MARCH). MARCH uses automata learning to learn RMs iteratively and combines this process with QRM-SG for learning the best-response policies. The RL episodes where the obtained rewards are inconsistent with the rewards from the current RMs trigger the inference of new RMs. We prove QRM-SG and MARCH converge to the best-response policies under certain conditions. Two scenarios are conducted to demonstrate the superior performance of QRM-SG and MARCH compared to baseline methods.

Framework design of distributed P2P2G energy transaction considering photovoltaic power and dispatch of manufacturing facilities

In this paper, a distributed peer-to-peer-to-grid (P2P2G) energy transaction framework considering photovoltaic generation and dispatch of manufacturing facilities is designed to increase the penetration of renewable energy resource. The factory with typical manufacturing assembly production task, photovoltaic panels and battery is regarded as a micro-grid (MG) to participate in the energy transaction market. The designed framework contains two levels. In the upper level, each MG determines the scheduling of manufacturing facilities and battery and the quantities of P2P2G energy transactions based on the given P2P energy transaction price and distribution locational marginal price (DLMP), in which the complex typical manufacturing assembly process is formulated as a kind of deferable load and the DLMP is updated according to the optimal power flow (OPF) of the network. In the lower level, peer-to-peer (P2P) energy transaction prices are updated by non-cooperative games and the homologous Rosen-Nash equilibrium points are obtained by using Nikaido-Isoda relaxation and alternating direction method of multipliers (ADMM) algorithm. The efficacy of the designed framework is demonstrated by a case study, and the results indicate the designed framework can improve the utilisation rate of renewable energy.

Task offloading framework to meet resiliency demand in mobile edge computing system

With the development of 5 G mobile users are increasing massively. Some mobile applications like healthcare are latency-critical and requires real-time data processing. A preference-based task offloading framework in mobile edge computing with a device-to-device offloading (MECD2D) system has been proposed to fulfill the latency demands of such applications for minimum energy consumption ensuring resiliency. The problem is formulated as a constraint-based non-linear optimization problem which is complex. The resources are allocated in two steps. In the first step, resources are allocated based on latency demand to ensure resiliency. In the second step, allocated resources are optimized using a non-cooperative mean field game for dynamic system. To ensure the performance of the system for dynamic network, the results are executed on a real-time Shanghai dataset. The computational results indicate that the proposed algorithm performs better in terms of energy consumption. Other parameters such as throughput, network utilization and task computation are also analysed. The results are verified by performing the proposed algorithm with existing Q learning and mean-field game algorithms. The results performed on the dataset indicate an improvement in energy consumption by 5–10 %, and 10–50 % as compared to Q learning and mean-field game respectively.

Incentivizing sustainable practices: Game-theoretic approach to peer-to-peer energy trading in the green transition era

The urgent global concern regarding climate change has highlighted the necessity for transitioning to power generation with zero carbon emissions to promote a sustainable and environmentally conscious society. A crucial element in this transformation is reducing our dependence on the primary grid, which is predominantly powered by fossil fuels, natural gas, and coal. An innovative strategy for achieving this essential transition is through peer-to-peer energy trading (P2PET). However, the effectiveness of P2PET relies on successfully aligning the energy-related objectives of its participants. Identifying and effectively addressing these goals is a significant challenge. In response, this paper introduces a game-theoretic framework designed to encourage subscribers to engage in P2PET, both in islanded microgrids and interconnected grid configurations. Our methodology begins by introducing a model that captures the core energy-related objectives of both energy producers and consumers. This model is supported by a layered architectural framework tailored for peer-to-peer (P2P) marketplaces, enhancing the identification and classification of existing technologies in this domain. Following this, we delve into the formulation of an extended-form game rooted in non-cooperative game theory. We systematically evaluate the presence of strict Nash equilibria within this game-theoretic structure. To promote active engagement and trading in the peer-to-peer energy market (P2PEM), we introduce an innovative energy allocation policy. This policy is strategically devised to ensure the inclusion of every subscriber in the market, irrespective of fluctuations in supply and demand dynamics. Our proposed P2PET scheme is tested on a representative system, specifically a 14-bus IEEE network, incorporating 8 energy producers and 11 consumers as active participants in the market. By conducting an extensive series of tests, we accurately evaluate the design's performance. The results, compared to previous studies, show a significant reduction in consumer energy bills, ranging from 33 % to 7 %. This convincing result underscores the effectiveness and robustness of our proposed energy trading framework. In a world grappling with the imperative to transition to sustainable energy practices, our game-theoretic approach to incentivizing participants in P2PET emerges as a pivotal contribution. It demonstrates tangible benefits, promotes green energy production, and encourages responsible energy consumption.

Optimization of Interaction with Counterparties: Selection Game Algorithm under Uncertainty

The purpose of this study is to develop a comprehensive algorithm for optimizing the interaction of economic entities with counterparties, taking into account the uncertainty of market conditions and the variety of behavioral strategies of participants. The developed algorithm aims to increase the stability and efficiency of the interactions between the economic entity under study and its counterparties, minimizing risks and optimizing cooperative and competitive strategies within the framework of existing market relations. The methodology uses game theory to devise interaction strategies using mutual influence indices, non-cooperative game principles, and payment matrices. The model analyzes various interaction scenarios with counterparties by using payment matrices and considering both competitive and cooperative conditions. The research methodology is supplemented by the calculation of integral estimates based on a set of financial and economic indicators, enabling the assessment of the impact of various interaction strategies on the overall efficiency of an economic entity. After testing the developed models, a set of data was obtained, which can be used to optimize strategic planning and manage the interaction of economic entities with counterparties. The developed algorithm is an effective tool for improving the operational analysis of enterprises, primarily in industrial sectors.

Adaptive Fault-Tolerant Fuzzy Containment Control for Networked Autonomous Surface Vehicles: A Noncooperative Game Approach

Adaptive Fault-Tolerant Fuzzy Containment Control for Networked Autonomous Surface Vehicles: A Noncooperative Game Approach

Quantity Conjectural Variations in Oligopoly Games under Different Demand and Cost Functions and Multilevel Leadership

This paper considers a noncooperative game of quantity competition among firms in an oligopoly market under general demand and cost functions. Each firm’s optimal response to the strategies of other firms is assessed by the magnitude and sign of its conjectural variation, expressing the firm’s expectation regarding the counterparty’s supply quantity change in response to the firm’s unit change in its supply quantity. A game of n firms with the sum of conjectural variations (SCV) regarding all counterparties as the generalized response characteristic is studied. The existence of a bifurcation of the players’ response is revealed; a bifurcation is a strategy profile of the game in which both positive and negative responses are possible with an infinite-magnitude SCV value. Methods are developed for calculating the SCV value under different types of inverse demand functions (linear and power) and cost functions (linear, power, and quadratic), and the impact of these characteristics of firms on the bifurcation state is comparatively analyzed.

Методологические проблемы оценки капиталовложений электросетевых компаний

In the article the author reveals the most significant methodological problems in this most important area, which is the basis for the reproduction of economic systems, on the basis of considering the most widely used methods of capital investment assessment. The article also provides a brief theoretical characterization of capital investments. It is concluded that the reason for methodological problems in the assessment of capital investments is the issue of the nature of capital, which has not been fully resolved in the field of fundamental economic theory. The author draws attention to the fact that the detailed methods of capital investment assessment for specific conditions of management of individual sectoral economic systems, elaborated in scientific articles, dissertations and other works, do not receive scientific generalization and balanced assessment of the community. In fact, it means that there are no methods proven and used by many researchers and practitioners. It is shown that it is inappropriate to completely identify the investment process, project appraisal and capital investments of enterprises. The author points out the gap between financial and technological efficiency existing in the theory and practice of work with investments. Further, using the example of the Russian electric power transmission industry, the author shows how general methods of capital investment assessment may not be applicable depending on specific industry conditions, and how theoretical and methodological problems in practice turn into a lack of tools for decision-making. In conclusion, based on the consideration of disproportions in the mechanism of capital investment assessment, implementation and regulation, another fundamental issue of the theory arises – the need to change approaches to the conflict of interests and regulation of natural monopolies as a non-cooperative game.

Non-cooperative games to control learned inverter dynamics of distributed energy resources

We propose a control scheme via a non-cooperative linear quadratic differential game to coordinate the inverter dynamics of Distributed Energy Resources (DERs) in a microgrid (MG). The MG can provide regulation services in support to the upper-level grid, in addition to serving its own load. The control scheme is designed for the MG to track a power reference, while each DER seeks to minimize its individual cost function subject to learned inverter dynamics and load perturbations. We use a nonlinear high-fidelity model developed by Sandia National Laboratories to learn inverter dynamics. We determine a Nash strategy for the DERs that uses state estimation of a Loop Transfer Recovery. Results show that the control scheme enables savings up to 9.3 to 208 times in the DERs objective cost functions and a time-domain response with no oscillations with up to 3 times faster settling times relative to using droop and PI control.

Context-aware resource allocation for vehicle-to-vehicle communications in cellular-V2X networks

Cellular Vehicle-to-Everything (C-V2X) networks provide critical support for intelligently connected vehicles (ICVs) and intelligent transport systems (ITS). C-V2X utilizes vehicle-to-vehicle (V2V) communication technology to exchange safety–critical information among neighbors. V2V communication has stringent high-reliability and low-latency requirements. The existing solutions on resource allocation for V2V communications mainly rely on channel states to optimize resource utilization but fail to consider vehicle safety requirements, which cannot satisfy safety application performance. In this paper, we focus on application-driven channel resource allocation strategy for V2V communications. First, we propose an inter-packet reception model to represent the delay between two consecutive and successful reception packets at a receiver. We then design an application-specific utility function where the utility depends on the packet reception performance and vehicle safety context. Finally, we formulate the channel resource allocation problem as a non-cooperative game model. The game model can guide each node to cooperate and achieve the trade-off between fairness and efficiency in channel resource allocation. The simulation results show that our work can significantly improve the reliability of V2V communications and guarantee the vehicle safety application performance.

General Model for the Creation of Social Capital Based on Non-cooperative Game Theory

General Model for the Creation of Social Capital Based on Non-cooperative Game Theory

Photonic implementation of the quantum Morra game

The Morra game, an age-old noncooperative game, traditionally played on one's hand, has proved to be a rich setting to study game-theoretic strategies, both classically and within the quantum realm. In this paper, we study a faithful translation of a two-player quantum Morra game, which builds on previous study by including the classical game as a special case. We propose a natural deformation of the game in the quantum regime in which Alice has a winning advantage, breaking the balance of the classical game. A Nash equilibrium can be found in some cases by employing a pure strategy, which is impossible in the classical game where a mixed strategy is always required. We prepared our states using photonic qubits on a linear optics setup, with an average deviation ≤2% with respect to the measured outcome probabilities. Finally, we discuss potential applications of the quantum Morra game to the study of quantum information and communication. Published by the American Physical Society 2024

Optimal design of robust control for belt conveyor systems based on fuzzy dynamic model and Nash game

An optimized robust control method is proposed in this paper to deal with the disturbances of nonlinear factors such as uncertainty in belt conveyor and to realize the speed control of belt conveyor. By controlling the subsystems to track the same desired speed, belt breaks and overlaps caused by varying belt speeds from one section to another are limited. First, a fuzzy model is developed by applying fuzzy set theory to describe uncertainties of belt conveyor such as masses, stiffness, damping and frictions. Second, a robust control is designed for the nominal system and compensating for the uncertainty. With the robust control, the belt conveyor system is proved to be uniformly bounded and uniformly ultimately bounded by the Lyapunov approach. Third, the parameters of controller are optimized based on a non-cooperative game approach. The optimal solution is the Nash equilibrium. Optimal solution allows the controller system to perform better and decrease the cost of control. Finally, by comparing with LQR control and different parameters, the feasibility of robust control and the rationality of the optimal parameters are proven.

Distributed Nash equilibrium seeking in noncooperative game with partial decision information of neighbors.

In the real world, individuals may conceal some of their real decision information to their neighbors due to competition. It is a challenge to explore the distributed Nash equilibrium when individuals play the noncooperative game with partial decision information in complex networks. In this paper, we investigate the distributed Nash equilibrium seeking problem with partial decision information of neighbors. Specifically, we construct a two-layer network model, where players in the first layer engage in game interactions and players in the second layer exchange estimations of real actions with each other. We also consider the case where the actions of some players remain unchanged due to the cost of updating or personal reluctance. By means of the Lyapunov function method and LaSalle's invariance principle, we obtain the sufficient conditions in which the consensus of individual actions and estimations can be achieved and the population actions can converge to the Nash equilibrium point. Furthermore, we investigate the case with switched topologies and derive the sufficient conditions for the convergence of individual actions to Nash equilibrium by the average dwell time method. Finally, we give numerical examples for cases of fixed and switched topologies to verify our theoretical results.

Distributed time-varying Nash equilibrium in resilient multi-objective formation control for cyber–physical systems

This paper investigates the problem of multi-objective time-varying formation control in cyber–physical systems (CPSs) with saturated constraints and malicious agents. By incorporating the concept of multi-agent systems and employing objective weight allocation method, the problem is reformulated as the seeking for the Nash equilibrium solution in a time-varying non-cooperative game. To evaluate the trustworthiness of agent behavior, we propose an adaptive weight learning algorithm based on reinforcement learning. Furthermore, we develop a novel smooth saturation control strategy and a predefined time controller to design a distributed resilient algorithm for CPSs with both known and unknown parameters. Additionally, we apply the designed algorithm to the resilient distributed multi-objective time-varying formation control problem of unmanned aerial vehicles (UAVs) to verify its effectiveness.

Real-time coordinated control strategy for the hybrid electric propulsion system of a flying car with model adaptation and game theory

Coordination among the multiple power components is essential for the hybrid electric propulsion systems since the decision made by one impacts the state and decisions of others due to the coupling mechanical and electrical dynamics. Traditional model-based approaches with weight-sum objectives may result in misleading optimization or even unstable situations because of the inherent poor scalability and synergy. To overcome this issue, this paper proposes a novel game theory-based control strategy with model adaptation mechanism for the hybrid electric flying car to enhance the control performance, system stability, and robustness. Firstly, a control-oriented model of the system is derived with the utilization of the recursive least square parameter estimation method. The non-cooperative game framework is then established with the turboshaft engine subsystem and electric supply subsystem treated as two independent players. The performance of Nash equilibrium solutions with closed-loop and open-loop information structures are investigated where the problem is iteratively solved by exploiting Pontryagin’s Minimum Principle and dynamic programming, respectively. The simulation results demonstrate that the game theory-based controllers can outperform MPC with the weight-sum objectives in terms of control efficiency and robustness improvement and the game theory-based controller with open-loop information structure shows excellent computation efficiency. Moreover, the result of the hardware-in-the-loop experiment demonstrates the real-time performance of the proposed controller.

Research on lateral stability control strategy for distributed drive electric vehicles considering driving style

To improve the handling stability of distributed drive electric vehicle (DDEV), a lateral stability control strategy considering driving style is proposed. Firstly, a driving style recognition model based on support vector machines (SVM) was constructed to identify driving styles in different lateral motion scenarios. Then, a layered architecture is used to design a driver/Active Front Steering (AFS)/Direct Yaw Control (DYC) stability coordination control strategy. The upper layer is based on the phase plane of the sideslip angle and extension control theory, dividing the vehicle operating range into classical, extension, and non domains. In the middle level, three control strategies are designed for different work domains. In the classical domain, a driver/AFS coordinated control strategy based on non-cooperative Nash games is constructed, and the weight coefficients in the AFS system cost function are modified based on driving characteristics; Construct a coordinated control strategy for AFS/DYC systems based on cooperative Pareto games in the extension domain; In non-domain, construct a direct yaw torque control strategy based on twin-delayed deep deterministic policy gradient (TD3) reinforcement learning. The lower layer aims to minimize tire utilization and optimizes the allocation of additional yaw moment. Finally, by establishing Hardware in the Loop (HIL) experiment platform, the effectiveness of the proposed control strategy is verified in lane changing scenarios of different styles of drivers. Compared to DYC stability control based solely on TD3, the maximum yaw rate and standard deviation of the lateral stability control strategy considering driving style reduce by 2.4 % and 10.82 %, respectively, and the maximum and standard deviation of the sideslip angle reduce by 29.29 % and 39.24 %, respectively, thereby improving the vehicle's handling stability.