Non-cooperative Game Research Articles

Deep Reinforcement Learning-Based Charging Pricing for Autonomous Mobility-on-Demand System

The autonomous mobility-on-demand (AMoD) system plays an important role in the urban transportation system. The charging behavior of AMoD fleet becomes a critical link between charging system and transportation system. In this paper, we investigate a strategic charging pricing scheme for charging station operators (CSOs) based on a non-cooperative Stackelberg game framework. The Stackelberg equilibrium investigates the pricing competition among multiple CSOs, and explores the nexus between the CSOs and AMoD operator. In the proposed framework, the responsive behavior of AMoD operator (order-serving, repositioning, and charging) is formulated as a multi-commodity network flow model to solve an energy-aware traffic flow problem. Meanwhile, a soft actor-critic based multi-agent deep reinforcement learning algorithm is developed to solve the proposed equilibrium framework while considering privacy-conservation constraints among CSOs. A numerical case study with city-scale real-world data is used to validate the effectiveness of the proposed framework.

Formation control for unmanned surface vessels: A game‐theoretic approach

AbstractTwo types of formation control problems for a group of unmanned surface vessels (USVs) are investigated by using a game‐theoretic approach. In the two formation frameworks, the USVs have the group formation objective of minimizing the formation errors with their local neighbors. Different from the existing formation control schemes for USVs with only the group objective, the formation control framework in this paper takes into account the individual objectives of the USVs in addition to the collaborative formation requirement for the USVs. Specifically, each USV in the first formation framework has the individual objective of aiming to be located at a given static reference target according to its own interests. And the USVs in the second formation framework have the individual objectives of following a given time‐varying reference target. Considering that the individual objectives of the USVs may conflict with the group formation objective, the formation problem of USVs is transformed to a noncooperative game among multiple USVs. Two controllers based on the Nash equilibrium seeking strategy are designed to solve the formation problem of USVs with conflicting objectives. It is proved that the USVs' states form an equilibrium formation pattern driven by the designed controllers. Simulations are performed to verify the theoretical results.

Nash equilibrium seeking for individual linear dynamics subject to limited communication resources

In this paper, non-cooperative game problems with individual linear dynamics of players are addressed. In the case of limited communication resources, event-triggering mechanisms equip players to determine event instants. A period of dwell time is enforced after each event. In the dwell time, monitoring event-triggering conditions is avoided for each player, which saves resources. After the dwell time, each player triggers the broadcast to its neighbors once a fully distributed event-triggering condition is satisfied. Along this line, the Zeno behavior is excluded. The proposed control algorithm with discrete-time communications converges the systems to an NE. The effectiveness of the proposed control algorithms and event-triggering mechanisms is demonstrated through simulations.

Strategic transfers between cooperative games

We consider a model where the same group of players is involved in more than one cooperative (transferable utility) game. A rule determines the payoffs per game, and for each player a utility function evaluates the resulting vector of payoffs. We assume that each player, independently, can make transfers of worth between different games, thereby affecting its payoff vector and, thus, utility. Two transfer systems are considered, resulting in two distinct noncooperative games, and the focus of the paper is on establishing existence and a characterization of Nash equilibria in these games.

An Economic and Non-cooperative Load-balancing Framework among Federated Cloudlets

Edge computing servers like cloudlets from different service providers compensate scarce computational and storage resources of mobile devices, are distributed across access networks. However, the dynamically varying computational requirements of associated mobile devices make cloudlets either overloaded or under-loaded. Hence, load-balancing among neighboring cloudlets appears to be an essential research problem. Especially, the load-balancing problem among federated cloudlets from the same as well as different service providers for low-latency applications needs significant attention. Thus, in this paper, we propose a load-balancing framework among federated cloudlets for low-latency applications that focuses on latency bound rather than latency minimization. In this framework, we employ dynamic processor slicing for internal handling of job requests from heterogeneous classes by the cloudlets. We propose a continuous-action reinforcement learning automata-based algorithm that enables cloudlets to independently compute the load-balancing strategies against their neighboring cloudlets in a software-defined networking enabled distributed network setting without any exhaustive control message exchange. To capture the economic interaction among federated cloudlets, we model this load-balancing problem as an economic and non-cooperative game and by scaffolding the properties of the game formulation, we achieve faster convergence of the reinforcement learning automata. Furthermore, through extensive simulations, we study the impacts of exploration and exploitation on learning accuracy.

Improved Particle Swarm Algorithm Using Rubik’s Cube Topology for Bilevel Building Energy Transaction

Abstract Following the rapid growth of distributed energy resources (e.g., renewables, battery), localized peer-to-peer energy transactions are receiving more attention for multiple benefits, such as reducing power loss and stabilizing the main power grid. To promote distributed renewables locally, the local trading price is usually set to be within the external energy purchasing and selling price range. Consequently, building prosumers are motivated to trade energy through a local transaction center. This local energy transaction is modeled in bilevel optimization game. A selfish upper level agent is assumed with the privilege to set the internal energy transaction price with an objective of maximizing its arbitrage profit. Meanwhile, the building prosumers at the lower level will response to this transaction price and make decisions on electricity transaction amount. Therefore, this non-cooperative leader-follower trading game is seeking for equilibrium solutions on the energy transaction amount and prices. In addition, a uniform local transaction price structure (purchase price equals selling price) is considered here. Aiming at reducing the computational burden from classical Karush–Kuhn–Tucker (KKT) transformation and protecting the private information of each stakeholder (e.g., building), swarm intelligence-based solution approach is employed for upper level agent to generate trading price and coordinate the transactive operations. On one hand, to decrease the chance of premature convergence in global-best topology, Rubik’s Cube topology is proposed in this study based on further improvement of a two-dimensional square lattice model (i.e., one local-best topology-Von Neumann topology). Rotating operation of the cube is introduced to dynamically changing the neighborhood and enhancing information flow at the later searching state. Several groups of experiments are designed to evaluate the performance of proposed Rubik’s Cube topology-based particle swarm algorithm. The results have validated the effectiveness of proposed topology and operators comparing with global-best version PSO and Von Neumann topology-based PSO and its scalability on larger scale applications.

Social Power and Non-cooperative Game Theory

This paper defends the use of non-cooperative game theory for analysing questions of governance. To do so it posits a way of extending the resource account of social power from cooperative games to noncooperative games in a way that side steps a range of criticism. This involves identifying tipping points in the reputations of certain agents for paying and punishing those in their thrall. These tipping points are what give threats and offers their credibility in the absence of enforcement mechanisms and stabilise the distribution of social resources in society.

Demand-side management and its impact on utility and consumers through a game theoretic approach

The modern smart grid facilitate improved demand flexibility, but the self-optimizing nature of participating users will impact the effectiveness and stability of the demand-side management (DSM) program. In this article, the DSM is modeled as a non-cooperative game between users. To ensure the effectiveness and stability of the solution, the existence of Nash equilibrium (NE), its quality and multiplicity are thoroughly investigated for the system model consisting home appliances, distributed energy storage (DES) and dispatchable generation (DPG). The payoff function of the formulated game is not strictly convex due to addition of DES and DPG. Therefore, the proximal decomposition algorithm (PDA) is utilized to get the NE of the game. The convergence of PDA has a strong dependency on system parameters. A rigorous mathematical proof is provided for the condition of convergence of the PDA. The formulated DSM model has been utilized for analyzing its impact on the users as well as distribution utility. In the case of former, its impact on the user’s energy bill, profile of DES and DPG, comfort and privacy concern are investigated. In latter case, its impact on system performance parameters such as system peak-to-average ratio (PAR), bus voltage, total system loss, bus voltage deviation index, voltage stability indices, and system reliability parameters are studied. The IEEE-33 bus distribution system is considered for the assessment of the proposed study. The obtained result shows a significant reduction in the energy bill and PAR and improved system performance parameters.

Discrete-time strategic job arrivals to a single machine with waiting and lateness penalties

A single machine processes a random number of identical jobs on a first-come first-served (FCFS) basis. Processing times are independent and identically distributed with a general integral distribution. Each job is held by a strategic player who needs to choose a time slot to arrive at. The individual objective is to reduce the sum of two types of linear in time costs: lateness and waiting. The resulting decision making model is, then, a symmetric non-cooperative game. We derive a symmetric Nash equilibrium for this game, and discuss its structural properties.

A variable neighborhood search based algorithm and game theory models for green supply chain design

With the increase in the volume of greenhouse gases and pollutants, supply chain managers have sought to design and set up networks that pay special attention to environmental factors besides the economic aspects. In this research, a three-level supply chain network is considered including one manufacturer, several retailers and several customers. Greenhouse gas emissions as an environmental issue, the dependence of demand on the selling price of products and cooperative advertising have been examined to design this network. Also, several transportation systems and production methods with different environmental effects and different costs have been considered. The problem has been modeled by both the general and advertising cost classification approaches. Each model has been linearized by McCormick and sequential linear programming methods, and in each approach, Stackelberg, Nash and cooperative games have been used to determine the relationship between members of the supply chain. Finally, an algorithm has been proposed from a combination of variable neighborhood search algorithm and mathematical modeling and after adjusting its parameters by Taguchi statistical method, Stackelberg, Nash and cooperative games have been implemented on it. The results of linearization methods and the meta-heuristic algorithm show that the profit of the manufacturers, retailers and the whole supply chain depends on the type of the game selected. The profit of the whole supply chain is greater in cooperative conditions than in non-cooperative conditions, and in non-cooperative games, the final profit of the manufacturer will be greater in Stackelberg game.

Robust perfect equilibrium in large games

This paper proposes a new equilibrium concept “robust perfect equilibrium” for non-cooperative games with a continuum of players, incorporating three types of perturbations. Such an equilibrium is shown to exist (in symmetric mixed strategies and also in pure strategies) and satisfy the important properties of admissibility, aggregate robustness, and ex post robust perfection. These properties strengthen relevant equilibrium results in an extensive literature on strategic interactions among a large number of agents. Illustrative applications to congestion games are presented. In the particular case of a congestion game with strictly increasing cost functions, we show that there is a unique symmetric robust perfect equilibrium.

Towards Design of an Efficient Sensing Data Acquisition Scheme for UAVs-Assisted Wireless Sensor Networks

This paper investigates sensing data acquisition issues from large-scale hazardous environments using UAVs-assisted WSNs. Most of the existing schemes suffer from low scalability, high latency, low throughput, and low service time of the deployed network. To overcome these issues, we considered a clustered WSN architecture in which multiple UAVs are dispatched with assigned path knowledge for sensing data acquisition from each cluster heads (CHs) of the network. This paper first presents a non-cooperative Game Theory (GT)-based CHs selection algorithm and load balanced cluster formation scheme. Next, to provide timely delivery of sensing information using UAVs, hybrid meta-heuristic based optimal path planning algorithm is proposed by combing the best features of Dolphin Echolocation and Crow Search meta-heuristic techniques. In this research work, a novel objective function is formulated for both load-balanced CHs selection and for optimal the path planning problem. Results analyses demonstrate that the proposed scheme significantly performs better than the state-of-art schemes.

Secure Artificial Intelligence of Things for Implicit Group Recommendations

The emergence of Artificial Intelligence of Things (AIoT) has provided novel insights for many social computing applications, such as group recommender systems. As the distances between people have been greatly shortened, there has been more general demand for the provision of personalized services aimed at groups instead of individuals. The existing methods for capturing group-level preference features from individuals have mostly been established via aggregation and face two challenges: 1) secure data management workflows are absent and 2) implicit preference feedback is ignored. To tackle these current difficulties, this article proposes secure AIoT for implicit group recommendations (SAIoT-GRs). For the hardware module, a secure Internet of Things structure is developed as the bottom support platform. For the software module, a collaborative Bayesian network model and noncooperative game are introduced as algorithms. This secure AIoT architecture is able to maximize the advantages of the two modules. In addition, a large number of experiments are carried out to evaluate the performance of SAIoT-GR in terms of efficiency and robustness.

Coordination of the Decentralized Concurrent Open-Shop

In concurrent open-shop, several jobs have to be completed, where each job consists of multiple components that are processed simultaneously by different dedicated machines. We assume that the components are sequenced on each machine in a decentralized manner, and analyze the resulting coordination problem under the objective of minimizing the weighted sum of disutility of completion times. The decentralized system is modeled as a non-cooperative game for two environments: (1) local completion times, where each machine considers only the completion times of their components, disregarding the other machines; and (2) global completion times, where each machine considers the job completion times from the perspective of the system, i.e. when all components of each job are completed. Tight bounds are provided on the inefficiency that might occur in the decentralized system, showing potentially severe efficiency loss in both environments. We propose and investigate scheduling based, coordinating job weighting mechanisms that use concise information, showing impossibility in the local completion times environment and possibility using the related weights mechanism in the global completion times environment. These results extend to a setting with incomplete information in which only the distribution of the processing times is commonly known, and each machine is additionally informed about their own processing times.

Game-Based Channel Selection for UAV Services in Mobile Edge Computing

Computation offloading is a hot research topic in mobile edge computing (MEC). Computation offloading among multiedge nodes in heterogeneous networks can help reduce offloading cost. In addition, the unmanned aerial vehicles (UAVs) play a key role in MEC, where UAVs in the air communicate with ground base stations to improve the network performance. However, limited channel resources can lead to the increase of transmission delay and the decline of communication quality. Effective channel selection mechanisms can help address those issues by improving transmission rate and ensuring communication quality. In this paper, we study channel selection during communication between multiple UAVs and base stations in an MEC system with heterogeneous networks. To maximize the transmission rate of each UAV user, we formulate a channel selection problem and model it as a noncooperative game. Then, we prove the existence of Nash equilibrium (NE). In addition, we design a multiple UAV-enabled transmission channel selection (UTCS) algorithm to obtain the equilibrium strategy profile of all the UAV users. Experimental results validate that UTCS algorithm can converge after a finite number of iterations and it outperforms random transmission algorithm (RTA) and sequential transmission algorithm (STA).

Exploiting environmental signals to enable policy correlation in large-scale decentralized systems

Can artificial agents benefit from human conventions? Human societies manage to successfully self-organize and resolve the tragedy of the commons in common-pool resources, in spite of the bleak prediction of non-cooperative game theory. On top of that, real-world problems are inherently large-scale and of low observability. One key concept that facilitates human coordination in such settings is the use of conventions. Inspired by human behavior, we investigate the learning dynamics and emergence of temporal conventions, focusing on common-pool resources. Extra emphasis was given in designing a realistic evaluation setting: (a) environment dynamics are modeled on real-world fisheries, (b) we assume decentralized learning, where agents can observe only their own history, and (c) we run large-scale simulations (up to 64 agents). Uncoupled policies and low observability make cooperation hard to achieve; as the number of agents grow, the probability of taking a correct gradient direction decreases exponentially. By introducing an arbitrary common signal (e.g., date, time, or any periodic set of numbers) as a means to couple the learning process, we show that temporal conventions can emerge and agents reach sustainable harvesting strategies. The introduction of the signal consistently improves the social welfare (by 258% on average, up to 3306%), the range of environmental parameters where sustainability can be achieved (by 46% on average, up to 300%), and the convergence speed in low abundance settings (by 13% on average, up to 53%).

On the Design of Magnetic Resonant Coupling for Wireless Power Transfer in Multicoil Networks

Wireless power transfer (WPT) is a promising technology for powering up distributed devices in machine type networks. Over the last decade magnetic resonant coupling (MRC) received significant interest from the research community, since it is suitable for realizing mid-range WPT. In this letter, we investigate the performance of a single cell MRC-WPT network with multiple receivers, each equipped with an electromagnetic coil and a load. We first consider pre-adjusted loads for the receivers and by taking into account spatial randomness, we derive the harvesting outage probability of a receiver; for both the strong and loosely coupling regions. Then, we develop a non-cooperative game for a fixed receiver topology, in order to acquire the optimal load which maximizes each receiver’s harvested power. Throughout our work, we obtain insights for key design parameters and present numerical results which validate our analysis.

Energy Efficient Power Allocation for Cell-Free mmWave Massive MIMO With Hybrid Precoder

This letter investigates the downlink of a cell-free millimeter wave (mmWave) massive multiple-input multiple-output (mMIMO) system, where many access points (APs) cooperatively serve a user. Although the intensive deployment of APs can dramatically improve the system capacity, it also increases the network energy consumption substantially. To track the non-concave global energy-efficiency (GEE) optimization problem, we decompose it into hybrid precoder design and power allocation design. A novel dynamic subarray with quantized phase shifters (DS-QPS) hybrid precoder is introduced, where each radio frequency (RF) chain only connects to a disjointed subset of antennas. The optimization problem of the number of RF chains is formulated as an eigenvalue maximization problem considering a realistic power consumption model. For power allocation, a new centralized framework is exploited to solve a sequence of simpler power allocation subproblems while still aiming at the GEE maximization by merging with fractional programming, non-cooperative game theory, and gradient-assisted binary search (GABS) algorithm. Simulations show that the joint design is more energy-efficient than the baselines.

Contracting Mechanisms for Stable Sourcing Networks

Problem definition: We study profit allocation for a sourcing network, in which a buyer sources from a set of differentiated suppliers with limited capacity under uncertain demand for the final product. Whereas the buyer takes the lead in forming the sourcing network and designing the contract mechanism, due to their substantial bargaining power, the suppliers take the lead in determining the terms of the contract. Academic/practical relevance: We identify contracting mechanisms that will ensure the stability of the sourcing network in the long term, where a stable sourcing network requires an effective profit-allocation scheme that motivates all members to join and stay in the network. Methodology: We apply methods from game theory to model the network and analyze the Nash equilibrium of a noncooperative game under a proposed contracting mechanism. We then use a cooperative game model to study the stability of the resulting equilibrium. Results: We show that the optimal network profit, as a set function of the set of suppliers, is submodular, which allows us to demonstrate that the core of the cooperative game is not empty. We also establish a set of conditions that are equivalent to, but much simpler than, the original conditions for the core. We use these results to demonstrate that the proposed fixed-fee contracting mechanism can implement a stable network in the competitive setting by achieving a profit allocation that is in the core of the cooperative game. We also demonstrate that the grand coalition is stable in a farsighted sense under the Shapley value allocation. Managerial implications: Under the fixed-fee mechanism, the buyer’s decisions maximize the network profit, and each supplier earns a profit equal to its marginal contribution. When the aggregate capacity of the supplier network is high relative to demand, or demand is more likely to be small, the fixed-fee mechanism is likely to outperform the Shapley value allocation from the perspective of the buyer.

Game Theory in Defence Applications: A Review.

This paper presents a succinct review of attempts in the literature to use game theory to model decision-making scenarios relevant to defence applications. Game theory has been proven as a very effective tool in modelling the decision-making processes of intelligent agents, entities, and players. It has been used to model scenarios from diverse fields such as economics, evolutionary biology, and computer science. In defence applications, there is often a need to model and predict the actions of hostile actors, and players who try to evade or out-smart each other. Modelling how the actions of competitive players shape the decision making of each other is the forte of game theory. In past decades, there have been several studies that applied different branches of game theory to model a range of defence-related scenarios. This paper provides a structured review of such attempts, and classifies existing literature in terms of the kind of warfare modelled, the types of games used, and the players involved. After careful selection, a total of 29 directly relevant papers are discussed and classified. In terms of the warfares modelled, we recognise that most papers that apply game theory in defence settings are concerned with Command and Control Warfare, and can be further classified into papers dealing with (i) Resource Allocation Warfare (ii) Information Warfare (iii) Weapons Control Warfare, and (iv) Adversary Monitoring Warfare. We also observe that most of the reviewed papers are concerned with sensing, tracking, and large sensor networks, and the studied problems have parallels in sensor network analysis in the civilian domain. In terms of the games used, we classify the reviewed papers into papers that use non-cooperative or cooperative games, simultaneous or sequential games, discrete or continuous games, and non-zero-sum or zero-sum games. Similarly, papers are also classified into two-player, three-player or multi-player game based papers. We also explore the nature of players and the construction of payoff functions in each scenario. Finally, we also identify gaps in literature where game theory could be fruitfully applied in scenarios hitherto unexplored using game theory. The presented analysis provides a concise summary of the state-of-the-art with regards to the use of game theory in defence applications and highlights the benefits and limitations of game theory in the considered scenarios.