Game-theoretic Learning Research Articles

To Save Mobile Crowdsourcing From Cheap-Talk: A Game Theoretic Learning Approach

To Save Mobile Crowdsourcing From Cheap-Talk: A Game Theoretic Learning Approach

Distributed service function chaining in NFV-enabled networks: A game-theoretic learning approach

In network function virtualization (NFV), Service Function Chaining (SFC) provides an ordered sequence of virtual network functions (VNFs) and subsequent steering of traffic flows through them to cater to end-to-end services. This paper addresses the NP-hard problem of minimum cost SFC deployment to support customer services that access the carrier network’s NFV infrastructure (NFVI) through some edge routers. To determine the mappings of VNFs to physical servers, a challenging aspect would be the inter-server latencies that may fluctuate over time because of the sharing nature of cloud data centers. To construct the SFC, we come up with three different formulations, each corresponding to a different informational assumption about the link latencies: First, a centralized integer linear programming (ILP) formulation is given under the assumption of the non-causal availability of exact and instantaneous inter-server latencies. The solution to this ILP can serve as a lower bound to benchmark more scalable and realistic schemes. Next, we give a distributed game-theoretic formulation (with service broker agents as players) which only requires the statistical knowledge of link latency fluctuations. The game provably admits a pure Nash equilibrium (NE) and can be solved iteratively through the well-known best response dynamics (BRD) algorithm. Our main novelty lies in the third formulation in which each service broker has neither instantaneous nor statistical knowledge of the latencies. Instead, it relies on a game-theoretic learning algorithm to compose its VNF chain only based on its own history of adopted decisions and experienced delays on each logical link. We prove that the proposed learning algorithm asymptotically converges to NE and evaluate its performance through simulations in terms of convergence and the impact of network parameters.

Joint Power and Coverage Control of Massive UAVs in Post-Disaster Emergency Networks: An Aggregative Game-Theoretic Learning Approach

Joint Power and Coverage Control of Massive UAVs in Post-Disaster Emergency Networks: An Aggregative Game-Theoretic Learning Approach

A timestamp-based log-linear algorithm for solving locally-informed multi-agent finite games

In this paper, the distributed Nash equilibrium seeking problem for multi-agent finite games is considered. Considering the locally-informed communication structure, previous centralized game-theoretic learning dynamic methods are not feasible. To this end, we propose a distributed log-linear algorithm by introducing a timestamp-based communication mechanism that enables agents to acquire estimates of the true action profile from their neighborhoods. It is shown that the proposed distributed log-linear algorithm significantly outperforms the existing best response dynamics regarding both effectiveness and reliability. For illustration, a locally-informed multi-player Sudoku puzzle is introduced, where each cell is taken as a player, and each player fills numbers only based on the local information of its neighboring players. The simulation results validate the effectiveness of our algorithm in solving the locally-informed multi-player Sudoku puzzle. Our methodology provides a viable and benchmark approach to address the fully distributed locally-informed multi-agent joint decision-making problem.

Learning to Resolve Social Dilemmas: A Survey

Social dilemmas are situations of inter-dependent decision making in which individual rationality can lead to outcomes with poor social qualities. The ubiquity of social dilemmas in social, biological, and computational systems has generated substantial research across these diverse disciplines into the study of mechanisms for avoiding deficient outcomes by promoting and maintaining mutual cooperation. Much of this research is focused on studying how individuals faced with a dilemma can learn to cooperate by adapting their behaviours according to their past experience. In particular, three types of learning approaches have been studied: evolutionary game-theoretic learning, reinforcement learning, and best-response learning. This article is a comprehensive integrated survey of these learning approaches in the context of dilemma games. We formally introduce dilemma games and their inherent challenges. We then outline the three learning approaches and, for each approach, provide a survey of the solutions proposed for dilemma resolution. Finally, we provide a comparative summary and discuss directions in which further research is needed.

Game-theoretic learning for the coordination of drone teams in autonomous cooperative inspection

Without the need for an on-board pilot, drones are designed to accomplish dull, dangerous and dirty missions. However, if a mission exhibits a large operative area and/or several objectives, it may entail poor performance when executed by a single drone. Drone teams may overcome this issue by acting as mobile sensor networks for proximal sensing. In such networks, cooperative autonomy is a key enabling behaviour for achieving resilient and cost-efficient systems. This work implements cooperative autonomous behaviour in the form of a dynamic and decentralized mission planner for a multi-drone inspection mission. The proposed design exploits multi-agent task allocation, distributed route planning and game theory for the assignment of inspection tasks and for the processing of optimal routes in reasonable time frames and with limited communication. In detail, it applies the learning-in-games framework for the coordination within the inspection team, by studying some ad-hoc variants of best response and of log linear learning. Moreover, this work presents some numerical results of model-in-the-loop tests for a comparison between the learning-in-games approaches.

A Finite-Horizon Game-Theoretic Learning Algorithm for Constrained Differential Games With Application to Robust Control of Quadrotor UAV

A Finite-Horizon Game-Theoretic Learning Algorithm for Constrained Differential Games With Application to Robust Control of Quadrotor UAV

A Timestamp-Based Inertial Best-Response Dynamics for Distributed Nash Equilibrium Seeking in Weakly Acyclic Games.

In this article, we consider the problem of distributed game-theoretic learning in games with finite action sets. A timestamp-based inertial best-response dynamics is proposed for Nash equilibrium seeking by players over a communication network. We prove that if all players adhere to the dynamics, then the states of players will almost surely reach consensus and the joint action profile of players will be absorbed into a Nash equilibrium of the game. This convergence result is proven under the condition of weakly acyclic games and strongly connected networks. Furthermore, to encounter more general circumstances, such as games with graphical action sets, state-based games, and switching communication networks, several variants of the proposed dynamics and its convergent results are also developed. To demonstrate the validity and applicability, we apply the proposed timestamp-based learning dynamics to design distributed algorithms for solving some typical finite games, including the coordination games and congestion games.

Misinformation Propagation in Online Social Networks: Game Theoretic and Reinforcement Learning Approaches

Misinformation in online social networks (OSNs) has been an ongoing problem, and it has been studied heavily over recent years. In this article, we use gamification to tackle misinformation propagation in OSNs. First, we construct a game based on the notion of cooperative games on graphs where the nodes of the social network are players. We use random regular networks and real networks in our simulations to show that the constructed game follows evolutionary dynamics and that the outcome of the game depends on the relation between the structural properties of the network and the benefit and cost variables defined in a cooperative game. Second, we create a game on the network level where the players control a set of nodes. We define agents whose goal is to maximize the total reward that we set up to be the number of nodes affected at the end of the game. We propose a deep reinforcement learning (RL) technique based on the multiagent deep deterministic policy gradient (MADDPG) algorithm. We test the proposed method along with well-known node selection algorithms and obtain promising results on different social networks.

PLC-Informed Distributed Game Theoretic Learning of Energy-Optimal Production Policies.

This article describes a novel concept to optimize manufacturing systems distributively through data-based learning. We propose a game-theoretic (GT) learning set-up that is incorporated with accessible control code of the programmable logic controller (PLC) to accelerate the optimal policies learning procedures, instead of learning everything from scratch. Therefore, we offer to process the accessible and available control code into a GT-based learning framework which is subsequently optimized in a fully distributed manner. To this end, we employ the recently developed framework of state-based potential games (PGs) and prove that under mild conditions PLC-informed (PLCi) learning forms a state-based PG framework. We conduct the experiment on a laboratory scale testbed in numerous production scenarios. The experiment's results highlight the major potential of using the PLCi GT-learning, which is the reduction of energy consumption of the production timescales and improvement of production efficiency while nearly halven the learning times.

Probability weighting effect in vertex cover of networks via prospect-theoretic learning

Game-theoretic learning methods for the vertex cover problem have been investigated in this paper. In the traditional game theory, the establishment of the game model is based on the complete objectivity of the players, and the existing game models describe the vertex cover problem mainly along this path. In contrast, this paper considers the impact of players' subjectivity on decision-making results. First, we present a covering game model, where the utility function of the player is established under the prospect theory. Then, by presenting a rounding function, the states of all vertices under Nash equilibrium satisfies vertex cover state of a general network. After then, we present a fictitious play distributed algorithm, which can guarantee that the states of all vertices converge a Nash equilibrium. Finally, the simulation results are presented to assess the impact of players' subjectivity on the overall cover results of networks.

Better Approximation for Distributed Weighted Vertex Cover via Game-Theoretic Learning

Toward better approximation for the minimum-weighted vertex cover (MWVC) problem in multiagent systems, we present a distributed algorithm from the perspective of learning in games. For self-organized coordination and optimization, we see each vertex as a potential game player who makes decisions using local information of its own and the immediate neighbors. The resulting Nash equilibrium is classified into two categories, i.e., the inferior Nash equilibrium (INE) and the dominant Nash equilibrium (DNE). We show that the optimal solution must be a DNE. To achieve better approximation ratios, local rules of perturbation and weighted memory are designed, with the former destroying the stability of an INE and the latter facilitating the refinement of a DNE. By showing the existence of an improvement path from any INE to a DNE, we prove that when the memory length is larger than 1, our algorithm converges in finite time to DNEs, which could not be improved by exchanging the action of a selected node with all its unselected neighbors. Moreover, additional freedom for solution efficiency refinement is provided by increasing the memory length. Finally, intensive comparison experiments demonstrate the superiority of the presented methodology to the state of the art, both in solution efficiency and computation speed.

Distributed Games Over Networks [About This Issue

This issue of <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">IEEE Control Systems</i> includes a special issue on distributed games over networks. Guest Editors Lacra Pavel and Yiguang Hong have assembled four articles for this special issue. The first article, by Tao Li, Guanze Peng, Quanyan Zhu, and Tamer Bas¸ar, provides a systematic treatment of game-theoretic learning methods and their application in network problems, which is key for understanding multiagents.

Anti-jamming channel access in 5G ultra-dense networks: a game-theoretic learning approach

This paper investigates the Quality of Experience (QoE) oriented channel access anti-jamming problem in 5th Generation Mobile Communication (5G) ultra-dense networks. Firstly, considering that the 5G base station adopts beamforming technology, an anti-jamming model under Space Division Multiple Access (SDMA) conditions is proposed. Secondly, the confrontational relationship between users and the jammer is formulated as a Stackelberg game. Besides, to achieve global optimization, we design a local cooperation mechanism for users and formulate the cooperation and competition among users as a local altruistic game. By proving that the local altruistic game is an Exact Potential Game (EPG), we further prove the existence of pure strategy Nash Equilibrium (NE) among users and Stackelberg Equilibrium (SE) between users and jammer. Thirdly, to obtain the equilibrium solutions of the proposed games, we propose an anti-jamming channel selection algorithm and improve its convergence speed through heterogeneous learning parameters. The simulation results validate the convergence and effectiveness of the proposed algorithm. Compared with the throughput optimization scheme, our proposed scheme obtain a greater network satisfaction rate. Finally, we also analyze user fairness changes during the algorithm convergence process and get some interesting conclusions.

Game-Theoretic Learning Anti-Jamming Approaches in Wireless Networks

In this article, the anti-jamming communication problem is investigated from a game-theoretic learning perspective. By exploring and analyzing intelligent anti-jamming communication, we present the characteristics of jammers and the requirements of an intelligent anti-jamming approach. Such an approach is required of self-sensing, self-decision making, self-coordination, self-evaluation, and learning ability. Then a game-theoretic learning anti-jamming (GTLAJ) paradigm is proposed, and its framework and challenges are introduced. Moreover, through three cases (i.e., Stackelberg anti-jamming game, Markov anti-jamming game, and hypergraph-based anti-jamming game), different anti-jamming game models and applications are discussed, and some future directions are presented.

Population Games on Dynamic Community Networks

In this letter, we deal with evolutionary game-theoretic learning processes for population games on networks with dynamically evolving communities. Specifically, we propose a novel mathematical framework in which a deterministic, continuous-time replicator equation on a community network is coupled with a closed dynamic flow process between communities, in turn governed by an environmental feedback mechanism. When such a mechanism is independent of the game-theoretic learning process, a closed-loop system of differential equations is obtained. Through a direct analysis of the system, we study its asymptotic behavior. Specifically, we prove that, if the learning process converges, it converges to a (possibly restricted) Nash equilibrium of the game, even when the dynamic flow process does not converge. Moreover, for a class of population games-two-strategy matrix games-a Lyapunov argument is employed to establish an evolutionary folk theorem that guarantees convergence to a subset of Nash equilibria, that is, the evolutionary stable states of the game. Numerical simulations are provided to illustrate and corroborate our findings.

Game Theoretical Adversarial Deep Learning With Variational Adversaries

A critical challenge in machine learning is the vulnerability of learning models in defending attacks from malicious adversaries. In this research, we propose game theoretical learning between a variational adversary and a Convolutional Neural Network (CNN), participating in a variable-sum two-player sequential Stackelberg game. Our adversary manipulates the input data distribution to make the CNN misclassify the manipulated data. Our ideal adversarial manipulation is a minimum change to the data which yet is large enough to mislead the CNNs. We propose an optimization procedure to find optimal adversarial manipulations by solving for the Nash equilibrium of the Stackelberg game. Specifically, the adversary's payoff function depends on the data manipulation which is determined by a Variational Autoencoder, while the CNN classifier's payoff functions are evaluated by misclassification errors. The optimization of our adversarial manipulations is defined by Alternating Least Squares and Simulated Annealing. Experimental results demonstrate that our game-theoretic manipulations are able to mislead CNNs that are well trained on the original data as well as on data generated by other models. We then let the CNNs to incorporate our manipulated data which leads to secure classifiers that are empirically the most robust in defending various types of adversarial attacks.

A memory-based game-theoretic defensive approach for digital forensic investigators

Digital investigators need to evaluate their existing counter-anti-forensic tools to assess the reliability of their tools against attackers' anti-forensic tools. The evaluation enables them to choose more reliable defensive strategies against the attackers. Game-theoretic algorithms simulate the interactions that happen between an attacker and an investigator (two players) in a forensic environment. The algorithms examine the most desired and stable players' strategies (the Nash equilibrium of the game). However, in the algorithms, it is assumed that the players' action spaces do not modify. Thus, if a player wants to expand his/her action space by introducing a new counter-anti-forensic or an anti-forensic tool, then the algorithms must re-simulate the game from the beginning to examine the Nash equilibrium of the game. In this paper, we introduce a player's capability to expand his/her action space using a memory mechanism. We test the hypothesis of applying the memory mechanism to a fictitious play algorithm and a gradient play algorithm to decrease the required players' iterations to reach the steady-state of the game. We introduce an extended game-theoretic learning algorithm based on the proposed memory mechanism and perform a set of experiments to evaluate the mechanism. Results of the experiments show that the mechanism reduces the required players' iterations to reach the steady-state of the game after expanding their action spaces. The results also show after employing the mechanism the players on average require 115 fewer iterations to reach the steady-state of the game. In comparison with the fictitious play algorithm, the gradient play algorithm needs fewer players' iterations to reach the steady-state of the game and it shows a smoother reduction in required players' iterations. Finally, we formulate the players' efficiency after employing the memory mechanism and present a set of assistive rules for the investigator.

Optimal No-Regret Learning in Strongly Monotone Games with Bandit Feedback

We consider online no-regret learning in unknown games with bandit feedback, where each agent only observes its reward at each time -- determined by all players' current joint action -- rather than its gradient. We focus on the class of smooth and strongly monotone games and study optimal no-regret learning therein. Leveraging self-concordant barrier functions, we first construct an online bandit convex optimization algorithm and show that it achieves the single-agent optimal regret of $\tilde{\Theta}(\sqrt{T})$ under smooth and strongly-concave payoff functions. We then show that if each agent applies this no-regret learning algorithm in strongly monotone games, the joint action converges in \textit{last iterate} to the unique Nash equilibrium at a rate of $\tilde{\Theta}(1/\sqrt{T})$. Prior to our work, the best-know convergence rate in the same class of games is $O(1/T^{1/3})$ (achieved by a different algorithm), thus leaving open the problem of optimal no-regret learning algorithms (since the known lower bound is $\Omega(1/\sqrt{T})$). Our results thus settle this open problem and contribute to the broad landscape of bandit game-theoretical learning by identifying the first doubly optimal bandit learning algorithm, in that it achieves (up to log factors) both optimal regret in the single-agent learning and optimal last-iterate convergence rate in the multi-agent learning. We also present results on several simulation studies -- Cournot competition, Kelly auctions, and distributed regularized logistic regression -- to demonstrate the efficacy of our algorithm.

Joint Computation Offloading, Channel Access and Scheduling Optimization in UAV Swarms: A Game-Theoretic Learning Approach

Coalition-based unmanned aerial vehicle (UAV) swarms havebeen widelyused in urgent missions. To fasten the completion, mobile edge computing (MEC) has been introduced into UAV networks where coalition leaders act as servers to help members with data computing. This paper investigates a relative delay optimization in MEC-assisted UAV swarms. Considering that the scheduling methods have great impact on the delay, some theoretical analyses are made and a scheduling method based on the shortest effective job first (SEJF) is proposed. Based on the coupled relationship between scheduling and resource allocation, the computation offloading and channel access problems are then jointly optimized. To solve the problem in distributed UAV networks, the optimization problem is formulated as an offloading game. It is proved that the game is an exact potential game (EPG) and it has at least one pure strategy Nash Equilibrium (PNE). To reach the PNE, a distributed offloading algorithm based on concurrent best-better response (CBBR) is designed. Finally, the simulations show that the performance of the proposed CBBR algorithm is better than traditional algorithms. Compared with other scheduling methods, the proposed scheduling method based on SEJF reduces the delay by up to 30%.