Reach Nash Equilibrium Research Articles

A fast optimization approach for seeking Nash equilibrium based on Nikaido–Isoda function, state transition algorithm and Gauss–Seidel technique

This paper proposes a fast optimization approach for non-cooperative games with complicated payoff functions (non-smooth, non-concave, etc.). The Nikaido–Isoda function is employed to convert knotty Nash equilibrium problems (NEPs) into large-scale optimization problems with complex objective functions. To efficiently seek Nash equilibrium, the resulting optimization problems are decomposed into many subproblems where each player tries to maximize its payoff when observing others’ current strategies. All players’ strategies are updated iteratively until reaching Nash equilibrium. Specifically, a dynamic state transition algorithm (STA) is proposed to seek global optima of subproblems at each iteration, and the sequential quadratic programming (SQP) is embedded into dynamic STA for convergence acceleration. A Gauss–Seidel technique is utilized for players’ strategy updates to improve computational efficiency further. Numerical examples drawn from multidisciplinary contexts validate that the proposed approach could effectively seek out Nash equilibrium for simultaneously decreasing the time-consuming remarkably.

A two-level game theoretic approach for task offloading in mobile edge computing

In the mobile edge computing system subject to wireless interference, the Edge Server Provider (ESP) aims to offer profitable computing resources to Device Managers (DMs), who make optimal strategies based on the provided prices. However, the existence of mixed variables typically constitutes an NP-hard problem, posing a significant challenge for optimization. In response to address this issue, we formulate a bi-level optimization problem, where the upper level is devoted to optimizing the pricing of computing resources. The lower level optimizes DM migration strategies and resource allocation at specified prices. Leveraging game theory, we achieve distributed and efficient computation offloading by formulating the distributed computing offloading strategy among lower-level DMs as a task offloading game. Our analysis identifies Nash equilibrium and finite improvement characteristics within the game. Based on these insights, we present a bi-level distributed computation offloading algorithm capable of reaching Nash equilibrium, thus optimizing the profit for both DMs and ESP. The experiments have demonstrated the algorithm’s effectiveness in reducing system costs and maximizing the profits of ESP with DMs across diverse scenarios.

A multilevel system cross-game dynamics approach for carbon emission governance strategy

The relationships within the carbon emissions game are diverse and intricate, encompassing both horizontal games between regions and vertical games involving governments, enterprises, and the public. To explore the complex game relationships among carbon emission entities, this study integrated horizontal and vertical games and applied the system dynamics method to construct a multilevel and multiagent cross-game model, analyzing the changing trends of strategies among different regional governments, enterprises, and the public. The research reveals that there is free-riding behavior in the governance of carbon emissions among regions. However, this free-riding behavior can lead to a virtuous cycle. The attitude of the government plays a crucial role in determining whether businesses engage in carbon emission management. Increasing government regulatory efficiency does not alter the outcome of the game, but it can reduce the time required to reach Nash equilibrium. The study enhances the simulation of carbon emission game theory and provides a reliable reference for government regulation and low-carbon development in businesses.

Ecological and Real-Time Route Selection Method for Multiple Vehicles in Urban Road Network

Traffic congestion has been a hot topic of research in the field of intelligent transportation, which can be alleviated by efficient route navigation. Most of the existing route planning methods are non-negotiated algorithms, which do not take into account the route conflicts and collaborative relationships between multiple vehicles. Also, most negotiated algorithms have not been comprehensively considered dynamic route collaboration between vehicles, large-scale efficient computation, environmental pollution, etc. Therefore, an ecological multivehicle real-time route selection model (EMR2SM) for urban road networks is firstly proposed in this paper, which combines real-time traffic conditions of the road network with travel time, distance, and exhaust emissions as optimization indicators. In order to solve the large-scale computation problem of traditional negotiated algorithms, an adaptive multiswarm bee colony (AMSBC) algorithm is designed, which efficiently solves the multivehicle dynamic route selection problem. AMSBC searches the optimal route for each vehicle in parallel through multiple population division and self-adaption mechanism, to make multivehicle route selection reach Nash equilibrium. Compared with three non-negotiated optimization algorithms based on swarm technology, EMR2SM is verified by experiments that it improves the efficiency and accuracy of the optimal route selection for multiple vehicles and reduces vehicle emissions, which can effectively reduce traffic congestion and environmental pollution.

NCG-TSM: A Noncooperative Game for the Taxi Sharing Model in Urban Road Networks

Taxi sharing is a promising method to save resource consumption and alleviate traffic congestion while satisfying people’s commuting needs. Existing research methods include taxi dispatching methods based on intelligent algorithms, single vehicle route recommendation algorithms, and route recommendation algorithms based on vehicle traffic history. However, these studies either focus on how to efficiently dispatch satisfactory vehicles for passengers, ignoring the effect of efficient routes on vehicle travel efficiency, or let vehicles follow the shortest detour distance recommended by the system, ignoring the traffic congestion caused by the influx of vehicles into the same road. To address the abovementioned problems, a noncooperative game for a taxi sharing model (NCG-TSM) in urban road networks is proposed in this paper combined with the traffic conditions of the optional routes, and a distribution estimation algorithm for the shared taxi game is designed to make multivehicle route selections reach Nash equilibrium. The effectiveness of NCG-TSM is verified through simulation experiments. When the number of vehicles reaches the congestion capacity of the road segment, compared to the three common frameworks, the travel time cost and fuel consumption cost can be reduced by 5.8% to 9.1% and 3.5% to 8.9%, respectively. Besides, the occupancy rate has been improved, especially compared to the BMP framework, by 5.5% to 40%.

Neuro-adaptive control for searching generalized Nash equilibrium of multi-agent games: A two-stage design approach

This paper investigates the generalized Nash equilibrium searching problem of multi-agent games over weight-unbalanced directed networks. In this problem, the feasible action set of each agent is not only constrained by private convex sets and shared coupling equality, but also constrained by its own uncertain dynamics. Moreover, the local cost function is related to both his own actions and others, and each agent is only allowed access to neighbor information. To address this problem, we propose a neuro-adaptive control strategy based on two-stage design. In the first stage of design process, an approximator with adaptive-gain is designed to generate a virtual trajectory that converges to a necessary Nash equilibrium, the compensator based on consensus-tracking is used to obtain non-neighbor information and several auxiliary state variables are used to exchange information with local neighborhoods on a digraphs to deal with equality constraints. In the second stage of the strategy, we further developed a neuro-adaptive tracking controller based on one-step design for tracking the virtual trajectory. In the controller design, the virtual control variables and the actual control laws are obtained in a collective way, without repeating the design process. We introduce a variable called the minimum learning parameter to reduce the number of online learning parameters of the neural network. By using tools from variational inequality theory and Lyapunov stability theory, we prove that the proposed control strategy can drive all agents reach Nash equilibrium. Finally, the effectiveness of control strategy is verified by simulation example.

ABNORMAL DETECTION OF WIND TURBINE CONVERTER BASED ON CWGANGP-CSSVM

Abnormal detection of wind turbine converter (WT) is one of the key technologies to ensure long-term stable operation and safe power generation of WT. The number of normal samples in the SCADA data of WT converter operation is much larger than the number of abnormal samples. In order to solve the problem of low abnormal data and low recognition rate of WTs, we propose a sample enhancement method for WT abnormality detection based on an improved conditional Wasserstein generative adversarial network. Since the anomaly samples of WT converters are few and difficult to obtain, the CWGANGP oversampling method is constructed to increase the anomaly samples in the WT converter dataset. The method adds additional category labels to the inputs of the generative and discriminative models of the generative adversarial network, constrains the generative model to generate few types of anomalous samples, and enhances the generative model’s ability to generate few types of anomalous samples, enabling data generation in a prescribed direction. The smooth continuous Wasserstein distance is used instead of JS divergence as a distance metric to measure the probability distribution of real and generated data in the conditional generative response network and reduce pattern collapse. The gradient constraint is added to the CWGANGP model to enhance the convergence of the WGAN model, so that the generative model can synthesize minority class anomalous samples more effectively and accurately under the condition of unbalanced sample data categories. The quality of anomalous sample generation is also improved. Finally, the anomaly detection is made on the actual operating variator dataset for the unbalanced dataset and the dataset after reaching Nash equilibrium. The experimental results show that the method used in this paper has lower MAR and FAR in WT converter anomaly detection compared with other oversampling data balance optimization methods such as SMOTE, RandomOverSampler, GAN, etc. The method can be well implemented for anomaly detection of large wind turbines and can be better applied in WT intelligent systems.

Parallel adversarial feature learning and enhancement of feature discriminability for fault diagnosis of a planetary gearbox under time-varying speed conditions

The intelligent fault diagnosis of a planetary gearbox under variable speed is still a challenging topic. Due to the similar spectrum structure, overlapping features occur and result in decreasing diagnosis accuracy. Autoencoder-based methods can extract features adaptively but few studies have proposed approaches to enhance the discriminability of features from different classes under variable speeds. Besides, the adverse variability of encoder weights may result in an adverse effect on the decoder. Adversarially learned inference (ALI) trains the encoder and decoder independently, but it is time-consuming to reach Nash equilibrium. To address the issues, a parallel adversarial learning inference (PALI) model is proposed, which aims at validating the parallel training of encoder and decoder and enhancing the discriminability of features. Specifically, time-frequency analysis is utilized to reveal the time-varying characteristics of raw signals and obtain time-frequency images as input for the encoder. Then, an explicit multi-dimensional uniform distribution is used for the merit of a simple probability density function to construct visualized and well-classified samples as input for the decoder. After that, a parallel adversarial game is explored to train the encoder and decoder simultaneously and independently, which will reduce computing interference and make the extracted features similar to the well-classified samples and reconstruct the raw signals. Finally, a Softmax classifier is trained and tested by the features. This method and its generability are validated via a planetary gearbox data set and a public bearing data under variable speed. The results indicate that the proposed parallel adversarial game is valid for training encoder and decoder independently, and that PALI works as well as the adversarial autoencoder (AAE) and outperforms ALI, the variational autoencoder (VAE) in obtaining well-clustered features over different training data. Compared to Wigner–Ville distribution (WVD) and continuous wavelet transform (CWT), PALI based on short-time Fourier transformation (STFT) works better over different training data.

An Architecture for Distributed Energies Trading in Byzantine-Based Blockchains

With the development of smart cities, not only are all corners of the city connected to each other, but also connected from city to city. They form a large distributed network together, which can facilitate the integration of Distributed Energy Stations (DESs) and corresponding smart aggregators. Nevertheless, because of potential security and privacy protection arising from trustless energies trading, how to make such energies trading go smoothly is a tricky challenge. In this paper, we propose a blockchain-based multiple energies trading (B-MET) system for secure and efficient energies trading by executing a smart contract we design. Because energies trading requires the blockchain in B-MET system to have high throughput and low latency, we design a new byzantine-based consensus mechanism (BCM) based on node’s credit to improve efficiency for the consortium blockchain under the B-MET system. Then, we take combined heat and power (CHP) system as a typical example that provides distributed energies. We quantify their utilities and model the interactions between aggregators and DESs in a smart city by a novel multi-leader multi-follower Stackelberg game. It is analyzed and solved by reaching Nash equilibrium between aggregators, which reflects the competition between aggregators to purchase energies from DESs. In the end, we conduct plenty of numerical simulations to evaluate and verify our proposed model and algorithms, which demonstrate their correctness and efficiency completely.

Application of Demand-side Technology in Power System Intelligent Regulation

To reduce peak demand for electricity, smooth load curve shape, improvepower system safety and efficiency, this paper, by using intelligent homeappliance user operation comfort model is set up to quantify the acceptance,this paper proposes a maximum minimum load management algorithm basedon optimization strategy to change electric power use time and power con-sumption mode.The results show that the proposed model and algorithm canforecast and manage the power load well, and can reduce the peak to averageratio by 14.3% and the total expenditure by 15.3% while maintaining theoperating comfort of power users to the maximum.The load managementproblem of multiple power users can reach Nash equilibrium in a finitenumber of iterations, and this Nash equilibrium point is also the globaloptimal point.

Optimal R&D Investment Strategy of Pollution Abatement and Incentive Mechanism Design under Asymmetric Information

The study examines optimal pollution control R&D investment strategy of firms under asymmetric information and further analyzes the impact of government incentive mechanism on it. We use stochastic optimal control theory to get the exact solution of R&D investment strategy and incentive mechanism. Our analysis reveals that if there is no supervisor, firms choose not to cooperate, but the government can take appropriate incentive compensation to make firms reach Nash equilibrium. If there are supervisors, the optimal strategy of the enterprise is to choose cooperation, and there will be Pareto optimum among the firms. Furthermore, the R&D investment level decreases with increasing environmental uncertainty.

Constrained Generative Adversarial Networks

Generative Adversarial Networks (GANs) are a powerful subclass of generative models. Yet, how to effectively train them to reach Nash equilibrium is a challenge. A number of experiments have indicated that one possible solution is to bound the function space of the discriminator. In practice, when optimizing the standard loss function without limiting the discriminator's output, the discriminator may suffer from lack of convergence. To be able to reach the Nash equilibrium in a faster way during training and obtain better generative data, we propose constrained generative adversarial networks, GAN-C, where a constraint on the discriminator's output is introduced. We theoretically prove that our proposed loss function shares the same Nash equilibrium as the standard one, and our experiments on mixture of Gaussians, MNIST, CIFAR-10, STL-10, FFHQ, and CAT datasets show that our loss function can better stabilize training and yield even better high-quality images.

Multi-agent graphical games with input constraints: an online learning solution

This paper studies an online iterative algorithm for solving discrete-time multi-agent dynamic graphical games with input constraints. In order to obtain the optimal strategy of each agent, it is necessary to solve a set of coupled Hamilton-Jacobi-Bellman (HJB) equations. It is very difficult to solve HJB equations by the traditional method. The relevant game problem will become more complex if the control input of each agent in the dynamic graphical game is constrained. In this paper, an online iterative algorithm is proposed to find the online solution to dynamic graphical game without the need for drift dynamics of agents. Actually, this algorithm is to find the optimal solution of Bellman equations online. This solution employs a distributed policy iteration process, using only the local information available to each agent. It can be proved that under certain conditions, when each agent updates its own strategy simultaneously, the whole multi-agent system will reach Nash equilibrium. In the process of algorithm implementation, for each agent, two layers of neural networks are used to fit the value function and control strategy, respectively. Finally, a simulation example is given to show the effectiveness of our method.

RESERVE: An Energy-Efficient Edge Cloud Architecture for Intelligent Multi-UAV

With the increasing attraction of unmanned aerial vehicles (UAVs) in civil, public, and military applications, multi-UAV systems can perform environmental and disaster monitoring, border surveillance, and search and rescue. It is foreseen that these multi-UAV-based applications will be an important trend for edge computing scenarios. However, due to UAVs’ limited energy supplies as well as their continuous increase in the number of sensors, energy efficiency is a critical issue in multi-UAV systems. We believe that the fusion of edge computing and cloud computing can provide effective support for energy savings. This article presents an energy-efficient edge cloud architecture called RESERVE for intelligent multi-UAV. Under RESERVE, we study the energy-efficient computation offloading decision-making problem in a decentralized manner. The problem is formulated as a three-layer game in which the discretionary approach to reaching Nash Equilibrium is presented. Based on the proposed game, we design decentralized algorithms for two different cases. The algorithms can both achieve Nash Equilibrium. Furthermore, we propose a decentralized computation offloading mechanism and analyze the performance of the game by its efficiency ratio. We conduct simulation experiments and design a framework prototype. Evaluation results demonstrate that the proposed game methods can achieve more than 30 percent extra energy consumption reduction compared with the state-of-the-art decentralized algorithm and less than 10 percent performance loss relative to the centralized solution. The prototype framework we have developed proves the concept we propose.

An Enhanced Model-Free Reinforcement Learning Algorithm to Solve Nash Equilibrium for Multi-Agent Cooperative Game Systems

Solving the Nash equilibrium is important for multi-agent game systems, and the speed of reaching Nash equilibrium is critical for the agent to quickly make real-time decisions. A typical scheme is the model-free reinforcement learning algorithm based on policy iteration, which is slow because each iteration will be calculated from the start state to the end state. In this paper, we propose a faster scheme based on value iteration, using Q-function in an online manner to solve the Nash equilibrium of the system. Since the calculation is based on the value from the last iteration, the convergence speed of the proposed scheme is much faster than the policy iteration. The rationality and convergence of this scheme are analyzed and proved theoretically. An actor-critic network structure is used to implement this scheme through simulation. The simulation results show that the convergence speed of our proposed scheme is about 10 times faster than that of the policy iteration algorithm.

An Economic Incentive for D2D Assisted Offloading Using Stackelberg Game

The explosive growth of traffic demand challenges the current cellular network. Device-to-Device assisted offloading is a promising solution for alleviating traffic overloading burdens. However, a content transmitter (CT) unwillingly participates in cellular offloading due to the selfish nature and non-cooperation. To this end, an incentive mechanism initiated by mobile users (MUs) is proposed to motivate the CT’s participation for maximizing the CT’s profit and mobile users’ utilities. In particular, we consider the utility of a mobile user by jointly combining effects of social awareness and network congestion. The MUs and content providers offer proper economic payments to compensate the CT’s energy consumption cost. The interactions among participants formulate a two-stage Stackelberg game. The main aim of this game is to reach Nash equilibrium (NE) that neither the CT nor MUs have incentives to deviate unilaterally. We theoretically prove the existence and uniqueness of NE. Furthermore, iterative algorithms are designed to calculate NE for obtaining the best strategies of the CT and MUs. Numerical results demonstrate the robustness and feasibility of our proposed model.

GrHDP Solution for Optimal Consensus Control of Multiagent Discrete-Time Systems

This paper develops a new online learning consensus control scheme for multiagent discrete-time systems by goal representation heuristic dynamic programming (GrHDP) techniques. The agents in the whole system are interacted with each other through a communication graph structure. Therefore, each agent can only receive the information from itself and its neighbors. Our goal is to design the GrHDP method to achieve consensus control which makes all the agents track the desired dynamics and simultaneously makes the performance indices reach Nash equilibrium. The new local internal reinforcement signals and local performance indices are provided for each agent and the corresponding distributed control laws are designed. Then, GrHDP algorithm is developed to solve the multiagent consensus control problem with the proof of convergence. It is shown that the designed local internal reinforcement signals are bounded signals and the local performance indices can monotonically converge to their optimal values. Moreover, the desired distributed control laws can also achieve optimal. Two simulation studies, including one with four agents and another with ten agents, are applied to validate the theoretical analysis and also demonstrate the effectiveness of the proposed method.

Q-learning solution for optimal consensus control of discrete-time multiagent systems using reinforcement learning

This paper investigates a Q-learning scheme for the optimal consensus control of discrete-time multiagent systems. The Q-learning algorithm is conducted by reinforcement learning (RL) using system data instead of system dynamics information. In the multiagent systems, the agents are interacted with each other and at least one agent can communicate with the leader directly, which is described by an algebraic graph structure. The objective is to make all the agents achieve synchronization with leader and make the performance indices reach Nash equilibrium. On one hand, the solutions of the optimal consensus control for multiagent systems are acquired by solving the coupled Hamilton–Jacobi–Bellman (HJB) equation. However, it is difficult to get analytical solutions directly of the discrete-time HJB equation. On the other hand, accurate mathematical models of most systems in real world are hard to be obtained. To overcome these difficulties, Q-learning algorithm is developed using system data rather than the accurate system model. We formulate performance index and corresponding Bellman equation of each agent i. Then, the Q-function Bellman equation is acquired on the basis of Q-function. Policy iteration is adopted to calculate the optimal control iteratively, and least square (LS) method is employed to motivate the implementation process. Stability analysis of proposed Q-learning algorithm for multiagent systems by policy iteration is given. Two simulation examples are experimented to verify the effectiveness of the proposed scheme.

SA-IGA: a multiagent reinforcement learning method towards socially optimal outcomes

In multiagent environments, the capability of learning is important for an agent to behave appropriately in face of unknown opponents and dynamic environment. From the system designer's perspective, it is desirable if the agents can learn to coordinate towards socially optimal outcomes, while also avoiding being exploited by selfish opponents. To this end, we propose a novel gradient ascent based algorithm (SA-IGA) which augments the basic gradient-ascent algorithm by incorporating social awareness into the policy update process. We theoretically analyze the learning dynamics of SA-IGA using dynamical system theory and SA-IGA is shown to have linear dynamics for a wide range of games including symmetric games. The learning dynamics of two representative games (the prisoner's dilemma game and the coordination game) are analyzed in details. Based on the idea of SA-IGA, we further propose a practical multiagent learning algorithm, called SA-PGA, based on Q-learning update rule. Simulation results show that SA-PGA agent can achieve higher social welfare than previous social-optimality oriented Conditional Joint Action Learner (CJAL) and also is robust against individually rational opponents by reaching Nash equilibrium solutions.

Leader Selection via Supermodular Game for Formation Control in Multiagent Systems.

Multiagent systems (MASs) are usually applied with agents classified into leaders and followers, where selecting appropriate leaders is an important issue for formation control applications. In this paper, we investigate two leader selection problems in second-order MAS, namely, the problem of choosing up to a given number of leaders to minimize the formation error and the problem of choosing the minimum number of leaders to achieve a tolerated level of error. We propose a game theoretical method to address them. Specifically, we design a supermodular game for the leader selection problems and theoretically prove its supermodularity. In order to reach Nash equilibrium of the game, we propose strategies for the agents to learn to select leaders based on stochastic fictitious play. Extensive simulation results demonstrate that our method outperforms existing ones.