Multi-agent Cooperation Research Articles

Enhancing Cooperative Multi-Agent Systems With Self-Advice and Near-Neighbor Priority Collision Control

Enhancing Cooperative Multi-Agent Systems With Self-Advice and Near-Neighbor Priority Collision Control

Priority Over Quantity: A Self-Incentive Credit Assignment Scheme for Cooperative Multiagent Reinforcement Learning

Priority Over Quantity: A Self-Incentive Credit Assignment Scheme for Cooperative Multiagent Reinforcement Learning

Multi-Disciplinary and Multi-Objective Optimization Method Based on Machine Learning

The optimization of aircraft is a typical multidisciplinary and multi-objective problem. To solve this problem, the difficulty lies not only in the high cost of discipline performance evaluation but also in the complex coupling relationship between different disciplines. To improve the optimization efficiency, a new optimization method is proposed, including two new algorithms: conditional generative adversarial nets with vector similarity (VS-CGAN) and distributed single-step deep reinforcement learning with transfer learning (TL-DSDRL). For low-cost disciplines, VS-CGAN learns the relationship between variables and objectives through presampling to compress the variable domains. The cosine function is used to evaluate the similarity between the random noise and generated variables to avoid mode collapse. For high-cost disciplines, TL-DSDRL improves optimization efficiency through pretraining. The newly designed reward function and multi-agent cooperation mechanism enhance the multi-objective search ability of reinforcement learning.

Nash bargaining based integrated energy agent optimal operation strategy considering negotiation pricing for tradable green certificate

Reasonable tradable green certificate (TGC) price is the key to guiding the orderly transaction of market agents. However, there is little literature on the negotiation pricing of green certificates, especially the multi-agent cooperation optimization operation strategy for green certificate negotiation. To improve the enthusiasm for TGC transactions and the low-carbon economy of the system, this paper studies TGC negotiation and cooperative operation among multi-agents of integrated energy. Firstly, under the influence of the supply and demand relationship and renewable energy generation cost on TGC price, a variable TGC price limit model is constructed. Secondly, the TGC-carbon emissions transaction (CET) interaction mechanism is introduced to the endogenous motivation for TGC transactions. Then, a Nash bargaining theory is used to establish an integrated energy multi-agent cooperative game model based on TGC negotiation pricing. Finally, the non-convex nonlinear model is transformed into a subproblem of distributed solution by ADMM principle, and the AS-ADMM algorithm is used to solve it. The example shows that the model proposed in this paper effectively promotes the transaction volume of green certificates between the agents. Compared with the non-cooperative situation, the transaction volume increased by 20.83%, indicating that the agent's enthusiasm for TGC transactions was improved, and the number of transactions exceeded their liability quotas.

Peer Incentive Reinforcement Learning for Cooperative Multiagent Games

Social learning, especially social incentives, is extremely important for humans to achieve a high level of coordination. Inspired by this, we introduce this concept into cooperative multi-agent reinforcement learning (MARL), to implicitly address the credit assignment problem and promote the inter-agent direct interactions for cooperations among agents in cooperative multi-agent games. In this paper, we propose a novel Intrinsic Reward method with Peer Incentives (IRPI) based on actor-critic policy gradient. This method can enable agents to incentivize each other for their cooperations through using causal influence among them. Specifically, a novel intrinsic reward mechanism is innovatively designed to empower each agent the ability to give positive or negative rewards to other peer agents' actions through considering the causal influence of the other agents on it. The mechanism is realized by a feed-forward neural network through utilizing causal influence between the agents. The causal influence of one agent on another is inferred via counterfactual reasoning using the joint action-value function in MARL. The quality of the influence is assessed via counterfactual reasoning using the individual value function in MARL. Simulations are carried out on two popular multi-agent game testbeds: Starcraft II Micromanagement and Multi-Agent Particle Environments. Simulational results demonstrate that the proposed IRPI can enhance cooperations among the agents to achieve better performance compared with a number of state-of-the-art MARL methods in a variety of cooperative multi-agent games.

Multi-agent Cooperative Optimization Strategy of a Virtual Power Plant Based on Game Theory

This paper proposes a multi-agent collaborative optimization method for virtual power plants based on the Bayesian game. Focusing on the internal scheduling of VPP, taking demand response load as the analysis object, aiming at the interactive optimization relationship between multiple intelligent communities in the virtual power plant under this scenario, a multi-agent collaborative optimization model of the virtual power plant is established by using Bayesian game theory. The effectiveness of the constructed game model and algorithm is verified by solving the actual example of the virtual power plant and analyzing the corresponding simulation results.

ATT-TA: A Cooperative Multiagent Deep Reinforcement Learning Approach for TSV Assignment in 3-D ICs

ATT-TA: A Cooperative Multiagent Deep Reinforcement Learning Approach for TSV Assignment in 3-D ICs

Optimal [formula omitted]-matrix completion with majorization ordered objectives

We propose and examine two inherently symmetric (0,1)-matrix completion problems with majorization ordered objectives, which provides a unique perspective for electric vehicle charging, portfolio optimization, and multi-agent cooperation. Our work elevates the seminal study by Gale and Ryser from feasibility to optimality in partial order programming (POP), referring to optimization with partially ordered objectives. Solving such integer POP (iPOP) problems is challenging because of the integer requirements and the fact that two objective values may not be comparable. Nevertheless, we prove the essential uniqueness of all optimal objective values and identify two particular ones for each iPOP problem. Furthermore, for every optimal objective value of each iPOP problem, we respectively develop a “peak-shaving” and a “valley-filling” algorithm to construct an associated optimal (0,1)-matrix via a series of sorting processes. We show that the resulting algorithms have linear time complexities and numerically verify their efficiency compared to the commonly used order-preserving method for POP.

Multi-Agent Cooperative Game Based Task Computing Mechanism for UAV-Assisted 6G NTN

Multi-Agent Cooperative Game Based Task Computing Mechanism for UAV-Assisted 6G NTN

Attention Enhanced Reinforcement Learning for Multi agent Cooperation.

In this article, a novel method, called attention enhanced reinforcement learning (AERL), is proposed to address issues including complex interaction, limited communication range, and time-varying communication topology for multi agent cooperation. AERL includes a communication enhanced network (CEN), a graph spatiotemporal long short-term memory network (GST-LSTM), and parameters sharing multi-pseudo critic proximal policy optimization (PS-MPC-PPO). Specifically, CEN based on graph attention mechanism is designed to enlarge the agents' communication range and to deal with complex interaction among the agents. GST-LSTM, which replaces the standard fully connected (FC) operator in LSTM with graph attention operator, is designed to capture the temporal dependence while maintaining the spatial structure learned by CEN. PS-MPC-PPO, which extends proximal policy optimization (PPO) in multi agent systems with parameters' sharing to scale to environments with a large number of agents in training, is designed with multi-pseudo critics to mitigate the bias problem in training and accelerate the convergence process. Simulation results for three groups of representative scenarios including formation control, group containment, and predator-prey games demonstrate the effectiveness and robustness of AERL.

Optimizing Multi-Agency Cooperation in Kenya’s National Security Planning and Response

Optimizing Multi-Agency Cooperation in Kenya’s National Security Planning and Response

Learning in Cooperative Multiagent Systems Using Cognitive and Machine Models

Developing effective multi-agent systems (MASs) is critical for many applications requiring collaboration and coordination with humans. Despite the rapid advance of multi-agent deep reinforcement learning (MADRL) in cooperative MASs, one of the major challenges that remain is the simultaneous learning and interaction of independent agents in dynamic environments in the presence of stochastic rewards. State-of-the-art MADRL models struggle to perform well in Coordinated Multi-agent Object Transportation Problems (CMOTPs) wherein agents must coordinate with each other and learn from stochastic rewards. In contrast, humans often learn rapidly to adapt to non-stationary environments that require coordination among people. In this article, motivated by the demonstrated ability of cognitive models based on Instance-based Learning Theory (IBLT) to capture human decisions in many dynamic decision-making tasks, we propose three variants of multi-agent IBL models (MAIBLs). The idea of these MAIBL algorithms is to combine the cognitive mechanisms of IBLT and the techniques of MADRL models to deal with coordination MASs in stochastic environments from the perspective of independent learners. We demonstrate that the MAIBL models exhibit faster learning and achieve better coordination in a dynamic CMOTP task with various settings of stochastic rewards compared to current MADRL models. We discuss the benefits of integrating cognitive insights into MADRL models.

MO-MIX: Multi-Objective Multi-Agent Cooperative Decision-Making With Deep Reinforcement Learning.

Deep reinforcement learning (RL) has been applied extensively to solve complex decision-making problems. In many real-world scenarios, tasks often have several conflicting objectives and may require multiple agents to cooperate, which are the multi-objective multi-agent decision-making problems. However, only few works have been conducted on this intersection. Existing approaches are limited to separate fields and can only handle multi-agent decision-making with a single objective, or multi-objective decision-making with a single agent. In this paper, we propose MO-MIX to solve the multi-objective multi-agent reinforcement learning (MOMARL) problem. Our approach is based on the centralized training with decentralized execution (CTDE) framework. A weight vector representing preference over the objectives is fed into the decentralized agent network as a condition for local action-value function estimation, while a mixing network with parallel architecture is used to estimate the joint action-value function. In addition, an exploration guide approach is applied to improve the uniformity of the final non-dominated solutions. Experiments demonstrate that the proposed method can effectively solve the multi-objective multi-agent cooperative decision-making problem and generate an approximation of the Pareto set. Our approach not only significantly outperforms the baseline method in all four kinds of evaluation metrics, but also requires less computational cost.

Multi-agent cooperation policy gradient method based on enhanced exploration for cooperative tasks

Multi-agent cooperation policy gradient method based on enhanced exploration for cooperative tasks

A Cooperative Multiagent Reinforcement Learning Framework for Droplet Routing in Digital Microfluidic Biochips

A Cooperative Multiagent Reinforcement Learning Framework for Droplet Routing in Digital Microfluidic Biochips

Multi-Agent Task Allocation with Multiple Depots Using Graph Attention Pointer Network

The study of the multi-agent task allocation problem with multiple depots is crucial for investigating multi-agent collaboration. Although many traditional heuristic algorithms can be adopted to handle the concerned task allocation problem, they are not able to efficiently obtain optimal or suboptimal solutions. To this end, a graph attention pointer network is built in this paper to deal with the multi-agent task allocation problem. Specifically, the multi-head attention mechanism is employed for the feature extraction of nodes, and a pointer network with parallel two-way selection and parallel output is introduced to further improve the performance of multi-agent cooperation and the efficiency of task allocation. Experimental results are provided to show that the presented graph attention pointer network outperforms the traditional heuristic algorithms.

A Variation-Aware Quantum Circuit Mapping Approach Based on Multi-Agent Cooperation

Quantum circuit mapping is an essential process required by executing quantum circuits using a noisy intermediate-scale quantum (NISQ) device. Since qubits and quantum gates of a NISQ device are error-prone and variable in quality, it is crucial to choose qubits or quantum gates in a variation-aware manner to maximize the success rate of executing circuits. To this end, this paper proposes a variation-aware method for quantum circuit mapping through the cooperation of multiple agents. Each agent in the proposed method can gradually construct a physical circuit that respects the device's connectivity constraints by inserting a SWAP gate at each step. Moreover, at each step, the circuit information of each agent is shared within the agent population through a communication mechanism that combines global and local information exchange, so that agents with poor fitness can get an opportunity to improve their physical circuits. The experimental results on extensive benchmark circuits confirm that the proposed method can effectively and consistently improve the overall circuit fidelity compared with the state-of-the-art methods.

Evolutionary reinforcement learning algorithm for large-scale multi-agent cooperation and confrontation applications

Evolutionary reinforcement learning algorithm for large-scale multi-agent cooperation and confrontation applications

Can multi-agent cooperation promote the ecological value realization of blue carbon in marine ranching?

Ecological value realization of blue carbon in marine ranching is essential to achieve carbon neutrality. The motivation for conducting the study is to verify whether multi-agent cooperation can promote the ecological value realization of blue carbon in marine ranching. Based on the blue carbon ecological value realization model of marine ranching enterprises, blue carbon demand enterprises, blue carbon trading platforms and government, this paper explores the cooperative governance strategy of marine ranching for each subject using cooperative game and non-cooperative game models. Further, we conduct a comparative analysis to arrive at the optimal strategy. The conclusions are as follows. Multi-agent cooperation is more conducive to the ecological value realization of blue carbon in marine ranching. Compared with non-cooperative governance, the platform's commission and blue carbon price are lower, and the blue carbon output, profit of each market subject, government utility and overall profit are higher in cooperative governance. The strengths of this paper lie in 2 aspects. First, we focuses on the ecological value of blue carbon in marine ranching instead of economic value, providing a new theoretical basis for ecological compensation in marine ranching. Second, we construct a government-led and market-oriented operation of marine ranching's blue carbon ecological value realization mechanism, incorporating blue carbon trading platform and government into the value realization model.

Mixture of personality improved spiking actor network for efficient multi-agent cooperation.

Adaptive multi-agent cooperation with especially unseen partners is becoming more challenging in multi-agent reinforcement learning (MARL) research, whereby conventional deep-learning-based algorithms suffer from the poor new-player-generalization problem, possibly caused by not considering theory-of-mind theory (ToM). Inspired by the ToM personality in cognitive psychology, where a human can easily resolve this problem by predicting others' intuitive personality first before complex actions, we propose a biologically-plausible algorithm named the mixture of personality (MoP) improved spiking actor network (SAN). The MoP module contains a determinantal point process to simulate the formation and integration of different personality types, and the SAN module contains spiking neurons for efficient reinforcement learning. The experimental results on the benchmark cooperative overcooked task showed that the proposed MoP-SAN algorithm could achieve higher performance for the paradigms with (learning) and without (generalization) unseen partners. Furthermore, ablation experiments highlighted the contribution of MoP in SAN learning, and some visualization analysis explained why the proposed algorithm is superior to some counterpart deep actor networks.