Cooperative Multi-agent Systems Research Articles

Learning Suboptimal Broadcasting Intervals in Multi-Agent Systems

In this paper, agents learn how often to exchange information with neighbors in cooperative Multi-Agent Systems (MASs) such that their user-defined cost functions are minimized. The investigated cost functions capture trade-offs between the MAS local control performance and energy consumption of each agent in the presence of exogenous disturbances. Agent energy consumption is critical for prolonging the MAS mission and is comprised of both control (e.g., acceleration, velocity) and communication efforts. The proposed methodology starts off by computing upper bounds on asynchronous broadcasting intervals that provably stabilize the MAS. Subsequently, we utilize these upper bounds as optimization constraints and employ an online learning algorithm based on Least Square Policy Iteration (LSPI) to minimize the cost function for each agent. Consequently, the obtained broadcasting intervals adapt to the most recent information (e.g., delayed and noisy agents’ inputs and/or outputs) received from neighbors and provably stabilize the MAS. Chebyshev polynomials are utilized as the approximator in the LSPI while Kalman Filtering (KF) handles sampled, corrupted and delayed data. The proposed methodology is exemplified in a consensus control problem with general linear agent dynamics.

Collective optimization problems with optimal decentralized selfish strategies

In this paper, we consider cooperative multi-agent systems minimizing a social cost. Cooperation is induced by the cost that penalizes the divergence of each agent from the average collective behavior. In principle, the optimal solution is centralized and requires a complete communication graph. We study this problem for two important system input-output norms, the l1 induced norm and the per-agent H2 norm squared, as function of the number of agents, n, and various cost structures. For the case of identical agents, and the simplest cost setting, we show that the optimal social solution is always the optimal decentralized selfish solution. For more general cost functions, we show that the optimal solution is always decentralized in the l1 induced norm case. In the case of the per-agent H2 norm squared cost, we show that the optimal decentralized selfish solution is socially optimal in the limit of large n. All of these also hold for several classes of problems with nonidentical agents. In simple terms, these results, identify important problem classes where decentralized/selfish behaviors are socially optimal, and for which inter-agent communication is or becomes unnecessary for large n.

On privacy vs. cooperation in multi-agent systems

ABSTRACTThis paper considers distributed systems arising when multiple agents cooperatively solve a quadratic optimisation problem. To maintain privacy of their states over time, agents implement a noise-adding mechanism according to the classic differential privacy framework. We characterise how the noise due to the privacy mechanism degrades the performance of the multi-agent system. Interestingly, we show that depending on the desired level of privacy (and thus noise), the system performance is optimised by reducing the level of cooperation among the agents. The notion of cooperation level, which is formally introduced and defined in the paper, models the trust of an agent towards the information received from neighbouring agents. For the prototypical examples of consensus and centroidal Voronoi tessellations, we are able to characterise the optimum cooperation level that maximises the system performance while ensuring a desired privacy level. Our results suggest that for the class of problems we study, and in fact for a broad class of multi-agent systems, it is always beneficial for the agents to reduce their cooperation level when the privacy level increases.

MetrIntPair-A Novel Accurate Metric for the Comparison of Two Cooperative Multiagent Systems Intelligence Based on Paired Intelligence Measurements

In this paper, we propose a novel metric called MetrIntPair (Metric for Pairwise Intelligence Comparison of Agent-Based Systems) for comparison of two cooperative multiagent systems problem-solving intelligence. MetrIntPair is able to make an accurate comparison by taking into consideration the variability in intelligence in problem-solving. The metric could treat the outlier intelligence indicators, intelligence measures that are statistically different from those others. For evaluation of the proposed metric, we realized a case study for two cooperative multiagent systems applied for solving a class of NP-hard problems. The results of the case study proved that the small difference in the measured intelligence of the multiagent systems is the consequence of the variability. There is no statistical difference between the intelligence quotients/level of the multiagent systems. Both multiagent systems should be classified in the same intelligence class.

Cooperative Multiagent System for Parking Availability Prediction Based on Time Varying Dynamic Markov Chains

Traffic congestion is one of the main issues in the study of transportation planning and management. It creates different problems including environmental pollution and health problem and incurs a cost which is increasing through years. One-third of this congestion is created by cars searching for parking places. Drivers may be aware that parking places are fully occupied but will drive around hoping that a parking place may become vacant. Opportunistic services, involving learning, predicting, and exploiting Internet of Things scenarios, are able to adapt to dynamic unforeseen situations and have the potential to ease parking search issues. Hence, in this paper, a cooperative dynamic prediction mechanism between multiple agents for parking space availability in the neighborhood, integrating foreseen and unforeseen events and adapting for long-term changes, is proposed. An agent in each parking place will use a dynamic and time varying Markov chain to predict the parking availability and these agents will communicate to produce the parking availability prediction in the whole neighborhood. Furthermore, a learning approach is proposed where the system can adapt to different changes in the parking demand including long-term changes. Simulation results, using synthesized data based on an actual parking lot data from a shopping mall in Geneva, show that the proposed model is promising based on the learning accuracy with service adaptation and performance in different cases.

Periodic Event-Triggered Synchronization of Linear Multi-Agent Systems With Communication Delays

Multi-agent systems cooperation to achieve global goals is usually limited by sensing, actuation, and communication issues. At the local level, continuous measurement and actuation is only approximated by the use of digital mechanisms that measure and process information in order to compute and update new control input values at discrete time instants. Interaction with other agents or subsystems takes place, in general, through a digital communication channel with limited bandwidth where transmission of continuous-time signals is not possible. Additionally, communication channels may be subject to other imperfections such as time-varying delays. This paper considers the problem of consensus (or synchronization of state trajectories) of multi-agent systems that are described by general linear dynamics and are connected using undirected graphs. An event-triggered consensus protocol is proposed, where each agent implements discretized and decoupled models of the states of its neighbors. This approach not only avoids the need for continuous communication between agents but also provides a decentralized method for transmission of information in the presence of time-varying communication delays where each agent decides its own broadcasting time instants based only on local information. This method gives more flexibility for scheduling information broadcasting compared to periodic and sampled-data implementations. The use of discretized models by each agent allows for a periodic event-triggered strategy where continuous actuation and continuous measurement of the states are not necessary.

The dynamics of reinforcement social learning in networked cooperative multiagent systems

Multiagent coordination in cooperative multiagent systems, as one of the fundamental problems in multiagent systems, and has been widely studied in the literature. In real environments, the interactions among agents are usually sparse and regulated by their underlying network structure, which, however, has received relatively few attentions in previous work. To this end, we firstly systematically investigate the multiagent coordination problems in cooperative environments under the networked social learning framework under four representative topologies. A networked social learning framework consists of a population of agents where each agent interacts with another agent randomly selected from its neighborhood in each round. Each agent updates its learning policy through repeated interactions with its neighbors via both individual learning and social learning. It is not clear a priori whether all agents are able to learn towards a consistent optimal coordination policy. Two types of learners are proposed: individual action learner and joint action learner. We evaluate the learning performances of both learners extensively in different cooperative (both single-stage and Markov) games. Besides, the influence of different factors (network topologies, different types of games, different topology parameters) is investigated and analyzed and new insights are obtained.

Distributed Finite-Time Voltage and Frequency Restoration in Islanded AC Microgrids

This paper investigates the distributed finite-time restoration of inverter voltage and frequency terms in an islanded ac microgrid. Formulating networked inverters of ac microgrids as a cooperative multiagent system, the voltage and frequency restoration can be cast as synchronization problems, while the active power sharing can be viewed as a consensus problem. One popular distributed control approach is the neighbor-based linear consensus protocol, where the consensus at the frequency and voltage set points is achieved over an infinite-time horizon with an exponential convergence. To achieve accelerated convergence and better disturbance rejection properties, a distributed finite-time control protocol, based on feedback linearization approach, is proposed for voltage restoration, which synchronizes the voltage term at each inverter to the reference value in finite time period. Then, a finite-time control protocol for both frequency restoration and active power sharing problems is proposed to synchronize the microgrid frequency to the nominal value, and share the active power among inverters based on their ratings in a finite time. Rigorous Lyapunov proofs are provided to establish the upper bounds on the convergence times. Numerical simulation studies verify the effectiveness of the proposed control protocols.

Cooperation of Multiagent Systems With Mismatch Parameters: A Viewpoint of Power Systems

This brief addresses a weak cooperation behavior, i.e., position cohesion and velocity consensus for a class of second-order multiagent systems with mismatch parameters from a viewpoint of power systems. All the agents can possess mismatch parameters and receive only the position information from its neighbors throughout the communication network. Sufficient conditions are established, and the hidden mechanism governing the weak cooperation behavior is elucidated. Simulations are given to illustrate the effectiveness of the derived theoretical results.

Incorporating the Negotiation Process in Urban Planning DSS

Cooperation in multi-agent systems is necessary in order to perform complex tasks and lead Multi-agent System (MAS) towards its objective. Contract-Net Protocol (CNP) is one of the communication and coordination mechanisms used by multi-agent systems which prefer cooperation through interaction protocols. This paper proposes a new cooperation and negotiation protocol based on the principals of the Contact Net Protocol (CNP). The auhtors' suggested negotiation protocol is used to solve one of the problems in the context of the city planning which is the problem of election of urban projects. Their proposed protocol is intended to the decision makers in order to help them resolve the problem of the evaluation and the selection of the best urban project without the need to be together in a decision urban room.

A Novel Robust Metric for Comparing the Intelligence of Two Cooperative Multiagent Systems

Cooperative multiagent systems are used for solving many computational hard problems. In the scientific literature, the intelligence of cooperative multiagent systems is considered at the systems’ level and is based on the “intelligent problem solving” consideration (highly efficient and flexible problems solving; difficult problem solving, with missing or erroneous data; efficient solving of NP – hard problems). In this paper, we propose a novel accurate metric called MetrIntComp (Metric for Cooperative Multiagent Systems Intelligence Comparison) for a robust comparison of two cooperative multiagent system's intelligence, effective even in the case of small differences in intelligence between the considered systems. For proving the effectiveness of the metric we considered an illustrative case study for two cooperative multiagent systems composed of simple agents, in that the intelligence emerge at the systems’ level, each of them specialized on solving the same type of computational difficult, NP-hard problem. The conclusion of the case study was that the metric is able to make a differentiation between the two multiagent systems even the numerical difference between the measured intelligence is small. Based on this fact, the two multiagent systems could not be considered that belong to the same class of intelligence.

Novelty-Driven Cooperative Coevolution.

Cooperative coevolutionary algorithms (CCEAs) rely on multiple coevolving populations for the evolution of solutions composed of coadapted components. CCEAs enable, for instance, the evolution of cooperative multiagent systems composed of heterogeneous agents, where each agent is modelled as a component of the solution. Previous works have, however, shown that CCEAs are biased toward stability: the evolutionary process tends to converge prematurely to stable states instead of (near-)optimal solutions. In this study, we show how novelty search can be used to avoid the counterproductive attraction to stable states in coevolution. Novelty search is an evolutionary technique that drives evolution toward behavioural novelty and diversity rather than exclusively pursuing a static objective. We evaluate three novelty-based approaches that rely on, respectively (1) the novelty of the team as a whole, (2) the novelty of the agents' individual behaviour, and (3) the combination of the two. We compare the proposed approaches with traditional fitness-driven cooperative coevolution in three simulated multirobot tasks. Our results show that team-level novelty scoring is the most effective approach, significantly outperforming fitness-driven coevolution at multiple levels. Novelty-driven cooperative coevolution can substantially increase the potential of CCEAs while maintaining a computational complexity that scales well with the number of populations.

Distributed Cooperative Control of Microgrid Storage

This paper proposes dynamic energy level balancing between distributed storage devices as a strategy to improve frequency regulation and reliability in droop controlled microgrids. This has been achieved with a distributed multi-agent cooperative control system which modifies the output power of droop controlled storage devices so that they reach a balanced energy state. As the storage devices approach a common energy level they are able to contribute their full power capacity to deal with generation and demand fluctuations in the microgrid. The cooperative control system also provides secondary frequency control, restoring the microgrid to the reference frequency. Simulations have been completed showing that the cooperative control system improves frequency regulation compared to traditional droop control strategies when the storage devices begin at different energy levels and the microgrid experiences generation or demand variability. A control input saturation constraint has been developed which ensures that the cooperative control system will not overload the storage devices.

Learning Multirobot Hose Transportation and Deployment by Distributed Round-Robin Q-Learning

Multi-Agent Reinforcement Learning (MARL) algorithms face two main difficulties: the curse of dimensionality, and environment non-stationarity due to the independent learning processes carried out by the agents concurrently. In this paper we formalize and prove the convergence of a Distributed Round Robin Q-learning (D-RR-QL) algorithm for cooperative systems. The computational complexity of this algorithm increases linearly with the number of agents. Moreover, it eliminates environment non sta tionarity by carrying a round-robin scheduling of the action selection and execution. That this learning scheme allows the implementation of Modular State-Action Vetoes (MSAV) in cooperative multi-agent systems, which speeds up learning convergence in over-constrained systems by vetoing state-action pairs which lead to undesired termination states (UTS) in the relevant state-action subspace. Each agent’s local state-action value function learning is an independent process, including the MSAV policies. Coordination of locally optimal policies to obtain the global optimal joint policy is achieved by a greedy selection procedure using message passing. We show that D-RR-QL improves over state-of-the-art approaches, such as Distributed Q-Learning, Team Q-Learning and Coordinated Reinforcement Learning in a paradigmatic Linked Multi-Component Robotic System (L-MCRS) control problem: the hose transportation task. L-MCRS are over-constrained systems with many UTS induced by the interaction of the passive linking element and the active mobile robots.

Improving the Multi-Agent General Cooperation Model Based on SVD

<p class="p16">The cooperation is the most important for a multi-agent system (MAS). In order to research the cooperation, creating a multi-agent general cooperation model, that not only thinking about the cooperation solving way, but also thinking about the cooperation necessity and how to solve the conflicts, was a feasible method. However, the cooperation model might be inefficient. This paper suggested, based on the singular value decomposition (SVD), simplifying multi-agent cooperation data, and then improving the general cooperation model. According to the simulation results, the improved model had higher efficiency and can provide a more reliable tool for the study of the cooperation in MAS.</p>

HCI Model with Learning Mechanism for Cooperative Design in Pervasive Computing Environment

This paper presents a human-computer interaction model with a three layers learning mechanism in a pervasive environment. We begin with a discussion around a number of important issues related to human-computer interaction followed by a description of the architecture for a multi-agent cooperative design system for pervasive computing environment. We present our proposed three-layer HCI model and introduce the group formation algorithm, which is predicated on a dynamic sharing niche technology. Finally, we explore the cooperative reinforcement learning and fusion algorithms; the paper closes with concluding observations and a summary of the principal work and contributions of this paper.

Privacy preserving existence recognition and construction of hypertree agent organization

Decentralized probabilistic reasoning, constraint reasoning, and decision theoretic reasoning are some essential tasks of cooperative multiagent systems. Several frameworks for these tasks organize agents into a junction tree (JT). We show that existing techniques for JT existence recognition and construction leak information on private variables, shared variables, agent identities and adjacency, that can potentially be protected. We present a scheme to quantify these privacy losses. We develop two novel algorithms for JT existence recognition and for JT construction when existing, that provide strong guarantee of agent privacy. Our experimental comparison shows that the proposed algorithms out-perform existing techniques, one of them having the lowest privacy loss and the other having no privacy loss, while being more efficient than most alternatives.

Special issue on learning and control in cooperative multi-agent systems

Special issue on learning and control in cooperative multi-agent systems

Improving the Quality of Information in Strategic Scanning System Network: Approach Based on Cooperative Multi-Agent System

Integrating Business Intelligence (BI) processes in an information system requires a form of strategic scanning system for which the information is the main source of efficiency and decision support. A process of strategic scanning system network is primarily a cooperative approach to sharing knowledge that actors are producers of information. The dynamics of the actor’s interactions allow gradual building of shared knowledge. This paper proposes Multi Agent System (MAS) architecture which facilitates the integration of a process of strategic scanning system network in the information system, to emerge relevant information from simple information while ensuring the quality and safety information . In particular, this approach is geared towards supporting system properties specially focused on cooperative multi-agent system. It gives finally an overview of implementation of a prototype of the proposed solution limited for the moment to the integration of processes most used in the majority of information systems.

A novel approach to task assignment in a cooperative multi-agent design system

The task assignment problem is an important topic in multi-agent systems research. Distributed real-time systems must accommodate a number of communication tasks, and the difficulty in building suc...