Large-scale Multi-agent Systems Research Articles

Leader-following consensus for nonlinear multiagent systems with large delays

This paper considers the leader-following consensus problem for nonlinear multiagent systems with large delays. The time delays switch alternately between two different kinds of delays, called a small delay and a large delay. New Lyapunov functionals that depend on graph information are constructed. To guarantee the desired leader-following consensus, the concepts of the length and frequency of the large delay periods are introduced. Sufficient conditions for consensus are given in terms of linear matrix inequalities. Furthermore, low-dimensional criteria that are easily implemented especially for large-scale multiagent systems are provided. Finally, two simulation examples are presented to validate the effectiveness of the proposed scheme.

Multiple Intentional Delays Can Facilitate Fast Consensus and Noise Reduction in a Multiagent System.

We investigate the use of a derivative-free control scheme called the multiple-delay proportional-retarded (PR) protocol to achieve fast consensus in a large-scale multiagent system. The delays are intentionally introduced in the PR with the purpose of creating derivative-like controllers that rely only on position measurements, thus mitigating undesirable noise effects. The main result places the spectral abscissa of the consensus dynamics at a desired locus and also achieves a user-defined spectral "separation," both of which directly influence convergence to consensus. Design rules and limitations arising from analytical derivations to achieve these results are laid out. Case studies are provided to demonstrate the concepts.

Distributed Clustering Algorithm for Energy Efficiency and Load-Balance in Large-Scale Multi-Agent Systems

To improve the energy efficiency and load-balance in large-scale multi-agent systems, a large-scale distributed cluster algorithm is proposed. At first, a parameter describing the spatial distribution of agents is designed to assess the information spreading capability of an agent. Besides, a competition resolution mechanism is proposed to tackle the competition problem in large-scale multiagent systems. Thus, the proposed algorithm can balance the load, adjust the system network locally and dynamically, reduce system energy consumption. Finally, simulations are presented to demonstrate the superiority of the proposed algorithm.

Quantitative analysis of multi-agent systems through statistical verification of simulation traces

Due to their immense complexity, large-scale multi-agent systems are often not amenable to exhaustive formal verification. Statistical approaches that focus on the verification of individual traces can provide an interesting alternative that circumvents combinatorial explosion. However, due to its focus on finite execution paths, trace-based verification is inherently limited to certain types of correctness properties. We show how, by combining sampling with the idea of trace fragmentation, statistical verification can be used to answer interesting quantitative correctness questions about multi-agent systems at different observational levels. The usefulness of the verification approach is illustrated with a simple case study from the area of swarm robotics.

On Scalable Supervisory Control of Multi-Agent Discrete-Event Systems

In this paper we study multi-agent discrete-event systems where the agents can be divided into several groups, and within each group the agents have similar or identical state transition structures. We employ a relabeling map to generate a "template structure" for each group, and synthesize a scalable supervisor whose state size and computational process are independent of the number of agents. This scalability allows the supervisor to remain invariant (no recomputation or reconfiguration needed) if and when there are agents removed due to failure or added for increasing productivity. The constant computational effort for synthesizing the scalable supervisor also makes our method promising for handling large-scale multi-agent systems. Moreover, based on the scalable supervisor we design scalable local controllers, one for each component agent, to establish a purely distributed control architecture.

Organisational Metamodel for Large-Scale Multi-Agent Systems: First Steps Towards Modelling Organisation Dynamics

The research presented in this paper is a thesis proposal with the main goal of defining an ontology comprising chosen organisational concepts applicable to large-scale multiagent systems (LSMAS), and building a metamodel for modelling selected organisational features in such systems. The method of applying aspects of human organisations to multiagent systems (MAS) comprising autonomous intelligent agents will be enriched through this research with a new perspective of modelling organisation dynamics in LSMAS. Results of this research, in their final version, will be tested using testbed scenarios based on a specific massively multi-player online role-playing game (MMORPG), since MMORPGs are one of the identified application domains of LSMAS. It is important to note that results described in this paper showcase partial results in their early stage of development. Nevertheless, first traces of a modelling tool that is expected to aid in development of LSMAS for numerous application domains, and ease their organisational design, are recognisable in the proposed combination of ontology engineering, metamodelling and code generating methods.

On convexity and monotonicity in generalized aggregative games

Large-scale multi-agent systems arise in several control applications, such as demand side management in smart power networks, traffic control in intelligent transportation systems, and congestion control in communication networks. All these systems comprise a large population of noncooperative agents whose decision making process can be modeled as a generalized aggregative game. In this paper, we analyze the usual convexity assumption and the monotonicity of the so-called game mapping. Specifically, we show necessary and sufficient conditions on the problem data such that monotonicity holds. The derived conditions show that monotonicity cannot be expected in general.

A Distributed Least-Squares Algorithm in Wireless Sensor Networks With Unknown and Limited Communications

In this article, the authors propose a new distributed least-squares algorithm to address the sensor fusion problem in using wireless sensor networks (WSN) to monitor the behaviors of large-scale multiagent systems. Under a mild assumption on network observability, that is, each sensor can take the measurements of a limited number of agents but the complete multiagent systems are covered under the union of all sensors in the network, the proposed algorithm achieves the estimation consensus if local information exchange can be performed among sensors. The proposed distributed least-squares algorithm can handle the directed communication network by explicitly estimating the left eigenvector corresponding to the largest eigenvalue of the sensing/communication matrix. The convergence of the proposed algorithm is analyzed, and simulation results are provided to further illustrate its effectiveness.

A Boltzmann approach to mean-field sparse feedback control

We study the synthesis of optimal control policies for large-scale multi-agent systems. The optimal control design induces a parsimonious control intervention by means of l, sparsity-promoting control penalizations. We study instantaneous and infinite horizon sparse optimal feedback controllers. In order to circumvent the dimensionality issues associated to the control of large-scale agent-based models, we follow a Boltzmann approach. We generate (sub)optimal controls signals for the kinetic limit of the multi-agent dynamics, by sampling of the optimal solution of the associated two-agent dynamics. Numerical experiments assess the performance of the proposed sparse design.

A fuzzy-based function approximation technique for reinforcement learning1

5 Abstract. Reinforcement learning is hard to solve optimization problems in multi-agent system because of the inefficiency of function approximation. Sparse distributed memories, which is implemented using Radial Basis Functions or Kanerva Coding, can be used to improve the efficiency. But this approach still often gives poor performance when applied to large-scale multi-agent systems. In this paper, we attempt to solve four-rooms problem in the predator-prey pursuit domain and argue that the poor performance that we observe is caused by frequent prototype collisions. We show that dynamic prototype allocation and adaptation can give better results by reducing these collisions. By using our novel approach about fuzzy Kanerva-based function approximation, that uses a fine-grained fuzzy membership grade to describe a state-action pair's adjacency with respect to each prototype, we give some results that prototype collisions are completely eliminated and learning performance is greatly improved. We further show that prototype density varies widely across the state-action space and that this variation causes prototypes' receptive fields to be unevenly distributed. This distribution limits the ability of fuzzy Kanerva Coding to achieve better results. It can be observed that fuzzy Kanerva Coding allows prototypes to adaptively tune their receptive fields for a target application. We conclude that fuzzy Kanerva Coding with prototype tuning and adaptation can significantly improve a reinforcement learner's ability to solve large-scale multi-agent problems. 6 7 8 9 10 11 12 13 14 15 16 17 18

Linear Quadratic Risk-Sensitive and Robust Mean Field Games

This paper considers two classes of large population stochastic differential games connected to optimal and robust decentralized control of large-scale multiagent systems. The first problem ( P1 ) is one where each agent minimizes an exponentiated cost function, capturing risk-sensitive behavior, whereas in the second problem ( P2 ) each agent minimizes a worst-case risk-neutral cost function, where the “worst case” stems from the presence of an adversary entering each agent’s dynamics characterized by a stochastic differential equation. In both problems, the individual agents are coupled through the mean field term included in each agent’s cost function, which captures the average or mass behavior of the agents. We solve both P1 and P2 via mean field game theory. Specifically, we first solve a generic risk-sensitive optimal control problem and a generic stochastic zero-sum differential game, where the corresponding optimal controllers are applied by each agent to construct the mean field systems of P1 and P2 . We then characterize an approximated mass behavior effect on an individual agent via a fixed-point analysis of the mean field system. For each problem, P1 and P2 , we show that the approximated mass behavior is in fact the best estimate of the actual mass behavior in various senses as the population size, $N$ , goes to infinity. Moreover, we show that for finite $N$ , there exist $\epsilon$ - Nash equilibria for both P1 and P2 , where the corresponding individual Nash strategies are decentralized in terms of local state information and the approximated mass behavior. We also show that $\epsilon$ can be taken to be arbitrarily small when $N$ is sufficiently large. We show that the $\epsilon$ - Nash equilibria of P1 and P2 are partially equivalent in the sense that the individual Nash strategies share identical control laws, but the approximated mass behaviors for P1 and P2 are different, since in P2 , the mass behavior is also affected by the associated worst-case disturbance. Finally, we prove that the Nash equilibria for P1 and P2 both feature robustness, and as the parameter characterizing this robustness becomes infinite, the two Nash equilibria become identical and equivalent to that of the risk-neutral case, as in the one-agent risk-sensitive and robust control theory.

Distributed consensus of large-scale multi-agent systems via linear-transformation-based partial stability approach

This paper investigates the consensus problem of large-scale multi-agent systems (MASs) with a directed communication topology, especially for the MAS whose topology contains small strongly connected components. A sufficient and necessary consensus criterion is proposed through the combination of two methods. Firstly, a communication searching algorithm is utilized to make sure that each agent obtains the local topology information of strongly connected component where it is located. Secondly, a state-linear-transformation decomposes the consensus problem into a group of stability problems based on the strongly connected components. The corresponding consensus criterion reduces the computational complexity. Moreover, according to the consensus criterion, a distributed design procedure of gain matrices is proposed based on the homotopy method.

A consensus recovery approach to nonlinear multi-agent system under node failure

In large-scale networks, node failure is unavoidable and it generally brings undesirable effects on the stability and performance of the systems. Then a natural question arises: is it possible to cope with the problem arising from the node failure in the networks, while retaining the corresponding network property (or make the network systems robust against the node failure)? In this paper, a consensus recovery approach will be proposed to investigate the consensus of general directed nonlinear multi-agent networks with node failure. The objective of the consensus recovery method is to compensate for the undesirable effects of the failure nodes, and to retain the consensus property. Numerical examples are provided to demonstrate the effectiveness of the theoretical results obtained, even for large-scale multi-agent systems.

A platform for large-scale game-playing multi-agent systems on a high performance computing infrastructure

The simulation of societies requires vast amounts of computing resources, which must be managed over distributed or high performance computing infrastructures to provide for cost-effective experimentation. To that end, this paper presents a novel platform for the segmentation and management of social simulation experiments in game-playing multi-agent systems; the platform, also serves as a working proof of concept for similar experiments. The platform is managed through a web-based graphical user interface, to combine the advantages of powerful grid infrastructure middleware and sophisticated workflow systems in a way that some generic functionality is sacrificed for the benefit of obtaining a smooth and brief learning curve, without compromising security. The paper sets out the architecture and implementation details of the platform and demonstrates its use with two sample games, RLGame and Rock Scissors Paper, to underline the scale of the experiments and to indicate the class of social simulation problems that it can help investigate. The platform can be loosely coupled with analytics software for data mining; for our sample problems, this analysis leads to associating the learning mechanism each agent employs with its eventual performance ranking.

Robust scalable stabilisability conditions for large-scale heterogeneous multi-agent systems with uncertain nonlinear interactions: towards a distributed computing architecture

ABSTRACTLarge-scale dynamic systems are becoming highly pervasive in their occurrence with applications ranging from system biology, environment monitoring, sensor networks, and power systems. They are characterised by high dimensionality, complexity, and uncertainty in the node dynamic/interactions that require more and more computational demanding methods for their analysis and control design, as well as the network size and node system/interaction complexity increase. Therefore, it is a challenging problem to find scalable computational method for distributed control design of large-scale networks. In this paper, we investigate the robust distributed stabilisation problem of large-scale nonlinear multi-agent systems (briefly MASs) composed of non-identical (heterogeneous) linear dynamical systems coupled by uncertain nonlinear time-varying interconnections. By employing Lyapunov stability theory and linear matrix inequality (LMI) technique, new conditions are given for the distributed control design of large-scale MASs that can be easily solved by the toolbox of MATLAB. The stabilisability of each node dynamic is a sufficient assumption to design a global stabilising distributed control. The proposed approach improves some of the existing LMI-based results on MAS by both overcoming their computational limits and extending the applicative scenario to large-scale nonlinear heterogeneous MASs. Additionally, the proposed LMI conditions are further reduced in terms of computational requirement in the case of weakly heterogeneous MASs, which is a common scenario in real application where the network nodes and links are affected by parameter uncertainties. One of the main advantages of the proposed approach is to allow to move from a centralised towards a distributed computing architecture so that the expensive computation workload spent to solve LMIs may be shared among processors located at the networked nodes, thus increasing the scalability of the approach than the network size. Finally, a numerical example shows the applicability of the proposed method and its advantage in terms of computational complexity when compared with the existing approaches.

Evaluating fault tolerance approaches in multi-agent systems

A multi-agent system (MAS) is a distributed system that consists of multiple agents working together to solve mutual problems. Even though MASs are well suited for the development of complex distributed systems, the number of real-world usages is still small. One of the main reasons for this is that MASs are very fragile. In a typical, large-scale MAS, the rate of failure grows with the number of hosts, the number of deployed agents, and the duration of the agent's task execution. For this reason, numerous approaches have been introduced to deal with aspects of failure handling. However, the absence of centralized control and a large number of individual intelligent components makes it difficult to detect and treat errors. The risk of uncontrollable fault propagation is high and can seriously impact on system performance. There are two important factors that limit the usage of MASs: (1) existing fault tolerance (FT) approaches are not generic, as they focus on and improve specific issues of FT; and (2) despite the plethora of available FT approaches and theories, there is a remarkable lack of general metrics, tools, benchmarks, and experimental methods for formal validation and comparison of existing or newly developed FT approaches. As FT approaches in MASs become a well-established field, the need for generalized, standardized evaluation of FT approaches emerges as imperative. In this paper, we first present a detailed overview of existing FT solutions, approaches, and techniques in agent platform hosted MASs. From that overview, we derive the commonalities in existing research. Next, we present the main contribution of our paper: an evaluation methodology, with a set of metrics, for comparing FT approaches in MASs. We adopt an engineering perspective on the problem, defining a methodology and metrics that are both implementation- and domain-independent. The metrics are formalized with an acyclic directed graph. By using our methodology, evaluators can select an appropriate FT approach for targeted MAS application, thus improving MAS usability, stability, and development speed. In order to show the viability of our approach, a case study that compares two FT approaches for a targeted MAS is presented. The case study results show that our methodology can be used for selecting an appropriate FT approach for the targeted MAS.

Realizing the NOSHAPE MAS Organizational Model

Dynamic reorganization of multi-agent systems (MAS) is currently an interesting and active research area. New emerging concepts such as self-organization and emergence are also getting great focus by MAS research community because of their captivating benefits for engineering complex large-scale MAS applications. Dynamic reorganization is considered as an effective way for building adaptive MAS. This paper provides an operational view of the conceptually proposed NOSHAPE dynamic organizational model for dynamically reorganized multi-agent systems (Abbas, 2014). The operational view concerns possible design decisions, implementation issues, and suggested usage of the NOSHAPE model for practically engineering real life applications. This paper also identifies the possible solutions to be applied throughout the system design to resolve these concerns.

Role and member selection in team formation using resource estimation for large-scale multi-agent systems

We propose an efficient team formation method for multi-agent systems consisting of self-interested agents in task-oriented domains where agents have no prior knowledge of the resources/abilities of the other agents. Internet services based on services computing and cloud computing, which have been rapidly increasing, are usually achieved by combining a number of service elements that are distributed over the Internet. We modelled the executions of these elements as teams of agents with the resources and abilities required in the corresponding service elements. This team formation method with the appropriate agents for the service elements makes the entire system efficient. Our proposed method is based on our previous parameter learning method that enables agents to identify their roles in forming a team but requires prior knowledge of all others' resources. This restricts the applicability to real systems. The contribution of this paper is twofold. First, we extended our original method by adding a resource estimation method. Second, we further improved the first extension for large scale multi-agent systems by introducing purviews, which are a relatively small set of agents that are potential members of the teams, for practical computational time and required memory size. We experimentally evaluated our first method by comparing it with the previous method and the task allocation using the contract net protocol (CNP). Then, after increasing the number of agents, we evaluated our second extended method and investigated how the number of agents and the size of the purview affected the overall performances. Results showed that the learning speed was faster in the proposed method so it outperformed other methods in a practical sense even though it did not require prior knowledge of resources in other agents in busy, large-scale, multi-agent systems.

Effects of Complex Network Characters on the Coordination Control of Large-Scale Multi-Agent System

随着分布式多智能体系统应用领域和系统规模的不断扩大,网络特性已成为影响系统性能的一个重要因素.通过研究和分析复杂网络特性对大规模分布式多智能体系统协同控制的影响,对多智能体系统性能的影响做出系统性分析,同时为提出大规模多智能体组织结构的优化算法提供依据.主要针对随机网络、小世界网络、网格网络和无尺度网络这4 种典型复杂网络特性,从理论和仿真两方面进行分析.在理论方面,通过基于马尔可夫链的信息传输过程在不同网络结构下的建模,对比分析了信息无偏随机游走模型和智能决策模型下的传输效率.在仿真建模中,主要从智能体间信息传输效率、不同应用领域中集成协同控制效率、对网络故障恢复的影响这3 个典型的多智能体系统协同控制应用对比分析复杂网络特性对系统性能的影响.研究结果表明,复杂网络特性如小世界和无尺度特性可以在相同的控制策略下形成明显的性能差异,如果设计合理的控制算法,复杂网络结构将有助于多智能体系统性能的提升.

Towards a Formal Conceptualization of Organizational Design Techniques for Large Scale Multi Agent Systems

Computing in the Internet of Things is increasingly pervasive, with everyday items including clothes, smart-phones, cars and various household appliances gaining sophisticated communication and computing capacities. It seems to be just a matter of time before devices have to collaborate and compete mutually as well as with their users, in order to provide better services to mankind. These embedded computers are increasingly autonomous and connected, and can thus be modeled as agents within multi-agent systems (MAS). Only 30 years ago it was science fiction that over a billion of people will exchange billions of e-mails on a daily basis. Today a scenario of millions of collaborating agents embedded in gadgets and appliances, across various networks may also sound futuristic. However given the current rate of development in electronics, we will soon have to manage large scale MAS (LSMAS) where millions of agents exist, collaborate and compete with each other. While a recent study shows that there are at least 50 organizational structure, superstructure and architecture types employed in modern organizations, there is a lack of research that would apply organizational design methods to organizations composed entirely of agents and agent systems in order to achieve alignment of organizational structure, processes, and reward system with the goals and strategy of the organization. Our research is therefore aimed towards enriching formal design methods for the development of LSMAS to foster the development of self-organizing and adaptable networks of devices that will contribute towards a sustainable development of the information society. In this work-in-progress study we apply a collaborative semantic wiki approach towards formalizing organizational design techniques in order to provide a foundation for future studies of automated LSMAS development.