Collaborative Multi-agent Systems Research Articles

Distributed Control of Collaborative Multi-Agent Systems using Interpreted Petri Nets through Indirect Interactions

This work introduces a framework to describe and control collaborative systems in a distributed manner. Within this framework, both a set of agents and a specification to be accomplished are modelled using Interpreted Petri nets. The agents’ output states are partially known, and the specification describes a subset of relevant states to be reached by the agents without explicitly detailing which agents are involved in achieving such specification or the precise states that each agent must reach individually. This framework enables agents to collaboratively discover how to fulfill a shared, high-level specification, even in the absence of detailed instructions. The approach promises to enhance scalability, flexibility, and efficiency in collaborative settings. For illustrative purposes, a simplified collaborative assembly system is showcased to demonstrate the proposed framework.

A comprehensive soft security model for Cognitive Internet of Things

The Cognitive Internet of Things (CIoT) is a rapidly evolving field that combines artificial intelligence (AI) with the Internet of Things (IoT) ecosystem. By augmenting IoT with AI, objects can sense, perceive, think, and make decisions independently with minimal initial knowledge. These cognitive objects form a society and work toward achieving their goals through cooperation. However, trust is a significant security challenge in such a society. To address this challenge, this paper proposes a soft security approach to model trust in CIoT using Collaborative Multi-Agent Systems (CMAS). Our model introduces an interactive, autonomous, and self-taught agent that can move toward a secure situation without needing a supervisor. We use a state machine to model an insecure ecosystem where agents can behave honestly, adversarially, neutrally, or hypocritically. We define inner trust as a combination of direct and indirect experiences and observations and global trust as the weighted average of inner trust and recommendations. Additionally, we employ the reinforcement learning algorithm to train agents. To evaluate our model, we developed a proprietary tool called ASSOCIATE and assessed our model using data from the Santander Smart City dataset and simulator-generated data. Our evaluations encompass three aspects: the success rate in achieving the goal, the recognition of agents’ behavior, and the quality of the diagnosis.

Conversational AI for multi-agent communication in Natural Language

Research at the Interaction Lab focuses on human-agent communication using conversational Natural Language. The ultimate goal is to create systems where humans and AI agents (including embodied robots) can spontaneously form teams and coordinate shared tasks through the use of Natural Language conversation as a universal communication interface. This paper first introduces machine learning approaches to problems in conversational AI in general, where computational agents must coordinate with humans to solve tasks using conversational Natural Language. It also covers some of the practical systems developed in the Interaction Lab, ranging from speech interfaces on smart speakers to embodied robots interacting using visually grounded language. In several cases communication between multiple agents is addressed. The paper surveys the central research problems addressed here, the approaches developed, and our main results. Some key open research questions and directions are then discussed, leading towards a future vision of conversational, collaborative multi-agent systems.

Mean Field Behavior of Collaborative Multiagent Foragers

Collaborative multi-agent robotic systems where agents coordinate by modifying a shared environment often result in undesired dynamical couplings that complicate the analysis and experiments when solving a specific problem or task. Simultaneously, biologically-inspired robotics rely on simplifying agents and increasing their number to obtain more efficient solutions to such problems, drawing similarities with natural processes. In this work we focus on the problem of a biologically-inspired multi-agent system solving collaborative foraging. We show how mean field techniques can be used to re-formulate such a stochastic multi-agent problem into a deterministic autonomous system. This de-couples agent dynamics, enabling the computation of limit behaviours and the analysis of optimality guarantees. Furthermore, we analyse how having finite number of agents affects the performance when compared to the mean field limit and we discuss the implications of such limit approximations in this multi-agent system, which have impact on more general collaborative stochastic problems.

Reward-Reinforced Generative Adversarial Networks for Multi-Agent Systems

Multi-agent systems deliver highly resilient and adaptable solutions for common problems in telecommunications, aerospace, and industrial robotics. However, achieving an optimal global goal remains a persistent obstacle for collaborative multi-agent systems, where learning affects the behaviour of more than one agent. A number of nonlinear function approximation methods have been proposed for solving the Bellman equation, which describe a recursive format of an optimal policy. However, how to leverage the value distribution based on reinforcement learning, and how to improve the efficiency and efficacy of such systems remain a challenge. In this work, we developed a reward-reinforced generative adversarial network to represent the distribution of the value function, replacing the approximation of Bellman updates. We demonstrated our method is resilient and outperforms other conventional reinforcement learning methods. This method is also applied to a practical case study: maximising the number of user connections to autonomous airborne base stations in a mobile communication network. Our method maximises the data likelihood using a cost function under which agents have optimal learned behaviours. This reward-reinforced generative adversarial network can be used as a generic framework for multi-agent learning at the system level.

GasMASk Annotation-based Code Generator as an Embedded Domain-Specific Language in Collaborative Multi-Agent Systems

Multi-agent systems have evolved with their application framework, analysis approaches and design complexities over the past few decades. In order for programming software agents, you need to handle analysis, design and implementation together; furthermore, privacy and trust should be integrated externally into the agent-oriented applications. Current frameworks do not consider privacy, trust, and data integrity in agent applications. In this study, we propose a domain-specific language that can be used for agent behaviors that consist of roles, organizational entity, goals through annotation processing with templates. Smart contracts can be generated to decrease the time for deployment and development stages. Templates and annotations are popular techniques to reduce boilerplate codebases from agent-oriented programming. These techniques can also be used for model-driven software engineering. This study will take the software agent development as a whole with analysis, design, and development with embedded domain-specific language development in terms of smart contract applications. Furthermore, we would like to refer to methodology, results of the research, and case study to enlighten readers in a better way. Finally, we summarize findings and highlight the main research points by inferencing in the conclusion section.

Multicriterial analyses of Pareto-efficiency for collaborative multi-agent systems using genetic algorithm with variations

The paper is devoted to an emerging trend in modern automation of production - collaborative multi-agent systems – characterized by the simultaneous use of the labor of people and robots in a single system in the framework of solving the target problem. The need in collaborative multi-agent systems is due to the fact that the industrial use of fully automated multicomponent systems is limited by the financial and infrastructural unavailability of various industries to switch to completely unmanned technologies. The article discusses the issue of the effectiveness of applying this approach as an alternative to full robotization of production. A multi-criteria assessment of the effectiveness of collaborative multi-agent systems as optimization knapsack problem is formulated. A multicriteria genetic algorithm based on the principle of small variations of the basic solution is presented for the solution of the stated problem. The resulting Pareto set is constructed according to the rank of possible solutions.

Modelling resilient collaborative multi-agent systems

Multi-agent systems constitute a wide class of decentralised systems. Their functions are usually carried out by collaborative activities of agents. To ensure resilience of multi-agent systems, we should endow them with a capability to dynamically reconfigure. Usually, as a result of reconfiguration, the existing relationships between agents are changed and new collaborations are established. This is a complex and error-prone process, which can be facilitated by the use of formal reasoning and automated verification. In this paper, we propose a generic resilience-explicit formalisation of the main concepts of multi-agent systems. Based on it, we introduce corresponding specification and refinement patterns in Event-B. Our patterns facilitate modelling behaviour of resilient multi-agent systems in a rigorous systematic way and verification of their properties. We demonstrate the application of the proposed approach by a case study—a smart warehouse system.

Collaborative Multiagent Systems — an Alternative to Full Automation of Production

The paper proposes a new approach as an alternative to full automation of processes that meets current economic trends — collaborative multi-agent systems. In this concept, people and robots are considered as agents in a single sensory-information field, who perform tasks to achieve the goals of the collaborative multi-agent system. The urgency of collaborative multi-agent systems results from the fact that the industrial use of fully automated multi-component systems is limited by the financial and infrastructural unavailability of various industries to switch to completely unmanned technologies. The proposed approach combines the latest, but remaining quite recouped, technological advances along with highly skilled human labor. The use of collaborative multi-agent systems will be economically justified in the manufacture of products in small batches, in the conditions of rapid change of product lines, as well as the presence of staff shortages. The article shows that such an approach can significantly reduce automation costs, while ensuring that the specified production indicators are met. This approach allows taking a fresh look at a human, considering him and a robot as equal partners within a collaborative system. The basic concepts and distinctive characteristics of collaborative multi-agent systems are formulated and presented in the work, justifications for their use are given. Creating a new class of collaborative multi-agent systems requires solving a number of problems associated with the interaction of man and robot. The article considers issues related to the work of a person within a collaborative system, with a rational separation of human functions and an automated production system, in accordance with the necessary level of collaboration. The inclusion of a person with his psychoemotional and physical characteristics as an equivalent agent of a multi-agent system causes difficulties in formalizing collaborative multi-agent systems associated with the need to take these features into account and create a sensory-information system. The inclusion of a person with his psychoemotional and physical characteristics as an equivalent agent of a multi-agent system causes difficulties in formalizing collaborative multi-agent systems associated with the need to take these features into account and create a sensory-information system. The paper discusses ways to formalize a collaborative multi-agent system and management approaches.

Engineering Kindness

The author provides first steps toward building a software agent/robot with compassionate intelligence. She approaches this goal with an example software agent, EM-2. She also gives a generalized software requirements guide for anyone wishing to pursue other means of building compassionate intelligence into an AI system. The purpose of EM-2 is not to build an agent with a state of mind that mimics empathy or consciousness, but rather to create practical applications of AI systems with knowledge and reasoning methods that positively take into account the feelings and state of self and others during decision making, action, or problem solving. To program EM-2 the author re-purposes code and architectural ideas from collaborative multi-agent systems and affective common sense reasoning with new concepts and philosophies from the human arts and sciences relating to compassion. EM-2 has predicates and an agent architecture based on a meta-cognition mental process that was used on India's worst prisoners to cultivate compassion for others, Vipassana or mindfulness. She describes and presents code snippets for common sense based affective inference and the I-TMS, an Irrational Truth Maintenance System, that maintains consistency in agent memory as feelings change over time, and provides a machine theoretic description of the consistency issues of combining affect and logic. The author summarizes the growing body of new biological, cognitive and immune discoveries about compassion and the consequences of these discoveries for programmers working with human-level AI and hybrid human-robot systems.

Distributed parametric and nonparametric regression with on-line performance bounds computation

In this paper we focus on collaborative multi-agent systems, where agents are distributed over a region of interest and collaborate to achieve a common estimation goal. In particular, we introduce two consensus-based distributed linear estimators. The first one is designed for a Bayesian scenario, where an unknown common finite-dimensional parameter vector has to be reconstructed, while the second one regards the nonparametric reconstruction of an unknown function sampled at different locations by the sensors. Both of the algorithms are characterized in terms of the trade-off between estimation performance, communication, computation and memory complexity. In the finite-dimensional setting, we derive mild sufficient conditions which ensure that a distributed estimator performs better than the local optimal ones in terms of estimation error variance. In the nonparametric setting, we introduce an on-line algorithm that allows the agents to simultaneously compute the function estimate with small computational, communication and data storage efforts, as well as to quantify its distance from the centralized estimate given by a Regularization Network, one of the most powerful regularized kernel methods. These results are obtained by deriving bounds on the estimation error that provide insights on how the uncertainty inherent in a sensor network, such as imperfect knowledge on the number of agents and the measurement models used by the sensors, can degrade the performance of the estimation process. Numerical experiments are included to support the theoretical findings.

A unified framework for reinforcement learning, co-learning and meta-learning how to coordinate in collaborative multi-agent systems

Coordination among multiple autonomous, distributed cognitive agents is one of the most challenging and ubiquitous problems in Distributed AI and its applications in general, and in collaborative multi-agent systems in particular. A particularly prominent problem in multi-agent coordination is that of group, team or coalition formation. A considerable majority of the approaches to this problem found in the literature assume fixed interactions among autonomous agents involved in the coalition formation process. Moreover, most of the prior research where agents are actually able to learn and adapt based on their past interactions mainly focuses on reinforcement learning techniques at the individual agent level. We argue that, in many important applications and contexts, complex large-scale collaborative multi-agent systems need to be able to learn and adapt at multiple organization, hierarchical and logical levels. In particular, the agents need to be able to learn both at the level of individual agents and at the system or agent ensemble levels, and then to integrate these different sources of learned knowledge and behavior, in order to be effective at solving complex tasks in typical dynamic, partially observable and noisy multi-agent environments. In this paper, we describe a conceptual framework for addressing the problem of learning how to coordinate effectively at three qualitatively distinct levels — those of (i) individual agents, (ii) small groups of agents, and (iii) very large agent ensembles (or alternatively, depending on the nature of a multi-agent system, at the system or central control level). We briefly illustrate the applicability and usefulness of the proposed conceptual framework with an example of how it would apply to an important practical coordination problem, namely that of distributed coordination of a large ensemble of unmanned vehicles on a complex multi-task mission.

Graphical Model Inference in Optimal Control of Stochastic Multi-Agent Systems

In this article we consider the issue of optimal control in collaborative multi-agent systems with stochastic dynamics. The agents have a joint task in which they have to reach a number of target states. The dynamics of the agents contains additive control and additive noise, and the autonomous part factorizes over the agents. Full observation of the global state is assumed. The goal is to minimize the accumulated joint cost, which consists of integrated instantaneous costs and a joint end cost. The joint end cost expresses the joint task of the agents. The instantaneous costs are quadratic in the control and factorize over the agents. The optimal control is given as a weighted linear combination of single-agent to single-target controls. The single-agent to single-target controls are expressed in terms of diffusion processes. These controls, when not closed form expressions, are formulated in terms of path integrals, which are calculated approximately by Metropolis-Hastings sampling. The weights in the control are interpreted as marginals of a joint distribution over agent to target assignments. The structure of the latter is represented by a graphical model, and the marginals are obtained by graphical model inference. Exact inference of the graphical model will break down in large systems, and so approximate inference methods are needed. We use naive mean field approximation and belief propagation to approximate the optimal control in systems with linear dynamics. We compare the approximate inference methods with the exact solution, and we show that they can accurately compute the optimal control. Finally, we demonstrate the control method in multi-agent systems with nonlinear dynamics consisting of up to 80 agents that have to reach an equal number of target states.

Graphical model inference in optimal control of stochastic multi-agent systems

In this article we consider the issue of optimal control in collaborative multi-agent systems with stochastic dynamics. The agents have a joint task in which they have to reach a number of target s...

The dynamics of intention in collaborative activity

An adequate formulation of collective intentionality is crucial for understanding group activity and for modeling the mental state of participants in such activities. Although work on collective intentionality in philosophy, artificial intelligence, and cognitive science has many points of agreement, several key issues remain under debate. This paper argues that the dynamics of intention – in particular, the inter-related processes of plan-related group decision making and intention updating – play crucial roles in an explanation of collective intentionality. Furthermore, it is in these dynamic aspects that coordinated group activity differs most from individual activity. The paper specifies a model of the dynamics of agent intentions in the context of collaborative activity. Its integrated treatment of group decision making and coordinated updating of group-related intentions fills an important gap in prior accounts of collective intentionality, thus helping to resolve a long-standing debate about the nature of intentions in group activity. The paper also defines an architecture for collaboration-capable computer agents that satisfies the constraints of the model and is a natural extension of the standard architecture for resource-bounded agents operating as individuals. The new architecture is both more principled and more complete than prior architectures for collaborative multi-agent systems.

The application of multi-agent systems for STEP-NC computer aided process planning of prismatic components

For many years, manufacturing firms have been seeking more efficient ways of manufacturing components with CNC machines. The emerging standards ISO 14649 and ISO 10303 (AP238) present an opportunity to revolutionize the way CNC machines are traditionally programmed. These standards better known as STEP-NC replace the traditional tool movement description languages with hierarchical data structures that allow a new breed of CNC to store part geometry together with the working steps of the operations required to manufacture the part. STEP-NC provides the ability to store and utilise high level and detailed information from the CAD system to the intelligent STEP compliant CNC controller. With the advent of STEP-NC, computer aided process planning has become a critical link in the CAx process chain with the major requirement to generate interoperable process plans. The authors therefore believe it is necessary to redefine CAPP to reflect the change from the traditional tool movement based programming to STEP-NC based programming. This paper examines the application of distributed artificial intelligence methods, namely collaborative multi-agent systems in designing an object-oriented process planning system for prismatic components in a STEP-NC compliant environment. The specification and design of a prototype system entitled the Multi-Agent System for Computer Aided Process Planning (MASCAPP) is outlined. Two test components have been designed, process planned, simulated on the machine controller and finally machined, to demonstrate the capabilities of the system and illustrate the activities required to implement STEP compliant manufacturing.

A Model of Adaptation in Collaborative Multi-Agent Systems

Adaptation is an essential requirement for autonomous agent systems functioning in uncertain dynamic environments. Adaptation allows agents to change their behavior in order to improve the overall sys tem performance. We describe a general mechanism for adaptation in multi-agent systems in which agents modify their behavior in response to changes in the environment or actions of other agents. The agents estimate the global state of the system from local observations and adjust their actions accordingly. We derive a mathematical model that describes the collective behavior of such adaptive systems. The model, consisting of coupled rate equations, governs how the collective behavior changes in time. We apply the model to study collaboration in a group of mobile robots. The system we study is an adaptive version of the collaborative stick pulling in a group of robots examined in detail in earlier works (Ijspeert, Martinoli, Billard, & Gambardela, 2001; Lerman, Galstyan, Martinoli, & Ijspeert, 2001). In adaptive stick pulling, robots estimate the number of robots and sticks in the system and adjust their individual behavior so as to improve collective performance. We solve the mathematical model and show that adaptation improves collective performance for all parameter values.

Solving Transition Independent Decentralized Markov Decision Processes

Formal treatment of collaborative multi-agent systems has been lagging behind the rapid progress in sequential decision making by individual agents. Recent work in the area of decentralized Markov Decision Processes (MDPs) has contributed to closing this gap, but the computational complexity of these models remains a serious obstacle. To overcome this complexity barrier, we identify a specific class of decentralized MDPs in which the agents' transitions are independent. The class consists of independent collaborating agents that are tied together through a structured global reward function that depends on all of their histories of states and actions. We present a novel algorithm for solving this class of problems and examine its properties, both as an optimal algorithm and as an anytime algorithm. To our best knowledge, this is the first algorithm to optimally solve a non-trivial subclass of decentralized MDPs. It lays the foundation for further work in this area on both exact and approximate algorithms.

Solving transition independent decentralized Markov decision processes

Formal treatment of collaborative multi-agent systems has been lagging behind the rapid progress in sequential decision making by individual agents. Recent work in the area of decentralized Markov Decision Processes (MDPs) has contributed to closing this gap, but the computational complexity of these models remains a serious obstacle. To overcome this complexity barrier, we identify a specific class of decentralized MDPs in which the agents' transitions are independent. The class consists of independent collaborating agents that are tied together through a structured global reward function that depends on all of their histories of states and actions. We present a novel algorithm for solving this class of problems and examine its properties, both as an optimal algorithm and as an anytime algorithm. To the best of our knowledge, this is the first algorithm to optimally solve a non-trivial subclass of decentralized MDPs. It lays the foundation for further work in this area on both exact and approximate algorithms.