Multi-agent Community Research Articles

Simulation of Epidemic Dynamics Using a Multi-Agent Model: Analysis of Social Distancing Strategies and Their Impacts on Public Health and Economy

Infectious disease epidemics have played a crucial role in shaping public health responses, particularly in global health crises. This study emerges as part of the efforts to prepare effective responses to potential future pandemics, leveraging lessons learned during the COVID-19 crisis. The research uses an adapted compartmental epidemiological model and a synthetic multi-agent community to investigate how social variables influence epidemic forecasts in socioeconomically vulnerable regions. Focusing on the simulation of epidemic dynamics in the socio-economically disadvantaged neighbourhood of Ilha Joana Bezerra in Recife, this study examines the impacts of social distancing strategies and other control measures, such as face masks and moderate social isolation. Through the adapted SEPAI3R3O model, which includes compartments for pre-symptomatic and asymptomatic states, this study provides a detailed analysis of disease dynamics in contexts characterised by high social vulnerability. The results underscore the importance of public health policies adapted to socio-economic factors, emphasising the need for continuous preparedness to manage future epidemic threats in vulnerable communities effectively.

Is this a violation? Learning and understanding norm violations in online communities

Using norms to guide and coordinate interactions has gained tremendous attention in the multi-agent community. However, new challenges arise as the interest moves towards dynamic socio-technical systems, where human and software agents interact, and interactions are required to adapt to human's changing needs. For instance, different agents (human or software) might not have the same understanding of what it means to violate a norm (e.g., what characterizes hate speech), or their understanding of a norm might change over time (e.g., what constitutes an acceptable response time). The challenge is to address these issues by learning the meaning of a norm violation from limited interaction data. For this, we use batch and incremental learning to train an ensemble of classifiers. Ensemble learning and data-sampling handle the imbalanced class distribution of the interaction stream. At the same time, the training approaches use different strategies to ensure that the ensemble models reflect the latest community view on the meaning of norm violation. Batch learning uses weight assignment, while incremental learning continuously updates the ensemble models as community members interact. Here, we extend our previous work by creating a different balance strategy for online learning and integrating interpretability to understand norm violations. Additionally, we evaluate the proposed approaches in the context of Wikipedia article edits, where interactions revolve around editing articles, and the norm in question is prohibiting vandalism. Lastly, we conduct ablation studies to compare the ensemble's performance against a single model approach and to examine the behavior of two data sampling techniques. Results indicate that the different machine learning frameworks can learn the meaning of a norm violation in a setting with data imbalance and concept drift, although with significant differences.

A multi-scenario approach to continuously learn and understand norm violations

Using norms to guide and coordinate interactions has gained tremendous attention in the multiagent community. However, new challenges arise as the interest moves towards dynamic socio-technical systems, where human and software agents interact, and interactions are required to adapt to changing human needs. For instance, different agents (human or software) might not have the same understanding of what it means to violate a norm (e.g., what characterizes hate speech), or their understanding of a norm might change over time (e.g., what constitutes an acceptable response time). The challenge is to address these issues by learning to detect norm violations from the limited interaction data and to explain the reasons for such violations. To do that, we propose a framework that combines Machine Learning (ML) models and incremental learning techniques. Our proposal is equipped to solve tasks in both tabular and text classification scenarios. Incremental learning is used to continuously update the base ML models as interactions unfold, ensemble learning is used to handle the imbalance class distribution of the interaction stream, Pre-trained Language Model (PLM) is used to learn from text sentences, and Integrated Gradients (IG) is the interpretability algorithm. We evaluate the proposed approach in the use case of Wikipedia article edits, where interactions revolve around editing articles, and the norm in question is prohibiting vandalism. Results show that the proposed framework can learn to detect norm violation in a setting with data imbalance and concept drift.

A hybrid tree-based algorithm to solve asymmetric distributed constraint optimization problems

Asymmetric distributed constraint optimization problems (ADCOPs) have emerged as an important formalism in multi-agent community due to their ability to capture personal preferences. However, the existing search-based complete algorithms for ADCOPs only exploit local knowledge to calculate lower bounds, which leads to inefficient pruning and prohibits them from solving large scale problems. On the other hand, inference-based complete algorithms (e.g., DPOP) for distributed constraint optimization problems are able to aggregate the global cost promptly but cannot be directly applied into ADCOPs due to a privacy concern. Thus, in this paper, we investigate the possibility of combining inference and search to effectively solve ADCOPs at an acceptable loss of privacy. Specifically, we propose a hybrid complete ADCOP algorithm called PT-ISABB which uses a tailored inference algorithm to provide tight lower bounds and upper bounds, and a tree-based complete search algorithm to guarantee the optimality. Furthermore, we introduce two suboptimal variants of PT-ISABB based on bounded-error approximation mechanisms to enable trade-off between theoretically guaranteed solutions and coordination overheads. We prove the correctness of PT-ISABB and its suboptimal variants. Finally, the experimental results demonstrate that PT-ISABB exhibits great superiorities over other state-of-the-art search-based complete algorithms and its suboptimal variants can quickly find a solution within the user-specified bounded-error.

A survey and critique of multiagent deep reinforcement learning

Deep reinforcement learning (RL) has achieved outstanding results in recent years. This has led to a dramatic increase in the number of applications and methods. Recent works have explored learning beyond single-agent scenarios and have considered multiagent learning (MAL) scenarios. Initial results report successes in complex multiagent domains, although there are several challenges to be addressed. The primary goal of this article is to provide a clear overview of current multiagent deep reinforcement learning (MDRL) literature. Additionally, we complement the overview with a broader analysis: (i) we revisit previous key components, originally presented in MAL and RL, and highlight how they have been adapted to multiagent deep reinforcement learning settings. (ii) We provide general guidelines to new practitioners in the area: describing lessons learned from MDRL works, pointing to recent benchmarks, and outlining open avenues of research. (iii) We take a more critical tone raising practical challenges of MDRL (e.g., implementation and computational demands). We expect this article will help unify and motivate future research to take advantage of the abundant literature that exists (e.g., RL and MAL) in a joint effort to promote fruitful research in the multiagent community.

The complexity of decision problems about equilibria in two-player Boolean games

Boolean games allow us to succinctly represent strategic games with binary payoffs in the case where the players' preferences have a structure readily expressible in propositional logic. Since their introduction, the computational aspects of Boolean games have been of interest to the multiagent community, but so far the focus has been exclusively on pure strategy equilibria. In this paper we consider the complexity of problems involving mixed strategy equilibria, such as the existence of an equilibrium satisfying a given payoff constraint. The results are obtained by the observation that a mixed strategy can hold enough information to encode the computation history of an exponential time Turing machine.

Simulating cardinal preferences in Boolean games: A proof technique

Boolean games are a succinct representation of strategic games with a logical flavour. While they have proved to be a popular formalism in the multiagent community, a commonly cited shortcoming is their inability to express richer utilities than success or failure. In addition to being a modelling limitation, this parsimony of preference has made proving complexity bounds difficult. We address the second of these issues by demonstrating how cardinal utilities can be simulated via expected utility. This allows us to prove that RationalNash and IrrationalNash are NEXP-hard, and to translate hardness results for IsNash and DValue into the Boolean games framework.

Secure information sharing in social agent interactions using information flow analysis

When we wish to coordinate complex, cooperative tasks in open multi-agent systems, where each agent has autonomy and the agents have not been designed to work together, we need a way for the agents themselves to determine the social norms that govern collective behaviour. An effective way to define social norms for agent communication is through the use of interaction models such as those expressed in the Lightweight Coordination Calculus (LCC), a compact executable specification language based on logic programming and pi-calculus. Open multi-agent systems have experienced growing popularity in the multi-agent community and gain importance as large scale distributed systems become more widespread. A major practical limitation to such systems is security, because the very openness of such systems opens the doors to adversaries to exploit vulnerabilities introduced through acceptance of social norms.This paper addresses a key vulnerability of security of open multi-agent systems governed by formal models of social norms (as exemplified by LCC). A fundamental limitation of conventional security mechanisms (e.g. access control and encryption) is the inability to prevent information from being propagated. Focusing on information leakage in choreography systems using LCC, we suggest a framework to detect insecure information flows. A novel security-typed LCC language is proposed to prevent information leakage.Both static (design-time) and dynamic (run-time) security type checking are employed to guarantee no information leakage can occur in annotated agent interaction models. The proposed security type system is discussed and then formally evaluated by proving its properties. Two disadvantages of the pure dynamic analysis are its late detection and its inability to detect implicit information flows. We overcome these issues by performing static analysis. The proposed security type system supports non-interference, i.e. high-security input to the program never affect low-security output. However, it disregards information leaks due to the termination of the program.

Trust-Based Situation Awareness: Comparative Analysis of Agent-Based and Population-Based Modeling

In recent years, to comprehend and analyze complex systems, multiagent systems modeling and simulation are being widely used across various disciplines. Two major approaches used for multiagent systems modeling and simulation are agent-based modeling (ABM) and population-based modeling (PBM). In multiagent community, it is a silent assumption that both approaches represent similar dynamics for large population size. One of the recent studies from literature has reported similar results for a model of situation awareness spread in multiagent systems. Trust is a significant factor that affects agents’ communication, and consequently it controls spread of situation awareness among agents in a multiagent system. Hence, current work firstly extends the reported model of situation awareness spread from literature, to incorporate interagent trust for both ABM and PBM. Later, these extended models are used for comparative evaluation of both approaches. Various simulation experiments for different population sizes (small and large) as well as population types (homogenous and heterogeneous) are conducted and analyzed. Results of these experiments show that for large and homogeneous population, ABM approximates behavior of PBM, but for even slightly heterogeneous population, these approaches do not produce similar results irrespective of population sizes. Thus, the current study reports that, under some conditions, ABM and PBM produce similar results for trust-based situation awareness spread in multiagent systems, but this assumption does not hold true at large.

Development and implementation of a technique for norms-adaptable agents in open multi-agent communities

In open normative multi-agent communities, an agent is not usually and explicitly given the norms of the host agents. Thus, when it is not able to adapt the communities’s norms, it is totally deprived of accessing resources and services from the host. Such circumstance severely affects its performance resulting in failure to achieve its goal. Consequently, this study attempts to overcome this deficiency by proposing a technique that enables an agent to detect the host’s potential norms via self-enforcement and update its norms even in the absence of sanctions from a third-party. The authors called this technique as the potential norms detection technique (PNDT). The PNDT consists of five components: Agent’s belief base; observation process; potential norms mining algorithm (PNMA); verification process; and updating process. The authors demonstrate the operation of the PNMA algorithm by testing it on a typical scenario and analyzing the results on several perspectives. The tests’ results show that the PNDT performs satisfactorily albeit the success rate depends on the environment variables settings.

A CASE FOR DOMAIN-INDEPENDENT DETERMINISTIC MULTIAGENT

The notion of planning using multiple agents has been around since the very beginning of planning itself. It has been approached from various viewpoints especially in the multiagent systems community. Recently, domain-independent multiagent planning has gained more attention also in the automated planning community. In this paper, we shortly present the current state of the art, question some aspects of the research field and discuss the rising challenges.

Leader-following consensus of linear multi-agent systems with randomly occurring nonlinearities and uncertainties and stochastic disturbances

For the multi-agent community, one of the key challenges is the agents operating in open environments, in which stochastic noises affect the dynamics of agents. This paper deals with the leader-following consensus problem for a class of linear multi-agent systems model with randomly occurring nonlinearities and uncertainties, and stochastic disturbances. The communication topology is assumed to be undirected and fixed. The stochastic Brownian motions are used to describe the source of extrinsic disturbances. The randomly occurring nonlinearities are introduced to describe the nonlinear intrinsic dynamics occurring in a probabilistic way. The randomly occurring uncertainties are adopted to reflect more realistic dynamical behaviors of the agent systems that are caused by noisy environment. Sufficient conditions are derived to make all follower agents asymptotically reach the state of leader in the mean square sense. Simulation results illustrate the theoretical results.

Beyond Reinforcement Learning and Local View in Multiagent Systems

Learning is an important component of an agent’s decision making process. Despite many messages in contrary, the fact is that, currently, in the multiagent community it is mostly likely that learning means reinforcement learning. Given this background, this paper has two aims: to revisit the “old days” motivations for multiagent learning, and to describe some of the work addressing the frontiers of multiagent systems and machine learning. The intention of the latter task is to try to motivate people to address the issues that are involved in the application of techniques from multiagent systems in machine learning and vice-versa.

Automatized Decision Making for Autonomous Agents

Utility theory and the principle of maximising the expected utility have, within the multi-agent community, had a great influence on multi-agent based decision. Even though this principle is often useful when evaluating a decision situation it is virtually impossible, except in very artificial situations, to use the more basic decision rules with its unrealistically strong requirements for the input data, and other candidate methods must be considered instead. This article provides an overview and brings attention to some of the possibilities to utilize more elaborated decision methods, while still keeping the computational issues at a tractable level.

Management of Demand and Routing in Autonomous Personal Transportation

Autonomous vehicles are becoming a reality. In particular, podcars were already launched in pilot projects. The wide use of such vehicles for personal transportation in urban areas is then just a matter of time and poses several research challenges and technical issues to the artificial intelligence and multiagent communities. This article focuses on issues related to the processing of demands for these vehicles and on routing them in an efficient way. An agent-based approach is proposed, which considers some variants for the processing of demands and for the routing. First, a centralized version is discussed. Then, extensions deal with decentralized routing, with enroute planning, intervehicle communication, and with a market-based approach where the manager of the service runs an auction to determine which customer to serve. All these variants aim at testing alternatives in order to shed light onto questions such as how the podcar service is to be provided and how trips have to be planned, as well as address challenges related to the management. Results show that, although this service can be provided by a central authority, communication, as well as reliability and fault-tolerance, should not be neglected.

Solving efficiently Decentralized MDPs with temporal and resource constraints

Optimizing the operation of cooperative multi-agent systems that can deal with large and realistic problems has become an important focal area of research in the multi-agent community. In this paper, we first present a new model, the OC-DEC-MDP (Opportunity Cost Decentralized Markov Decision Process), that allows us to represent large multi-agent decision problems with temporal and precedence constraints. Then, we propose polynomial algorithms to efficiently solve problems formalized by OC-DEC-MDPs. The problems we deal with consist of a set of agents that have to execute a set of tasks in a cooperative way. The agents cannot communicate during task execution and they must respect resource and temporal constraints. Our approach is based on Decentralized Markov Decision Processes (DEC-MDPs) and uses the concept of opportunity cost borrowed from economics to obtain approximate control policies. Experimental results show that our approach produces good quality solutions for complex problems which are out of reach of existing approaches.

Multi agent Community to support information processing in Wireless Sensor Network applications

One of the key challenges of applications concerning with Wireless Sensor Network (WSN) is finding suitable and timely information in the dynamic environment. To address these issues, this paper proposes to employ multi agent community by combined used of intelligent sensors and middle agent architecture. The idea is to use intelligent sensor as a service provider agent that announces for the active events and middle agent as a directory facilitator that assists in locating the service providers with the information requesters in the dynamic WSN environment. Simulation results have approved that proposed system enhances the information searching time, reduces the service discovery cost and communication cost.

Collective agency, direct action and dynamic operators

We review the stit semantic framework and the main stit operators that have been proposed by Belnap, Perloff and Horty (among others), a theory that has recently attracted the attention of the multiagent community. We discuss the problem of how to model the notion of collective agency, both in the sense of a joint action of a group of agents and as the agency of collective entities, like organisations. We show how we can define the direct and immediate effects of agents’ actions, both in the achievement sense and in the deliberative sense, without any assumptions on the nature of time and considering the particular case of a discrete time. Finally, we also show how we can define a kind of dynamic logic operator that allows us to express what would obtain, if a direct “sees to it” action is performed, seeing it as a particular case of formulae of the form [a: φ]Ψ, that have the following informal meaning: “if agent a selects an action that leads to the truth of φ, after such action is performed, Ψ is the case” (or, simply, “after a choosing φ, Ψ is the case”).

Evaluation of a selective distributed discovery strategy in a fully decentralized biologically inspired environment

The increased demand and complexity of services operating within open distributed environments has emphasized the need for systems that are adaptive, self-organizing and more robust. In order to address these issues some agent oriented approaches have adopted ideas from natural systems as possible solutions. The introduction of biological properties, especially birth and death of agents as expected events, generates an extremely dynamic environment where it is difficult to maintain the overall connectivity of the overlay network and facilitate efficient discovery processes. In this paper we evaluate the performance of a selective discovery mechanism in a distributed bio-inspired multi-agent community through a simulation study. The primary focus of the study is on the impacts which death and (sexual/asexual) reproduction events have on the effectiveness of the discovery process in different overlay networks.

An integrated role-based approach for modeling, designing and implementing multi-agent systems

Abstract To facilitate the development of multi-agent systems and improve the reusability, robustness and feasibility of these systems, we have developed a role-based agent development framework (RADE). In this paper, we present an integrated approach for modeling, designing and implementing multi-agent systems using RADE. We describe the design of agents and motivations within such framework. We introduce a practical approach for modeling agent’s motivation and specifying agent’s goals, where a role-agent mapping mechanism is developed based on this design. Dynamic task allocation is achieved through the creation of role instances and the mapping from role instances to agents. We also introduce the RTÆMS language based on the extension of TÆMS to model the plan tree for each goal. This representation enables the reuse of general planning/scheduling and collaboration/cooperation mechanisms developed in multi-agent system research community. We have developed an automatic agent generation interface and also implemented a simple demo system in health care domain.