Multi-agent Case Research Articles

Bearing-Based Formation Tracking Control of Nonholonomic Mobile Agents With a Persistently Exciting Leader

This paper investigates the leader-following formation control problem of nonholonomic agents with a persistently exciting leader based on bearing measurements. A follower needs to maintain a desired relative position to the leader in its local coordinate frame using bearing measurements. An integrated method consisting of a bearing-based estimator and a tracking controller is proposed to achieve the leader-following control objective. The estimator is designed to estimate the relative position to the leader in the follower's local coordinate frame. Based on the estimated relative position, the controller is proposed to drive the follower to achieve the desired formation. Different from existing integrated methods, our method does not require any conditions on the relative bearing between the leader and the follower in the global frame, which are difficult to guarantee. We characterize the conditions locally on the leader only, namely, conditions on the leader's linear and angular velocities with persistently exciting steering speed, such that the overall system is asymptotically stable with our integrated method. Besides, the proposed method is extended to the multi-agent formation control problem. A reasonable solution is proposed to the multi-agent case with a minimal leader follower structure. Simulations prove the effectiveness of the proposed methods in both cases.

Multiagent, Multitarget Path Planning in Markov Decision Processes

Missions for autonomous systems often require agents to visit multiple targets in complex operating conditions. This work considers the problem of visiting a set of targets in minimum time by a team of non-communicating agents in a Markov decision process (MDP). The single-agent problem is at least NP-complete by reducing it to a Hamiltonian path problem. We first discuss an optimal algorithm based on Bellman's optimality equation that is exponential in the number of target states. Then, we trade-off optimality for time complexity by presenting a suboptimal algorithm that is polynomial at each time step. We prove that the proposed algorithm generates optimal policies for certain classes of MDPs. Extending our procedure to the multi-agent case, we propose a target partitioning algorithm that approximately minimizes the expected time to visit the targets. We prove that our algorithm generates optimal partitions for clustered target scenarios. We present the performance of our algorithms on random MDPs and gridworld environments inspired by ocean dynamics. We show that our algorithms are much faster than the optimal procedure and more optimal than the currently available heuristic.

A Repeated Unknown Game: Decentralized Task Offloading in Vehicular Fog Computing

Offloading computation to nearby edge/fog computing nodes, including the ones carried by moving vehicles, e.g., vehicular fog nodes (VFN), has proved to be a promising approach for enabling low-latency and compute-intensive mobility applications, such as cooperative and autonomous driving. This work considers vehicular fog computing scenarios where the clients of computation offloading services try to minimize their own costs while deciding which VFNs to offload their tasks. We focus on decentralized multi-agent decision-making in a repeated unknown game where each agent, e.g., service client, can observe only its own action and realized cost. In other words, each agent is unaware of the game composition or even the existence of opponents. We apply a completely uncoupled learning rule to generalize the decentralized decision-making algorithm presented in [7] for the multi-agent case. The multi-agent solution proposed in this work can capture the unknown offloading cost variations susceptive to resource congestion under an adversarial framework where each agent may take implicit cost estimation and suitable resource choice adapting to the dynamics associated with volatile supply and demand. According to the evaluation via simulation, this work reveals that such individual perturbations for robustness to uncertainty and adaptation to dynamicity ensure a certain level of optimality in terms of social welfare, e.g., converging the actual sequence of play with unknown and asymmetric attributes and lowering the correspondent cost in social welfare due to the self-interested behaviors of agents.

Improving professional practice in the investigation and management of intrafamilial child sexual abuse: Qualitative analysis of serious child protection reviews

BackgroundLocal multi-agency case reviews are regularly held in England when children have suffered significant harm from abuse or neglect, including from child sexual abuse (CSA). Most CSA takes place within families, is common but under-reported and can cause long-term harm. ObjectiveThe aim was to analyse English child protection reviews relating to intrafamilial CSA to identify improvements for professional practice. Participants and settingLocal Child Safeguarding Practice Reviews (LCSPRs) and Serious Case Reviews (SCRs) relating to serious incidents of intrafamilial CSA occurring between 01 April 2017 and 31 March 2020. MethodsLCSPRs and SCRs were obtained from the National Case Review Repository and thematically analysed. ResultsThere were 243 reviews, of which 25 featured intrafamilial CSA. The main themes related to perpetrators, vulnerable families, and professional practice. Half of perpetrators were known by services to have previously abused children, but issues with professional practice enabled them to continue. Most children did not disclose CSA verbally showing challenging or sexualised behaviour; but professionals lacked knowledge and confidence on how to intervene without verbal disclosure, which limited safeguarding actions. Non-engagement by families with services was common, with some non-abusing parents complicit in abuse and deception. Significant neglect occurred in half the families, which diverted professional attention away from CSA. ConclusionCSA is deliberate abuse of children involving considerable deception by perpetrators in contrast to some other types of child abuse. This difference in abuser behaviour makes child protection more difficult, particularly when professionals do not recognise and respond to children's non-verbal disclosures.

LTL cross entropy optimisation for quadcopter task orchestration

ABSTRACT This paper presents a task orchestration framework for multi-agent systems utilising linear temporal logic (LTL) and cross entropy optimisation, a stochastic optimisation technique used for rare-event sampling. We define task orchestration as a combination of task decomposition, allocation and planning for a quadcopter or team of quadcopters given a high-level specification. Specifically, we consider tasks that are complex and consist of environment constraints, system constraints, or both, that must be satisfied. We first approach motion planning for the single agent case where transition systems for the environment allow tasks to be developed as linear temporal logic (LTL) specifications. Trajectories are then generated via motion primitives for a single quadcopter and optimised via cross entropy to ensure optimal satisfaction of a cost function. We extend this work to the multi-agent case where a team of homogeneous quadcopters are considered to satisfy an LTL specification. In order to provide faster computations and initial cost-agnostic sampling, we formulate the online version of multi-agent task allocation via cross entropy for tasks specified in LTL specifications. The results of this framework are verified in simulation and experimentally with a team of quadcopters.

Multi-Agent Reinforcement Learning in NOMA-Aided UAV Networks for Cellular Offloading

A novel framework is proposed for cellular offloading with the aid of multiple unmanned aerial vehicles (UAVs), while non-orthogonal multiple access (NOMA) technique is employed at each UAV to further improve the spectrum efficiency of the wireless network. The optimization problem of joint three-dimensional (3D) trajectory design and power allocation is formulated for maximizing the throughput. Since ground mobile users are considered as roaming continuously, the UAVs need to be re-deployed timely based on the movement of users. In an effort to solve this pertinent dynamic problem, a K-means based clustering algorithm is first adopted for periodically partitioning users. Afterward, a mutual deep Q-network (MDQN) algorithm is proposed to jointly determine the optimal 3D trajectory and power allocation of UAVs. In contrast to the conventional deep Q-network (DQN) algorithm, the MDQN algorithm enables the experience of multi-agent to be input into a shared neural network to shorten the training time with the assistance of state abstraction. Numerical results demonstrate that: 1) the proposed MDQN algorithm is capable of converging under minor constraints and has a faster convergence rate than the conventional DQN algorithm in the multi-agent case; 2) The achievable sum rate of the NOMA enhanced UAV network is 23% superior to the case of orthogonal multiple access (OMA); 3) By designing the optimal 3D trajectory of UAVs with the MDON algorithm, the sum rate of the network enjoys 142% and 56% gains than invoking the circular trajectory and the 2D trajectory, respectively.

Field estimation using binary measurements

In this paper we consider the problem of field estimation using binary sensor measurements. A number of agents are equipped with sensors which can make binary measurements of the field. The agents are mobile and can move around to different locations to collect measurements. In estimating the field, we model the field as a sum of radial basis functions, whose parameters are then estimated using sequential Monte Carlo techniques. We incorporate an active sensing mechanism for the agents to adaptively choose their next measurement location, given the information currently collected. In the multi-agent case, we consider both a centralized setup, and a decentralized setup where agents share their past measurements with each other provided they are within communication range. Simulation studies illustrate the performance of the proposed algorithms.

Logics for Knowability

In this paper, we propose three knowability logics LK, LK−, and LK=. In the single-agent case, LK is equally expressive as arbitrary public announcement logic APAL and public announcement logic PAL, whereas in the multi-agent case, LK is more expressive than PAL. In contrast, both LK− and LK= are equally expressive as classical propositional logic PL. We present the axiomatizations of the three knowability logics and show their soundness and completeness. We show that all three knowability logics possess the properties of Church-Rosser and McKinsey. Although LK is undecidable when at least three agents are involved, LK− and LK= are both decidable.

A Distributed Approach to Summarizing Spaces of Multiagent Schedules

We introduce the Multiagent Disjunctive Temporal Problem (MaDTP), a new distributed formulation of the widely-adopted Disjunctive Temporal Problem (DTP) representation. An agent that generates a summary of all viable schedules, rather than a single schedule, can be more useful in dynamic environments. We show how a (Ma)DTP with the properties of minimality and decomposability provides a particularly efficacious solution space summary.However, in the multiagent case, these properties sacrifice an agent's strategic interests while incurring significant computational overhead. We introduce a new property called local decomposability that exploits loose-coupling between agents' problems, protects strategic interests, and supports typical queries. We provide and evaluate a new distributed algorithm that summarizes agents' solution spaces in significantly less time and space by using local, rather than full, decomposability.

Repeated-Task Canadian Traveler Problem

In the Canadian Traveler Problem (CTP) a traveling agent is given a weighted graph, where some of the edges may be blocked, with a known probability. The agent needs to travel to a given goal. A solution for CTP is a policy, that has the smallest expected traversal cost. CTP is intractable. Previous work has focused on the case of a single agent. We generalize CTP to a repeated task version where a number of agents need to travel to the same goal, minimizing their combined travel cost. We provide optimal algorithms for the special case of disjoint path graphs. Based on a previous UCT-based approach for the single agent case, a framework is developed for the multi-agent case and four variants are given - two of which are based on the results for disjoint-path graphs. Empirical results show the benefits of the suggested framework and the resulting heuristics. For small graphs where we could compare to optimal policies, our approach achieves near optimal results at only a fraction of the computation cost.

A Behavior-Based Mobile Robot Navigation Method with Deep Reinforcement Learning

Deep reinforcement learning-based mobile robot navigation has attracted some recent interest. In the single-agent case, a robot can learn to navigate autonomously without a map of the environment. In the multi-agent case, robots can learn to avoid collisions with each other. In this work, we propose a behavior-based mobile robot navigation method which directly maps the raw sensor data and goal information to the control command. The learned navigation policy can be applied in both single-agent and multi-agent scenarios. Two basic navigation behaviors are considered in our method, which are goal reaching and collision avoidance. The two behaviors are fused based on the risk-level estimation of the current state. The navigation task is decomposed using the behavior-based framework, which is capable of reducing the complexity of the learning process. The simulations and real-world experiments demonstrate that the proposed method can enable the collision-free autonomous navigation of multiple mobile robots in unknown environments.

Optimal Disclosure of Information to a Privately Informed Receiver

We study information design settings where the designer controls information about a state, and there are multiple agents interacting in a game who are privately informed about their types. Each agent’s utility depends on all agents’ types and actions, as well as (linearly) on the state. To optimally screen the agents, the designer first asks agents to report their types and then sends a private action recommendation to each agent whose distribution depends on all reported types and the state. We show that there always exists an optimal mechanism which is laminar partitional. Such a mechanism partitions the state space for each type profile and recommends the same action profile for states that belong to the same partition element. Furthermore, the convex hulls of any two partition elements are such that either one contains the other or they have an empty intersection. In the single-agent case, each state is either perfectly revealed or lies in an interval in which the number of different signal realizations is at most the number of different types of the agent plus two. A similar result is established for the multi-agent case. We also highlight the value of screening: without screening the best achievable payoff could be as low as one over the number of types fraction of the optimal payoff. Along the way, we shed light on the solutions of optimization problems over distributions subject to a mean-preserving contraction constraint and additional side constraints, which might be of independent interest.

A Multi-Agent Off-Policy Actor-Critic Algorithm for Distributed Reinforcement Learning

This paper extends off-policy reinforcement learning to the multi-agent case in which a set of networked agents communicating with their neighbors according to a time-varying graph collaboratively evaluates and improves a target policy while following a distinct behavior policy. To this end, the paper develops a multi-agent version of emphatic temporal difference learning for off-policy policy evaluation, and proves convergence under linear function approximation. The paper then leverages this result, in conjunction with a novel multi-agent off-policy policy gradient theorem and recent work in both multi-agent on-policy and single-agent off-policy actor-critic methods, to develop and give convergence guarantees for a new multi-agent off-policy actor-critic algorithm.

Reach control problem for affine multi-agent systems on simplices

This paper studies the reach control problem for a coupled affine multi-agent system, which aims to find an affine feedback control for the trajectories of the agents to reach and exit a particular facet of a given simplex in the state space in finite time. The interactions between agents characterized by diffusive coupling prevent the effective construction of controller using the well developed techniques to study similar problems for affine single-agent systems. In fact, the affine feedback control designed for a single affine system may not work for the multi-agent case anymore as some agent can be driven to exit the simplex through a restricted facet under the influence from its coupled peers. A sufficient condition is developed to guarantee that all the agents move continuously in a cone containing the simplex and exit through the exit facets in finite time under an affine feedback control. A numerical example is given to verify the effectiveness of our derived result.

Price Dynamics under the Information-Based Dealer Model

We investigate the stochastic behavior of the transaction price process formed by the information-based dealer model. We extend two-agent model of Nadi Serhan Ayd?n to multi-agent case to see the effect of the market size and noise correlations. Applying the Monte Carlo method, our numerical findings are summarized as 1) volatility of the transaction price depends on both of the noise correlation and the market size, 2) the price process has a long term memory and its Hurst extend depends on both of the noise correlation and the market size.

The brain attics: the strategic role of memory in single and multi-agent inquiry

M. B. Hintikka (1939–1987) and J. Hintikka (1929–2016) claimed that their reconstruction of the ‘Sherlock Holmes sense of deduction’ can “serve as an explication for the link between intelligence and memory” (1983, p. 159). The claim is vindicated, first for the single-agent case, where the reconstruction captures strategies for accessing the content of a distributed and associative memory; then, for the multi-agent case, where the reconstruction captures strategies for accessing knowledge distributed in a community. Moreover, the reconstruction of the ‘Sherlock Holmes sense of deduction’ allows to conceptualize those strategies as belonging to a continuum of behavioral strategies.

Performance limits of stochastic sub-gradient learning, part II: Multi-agent case

The analysis in Part I [1] revealed interesting properties for subgradient learning algorithms in the context of stochastic optimization. These algorithms are used when the risk functions are non-smooth or involve non-differentiable components. They have been long recognized as being slow converging methods. However, it was revealed in Part I [1] that the rate of convergence becomes linear for stochastic optimization problems, with the error iterate converging at an exponential rate αi to within an O(μ)−neighborhood of the optimizer, for some α ∈ (0, 1) and small step-size μ. The conclusion was established under weaker assumptions than the prior literature and, moreover, several important problems were shown to satisfy these weaker assumptions automatically. These results revealed that sub-gradient learning methods have more favorable behavior than originally thought. The results of Part I [1] were exclusive to single-agent adaptation. The purpose of current Part II is to examine the implications of these discoveries when a collection of networked agents employs subgradient learning as their cooperative mechanism. The analysis will show that, despite the coupled dynamics that arises in a networked scenario, the agents are still able to attain linear convergence in the stochastic case; they are also able to reach agreement within O(μ) of the optimizer.

Decentralised Detection of Emergence in Complex Adaptive Systems

This article describes Decentralised Emergence Detection (DETect), a novel distributed algorithm that enables agents to collaboratively detect emergent events in Complex Adaptive Systems (CAS). Non-deterministic interactions between agents in CAS can give rise to emergent behaviour or properties at the system level. The nature, timing, and consequence of emergence is unpredictable and may be harmful to the system or individual agents. DETect relies on the feedback that occurs from the system level (macro) to the agent level (micro) when emergence occurs. This feedback constrains agents at the micro level and results in changes occurring in the relationship between an agent and its environment. DETect uses statistical methods to automatically select the properties of the agent and environment to monitor and tracks the relationship between these properties over time. When a significant change is detected, the algorithm uses distributed consensus to determine if a sufficient number of agents have simultaneously experienced a similar change. On agreement of emergence, DETect raises an event, which its agent or other interested observers can use to act appropriately. The approach is evaluated using a multi-agent case study.

Stochastic programming bilevel models for service provision with a balancing coordinator

This paper introduces a framework for the quantitative analysis of collaborative service platforms. Services built around platforms require the coordinated collaboration of several independent agents, each from a different field or area of expertise and each providing one or more components to the overall service. One of these agents takes the further role of platform coordinator, whose role consists in assembling the services from components provided by other agents and sharing the revenue between the participants. The agents decide which services to make a contribution to by selecting service portfolios that maximize their expected profit under constraints on risk and capacity. The coordinator selects the revenue-sharing scheme that balances the offers from other agents and ensures the functioning of the platform. We develop two stochastic optimization models with bilevel structure for the analysis of the collaborative service platform provision, analyse the properties of the solutions and provide numerical experiments that show the qualitative difference of the profit//risk trade-off in the multiagent case compared with the classical single agent case.

The interrogative model of inquiry meets dynamic epistemic logics

The Interrogative Model of Inquiry (IMI) and Dynamic Epistemic Logics (DELs) are two central paradigms in formal epistemology. This paper is motivated by the observation of a significant complementarity between them: on the one hand, the IMI provides a framework for investigating inquiry represented as an idealized game between an Inquirer and Nature, along with an account of the interaction between questions and inferences in information-seeking processes, but is lacking a formulation in the multi-agent case; on the other hand, DELs model various operations of information change in multi-agent systems, but the field is lacking a proper integration of question and inference dynamics, along with an application to the investigation of inquiry processes. The goal of this paper is to integrate the two paradigms in such a way as to combine their respective insights. To this end, we develop a formal system called DEL\(_\mathrm{IMI }\) which aims to represent the interaction between question and inference dynamics in inquiry—as described by the IMI—in a multi-agent setting, and this in such a way as to enable an investigation of inquiry games with multi-agent dimensions. The DEL\(_\mathrm{IMI }\) system is designed to represent the possible moves of such inquiry games through three types of epistemic actions: agents addressing questions to Nature, agents addressing questions to other agents, agents drawing logical inferences. We then show how the resulting framework can be used to formally define multi-agent inquiry games. We conclude by evaluating the interest of the DEL\(_\mathrm{IMI }\) system for the IMI and DELs paradigms.