Hundreds Of Agents Research Articles

Efficient and scalable reinforcement learning for large-scale network control

The primary challenge in the development of large-scale artificial intelligence (AI) systems lies in achieving scalable decision-making—extending the AI models while maintaining sufficient performance. Existing research indicates that distributed AI can improve scalability by decomposing complex tasks and distributing them across collaborative nodes. However, previous technologies suffered from compromised real-world applicability and scalability due to the massive requirement of communication and sampled data. Here we develop a model-based decentralized policy optimization framework, which can be efficiently deployed in multi-agent systems. By leveraging local observation through the agent-level topological decoupling of global dynamics, we prove that this decentralized mechanism achieves accurate estimations of global information. Importantly, we further introduce model learning to reinforce the optimal policy for monotonic improvement with a limited amount of sampled data. Empirical results on diverse scenarios show the superior scalability of our approach, particularly in real-world systems with hundreds of agents, thereby paving the way for scaling up AI systems.

Large-scale immigration worksite raids: community disaster, community response

ABSTRACT Immigration enforcement – or the surveillance, detention, and deportation of noncitizens – detrimentally impacts health, with spillover effects on the families of those targeted. However, less is known about how immigration enforcement impacts communities, regardless of the immigration statuses of members. Immigration worksite raids can involve hundreds of agents, helicopters, and buses descending unannounced upon rural towns with small Latino populations and removing dozens of members in a day. This egregious enforcement tactic presents the opportunity to better understand how communities organize to mitigate the harm of these events. We conducted semistructured interviews in Spanish and English in six communities that experienced the largest worksite raids in 2018. Participants were 77 adults who provided material, emotional, or professional support following raids. We used qualitative analysis methods to code all interviews and develop a timeline characterizing raid impact and response. The day of the raid is characterized by chaos, confusion, and fear, which resulted in hiding out in churches and schools and supporting the children of those who disappeared. The days and weeks that followed were characterized by hunger, resulting in multiple strategies aimed at keeping the community fed, along with efforts to re-integrate children into schools. Months after, communities began to organize rideshares and finally address health concerns. We end with recommendations to mitigate the community-wide damage of worksite raids while advocating against their use.

Stylized Crowd Formation Transformation Through Spatiotemporal Adversarial Learning

Achieving crowd formation transformations has wide‐ranging applications in fields such as unmanned aerial vehicle formation control, crowd simulation, and large‐scale performances. However, planning trajectories for hundreds of agents is a challenging and tedious task. When modifying crowd formation change schemes, adjustments are typically required based on the style of formation change. Existing methods often involve manual adjustments at each crucial step, leading to a substantial amount of physical labor. Motivated by these challenges, this study introduces a novel generative adversarial network (GAN) for generating crowd formation transformations. The proposed GAN learns specific styles from a series of crowd formation transformation trajectories and can transform a new crowd with an arbitrary number of individuals into the same styles with minimal manual intervention. The model incorporates a space–time transformer module to aggregate spatiotemporal information for learning distinct styles of formation transformation. Furthermore, this article investigates the relationship between the distribution of training data and the length of trajectory sequences, providing insights into the preprocessing of training data.

Multi-Unit Auctions for Allocating Chance-Constrained Resources

Sharing scarce resources is a key challenge in multi-agent interaction, especially when individual agents are uncertain about their future consumption. We present a new auction mechanism for preallocating multi-unit resources among agents, while limiting the chance of resource violations. By planning for a chance constraint, we strike a balance between worst-case approaches, which under-utilise resources, and expected-case approaches, which lack formal guarantees. We also present an algorithm that allows agents to generate bids via multi-objective reasoning, which are then submitted to the auction. We then discuss how the auction can be extended to non-cooperative scenarios. Finally, we demonstrate empirically that our auction outperforms state-of-the-art techniques for chance-constrained multi-agent resource allocation in complex settings with up to hundreds of agents.

Optimal Pathfinding on Weighted Grid Maps

In many computer games up to hundreds of agents navigate in real-time across a dynamically changing weighted grid map. Pathfinding in these situations is challenging because the grids are large, traversal costs are not uniform, and because each shortest path has many symmetric permutations, all of which must be considered by an optimal online search. In this work we introduce Weighted Jump Point Search (JPSW), a new type of pathfinding algorithm which breaks weighted grid symmetries by introducing a tiebreaking policy that allows us to apply effective pruning rules in symmetric regions. We show that these pruning rules preserve at least one optimal path to every grid cell and that their application can yield large performance improvements for optimal pathfinding. We give a complete theoretical description of the new algorithm, including pseudo-code. We also conduct a wide-ranging experimental evaluation, including data from real games. Results indicate JPSW is up to orders of magnitude faster than the nearest baseline, online search using A*.

LaCAM: Search-Based Algorithm for Quick Multi-Agent Pathfinding

We propose a novel complete algorithm for multi-agent pathfinding (MAPF) called lazy constraints addition search for MAPF (LaCAM). MAPF is a problem of finding collision-free paths for multiple agents on graphs and is the foundation of multi-robot coordination. LaCAM uses a two-level search to find solutions quickly, even with hundreds of agents or more. At the low-level, it searches constraints about agents' locations. At the high-level, it searches a sequence of all agents' locations, following the constraints specified by the low-level. Our exhaustive experiments reveal that LaCAM is comparable to or outperforms state-of-the-art sub-optimal MAPF algorithms in a variety of scenarios, regarding success rate, planning time, and solution quality of sum-of-costs.

Double-Deck Multi-Agent Pickup and Delivery: Multi-Robot Rearrangement in Large-Scale Warehouses

We introduce a new problem formulation, Double-Deck Multi-Agent Pickup and Delivery (DD-MAPD), which models the multi-robot shelf rearrangement problem in automated warehouses. DD-MAPD extends both Multi-Agent Pickup and Delivery (MAPD) and Multi-Agent Path Finding (MAPF) by allowing agents to move beneath shelves or lift and deliver a shelf to an arbitrary location, thereby changing the warehouse layout. We show that solving DD-MAPD is NP-hard. To tackle DD-MAPD, we propose MAPF-DECOMP, an algorithmic framework that decomposes a DD-MAPD instance into a MAPF instance for coordinating shelf trajectories and a subsequent MAPD instance with task dependencies for computing paths for agents. We also present an optimization technique to improve the performance of MAPF-DECOMP and demonstrate how to make MAPF-DECOMP complete for well-formed DD-MAPD instances, a realistic subclass of DD-MAPD instances. Our experimental results demonstrate the efficiency and effectiveness of MAPF-DECOMP, with the ability to compute high-quality solutions for large-scale instances with over one thousand shelves and hundreds of agents in just minutes of runtime.

Prioritized Safe Interval Path Planning for Multi-Agent Pathfinding With Continuous Time on 2D Roadmaps

We address a challenging multi-agent pathfinding (MAPF) problem for hundreds of agents moving on a 2D roadmap with continuous time. Despite its known potential for producing better solutions compared to typical grid and discrete-time cases, few approaches have been established to solve this problem due to the intractability of collision checks on a large scale. In this work, we propose Prioritized Safe-Interval Path Planning with Continuous-Time Conflicts (PSIPP/CTC) that extends a scalable prioritized planning algorithm to work on the 2D roadmap and continuous-time setup by alleviating intensive collision checks. Our approach involves a novel concept named Continuous-Time Conflict (CTC), which describes a pair among vertices and edges associated with continuous-time intervals within which collisions can happen between agents. We pre-compute CTCs using geometric neighbor-search and sweeping techniques and annotate roadmaps with the CTCs just once before planning starts. Doing so allows us to efficiently enumerate collision-free time intervals for all vertices and edges and find each agent's path with continuous time in prioritized planning. Extensive experimental evaluations demonstrate that PSIPP/CTC significantly outperforms existing methods in terms of planning success rate and runtime while maintaining an acceptable solution quality. As a proof of concept, we also confirmed the effectiveness of the proposed approach on a physics simulation with differential wheeled robots.

Priority inheritance with backtracking for iterative multi-agent path finding

In the Multi-Agent Path Finding (MAPF) problem, a set of agents moving on a graph must reach their own respective destinations without inter-agent collisions. In practical MAPF applications such as navigation in automated warehouses, where occasionally there are hundreds or more agents, MAPF must be solved iteratively online on a lifelong basis. Such scenarios rule out simple adaptations of offline compute-intensive optimal approaches; and scalable sub-optimal algorithms are hence appealing for such settings. Ideal algorithms are scalable, applicable to iterative scenarios, and output plausible solutions in predictable computation time.For the aforementioned purpose, this study presents Priority Inheritance with Backtracking (PIBT), a novel sub-optimal algorithm to solve MAPF iteratively. PIBT relies on an adaptive prioritization scheme to focus on the adjacent movements of multiple agents; hence it can be applied to several domains. We prove that, regardless of their number, all agents are guaranteed to reach their destination within finite time when the environment is a graph such that all pairs of adjacent nodes belong to a simple cycle (e.g., biconnected). Experimental results covering various scenarios, including a demonstration with real robots, reveal the benefits of the proposed method. Even with hundreds of agents, PIBT yields acceptable solutions almost immediately and can solve large instances that other established MAPF methods cannot. In addition, PIBT outperforms an existing approach on an iterative scenario of conveying packages in an automated warehouse in both runtime and solution quality.

Concentration Network for Reinforcement Learning of Large-Scale Multi-Agent Systems

When dealing with a series of imminent issues, humans can naturally concentrate on a subset of these concerning issues by prioritizing them according to their contributions to motivational indices, e.g., the probability of winning a game. This idea of concentration offers insights into reinforcement learning of sophisticated Large-scale Multi-Agent Systems (LMAS) participated by hundreds of agents. In such an LMAS, each agent receives a long series of entity observations at each step, which can overwhelm existing aggregation networks such as graph attention networks and cause inefficiency. In this paper, we propose a concentration network called ConcNet. First, ConcNet scores the observed entities considering several motivational indices, e.g., expected survival time and state value of the agents, and then ranks, prunes, and aggregates the encodings of observed entities to extract features. Second, distinct from the well-known attention mechanism, ConcNet has a unique motivational subnetwork to explicitly consider the motivational indices when scoring the observed entities. Furthermore, we present a concentration policy gradient architecture that can learn effective policies in LMAS from scratch. Extensive experiments demonstrate that the presented architecture has excellent scalability and flexibility, and significantly outperforms existing methods on LMAS benchmarks.

Distributed Policy Synthesis of Multiagent Systems With Graph Temporal Logic Specifications

We study the distributed synthesis of policies for multiagent systems to perform <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">spatial–temporal</i> tasks. We formalize the synthesis problem as a <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">factored</i> Markov decision process subject to <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">graph temporal logic</i> specifications. The transition function and the task of each agent are functions of the agent itself and its neighboring agents. In this work, we develop another distributed synthesis method, which improves the scalability and runtime by two orders of magnitude compared to our prior work. The synthesis method decomposes the problem into a set of smaller problems, one for each agent by leveraging the structure in the model, and the specifications. We show that the running time of the method is linear in the number of agents. The size of the problem for each agent is exponential only in the number of neighboring agents, which is typically much smaller than the number of agents. We demonstrate the applicability of the method in case studies on disease control, urban security, and search and rescue. The numerical examples show that the method scales to hundreds of agents with hundreds of states per agent and can also handle significantly larger state spaces than our prior work.

Extended Abstract: Lifelong Path Planning with Kinematic Constraints for Multi-Agent Pickup and Delivery

The Multi-Agent Pickup and Delivery (MAPD) problem models applications where a large number of agents attend to a stream of incoming pickup-and-delivery tasks. Token Passing (TP) is a recent MAPD algorithm that is efficient and effective. We make TP even more efficient and effective by using a novel combinatorial search algorithm, called Safe Interval Path Planning with Reservation Table (SIPPwRT), for single-agent path planning. SIPPwRT uses an advanced data structure that allows for fast updates and lookups of the current paths of all agents in an online setting. The resulting MAPD algorithm TP-SIPPwRT takes kinematic constraints of real robots into account directly during planning, computes continuous agent movements with given velocities that work on non-holonomic robots rather than discrete agent movements with uniform velocity, and is complete for well-formed MAPD instances. We demonstrate its benefits for automated warehouses using both an agent simulator and a standard robot simulator. For example, we demonstrate that it can compute paths for hundreds of agents and thousands of tasks in seconds and is more efficient and effective than existing MAPD algorithms that use a post-processing step to adapt their paths to continuous agent movements with given velocities. This paper was published at AAAI 2019.

Experimental Evaluation of Classical Multi Agent Path Finding Algorithms

Modern optimal multi-agent path finding (MAPF) algorithms can scale to solve problems with hundreds of agents. To facilitate comparison between these algorithms, a benchmark of MAPF problems was recently proposed. We report a comprehensive evaluation of a diverse set of state-of-the-art optimal MAPF algorithms over the entire benchmark. The results show that in terms of coverage, the recently proposed LazyCBS algorithm outperforms all others significantly, but it is usually not the fastest algorithm. This suggests algorithm selection methods can be beneficial. Then, we characterize different setups for algorithm selection in MAPF, and evaluate simple baselines for each setup. Finally, we propose an extension of the existing MAPF benchmark in the form of different ways to distribute the agents’ source and target locations.

A Multi-Label A* Algorithm for Multi-Agent Pathfinding

Given a set of agents, the multi-agent pathfinding problem consists in determining, for each agent, a path from its start location to its assigned goal while avoiding collisions with other agents. Recent work has studied variants of the problem in which agents are assigned a sequence of goals (tasks) that become available over time, such as the online multi-agent pickup and delivery (MAPD) problem. In this paper, we propose a multi-label A* algorithm (MLA*) for this problem. It extends the classic A* algorithm by allowing the computation of paths with multiple ordered goals (such as a pickup and delivery). Moreover, we develop a new h-value-based centralized heuristic for the MAPD. Computational experiments show that our proposed MLA* obtains substantial improvements in terms of makespan and service time as compared to existing methods, while being more computationally efficient. On instances with a thousand tasks and hundreds of agents, our method reduces the average service time by 43% compared to the state of the art, with considerably less computational effort.

Heter-Sim: Heterogeneous Multi-Agent Systems Simulation by Interactive Data-Driven Optimization.

Interactive multi-agent simulation algorithms are used to compute the trajectories and behaviors of different entities in virtual reality scenarios. However, current methods involve considerable parameter tweaking to generate plausible behaviors. We introduce a novel approach (Heter-Sim) that combines physics-based simulation methods with data-driven techniques using an optimization-based formulation. Our approach is general and can simulate heterogeneous agents corresponding to human crowds, traffic, vehicles, or combinations of different agents with varying dynamics. We estimate motion states from real-world datasets that include information about position, velocity, and control direction. Our optimization algorithm considers several constraints, including velocity continuity, collision avoidance, attraction, direction control. Other constraints are implemented by introducing a novel energy function to control the motions of heterogeneous agents. To accelerate the computations, we reduce the search space for both collision avoidance and optimal solution computation. Heter-Sim can simulate tens or hundreds of agents at interactive rates and we compare its accuracy with real-world datasets and prior algorithms. We also perform user studies that evaluate the plausible behaviors generated by our algorithm and a user study that evaluates the plausibility of our algorithm via VR.

C-Nav: Distributed coordination in crowded multi-agent navigation

In crowded multi-agent navigation, the motion of the agents is significantly constrained by the motion of the nearby agents. This makes planning paths very difficult and leads to inefficient global motion. To address this problem, we propose a distributed approach, which we call C-Nav, that introduces politeness into multi agent navigation. With our approach, agents take into account the velocities and goals of their neighbors and optimize their motion accordingly and in real-time. Further, we perform a theoretical analysis of the algorithm, and experimentally demonstrate its advantages in simulation, with hundreds of agents in a variety of scenarios, and in real world navigation tasks with several mobile robots.

Occupational Exposure to Metals and Solvents: Allergy and Airway Diseases.

Occupational allergic diseases (OAD) such as occupational contact dermatitis (OCD), occupational asthma (OA), and occupational rhinitis (OR) are the most prevalent occupational diseases in industrialized countries. The purpose of this review is to provide an update about the main occupational metal and solvent exposures related to allergy and airway diseases and to discuss newly defined causative agents and industries in this field. Currently for over 400 causative agents for OA and OCD, several hundreds of agents for OR have been identified. Although many studies have reported an overall decline in OAD related to known agents after implementation of efficient and effective workplace preventive measures, the constant development of new products continuously introduces to the market potential unknown respiratory hazards. Workplace allergens are often high molecular weight (HMW) agents that are > 10kDa molecular weight and capable of eliciting IgE sensitization. Sensitizing low molecular weight (LMW) agents are often reactive chemicals. Metals and solvents are two large causative agent groups related to OADs that mainly behave as LMW (< 10kDa) sensitizers and/or irritants. Avoidance of causative exposures through control strategies is the primary prevention approach for OADs. These strategies must be applied and covered for all known and newly defined causative agents. This review aims to summarize current status of known occupational metal and solvent exposures related to allergy and airway diseases and to discuss newly defined causative agents and industries in this field.

Algorithm Selection for Optimal Multi-Agent Pathfinding

The challenge of finding an optimal solution to a multi-agent path finding (MAPF) problem has attracted significant academic and industrial interest in recent years. While the problem is NP-Hard, modern optimal MAPF algorithms can scale to solve problems with hundreds of agents. Nevertheless, no single optimal MAPF algorithm dominates all benchmarks problems, and there are no clear, provable, guidelines for when each algorithm should be used. To address this, we present the first successful Algorithm Selection (AS) model for optimal MAPF. We propose two approaches for learning an AS model. The first approach uses a standard supervised learning algorithm with a set of handcrafted MAPF-specific features. The second approach, casts a MAPF problem to an image and applies a deep Convolutional Neural Network to classify it. We evaluate both approaches over a large dataset and show that using an AS model to select which algorithm to use for each instance results in solving more problems and in a shorter runtime compared to the state of the art.

LSwarm: Efficient Collision Avoidance for Large Swarms With Coverage Constraints in Complex Urban Scenes

In this letter, we address the problem of collision avoidance for a swarm of UAVs used for continuous surveillance of an urban environment. Our method, LSwarm, efficiently avoids collisions with static obstacles, dynamic obstacles and other agents in three-dimensional urban environments while considering coverage constraints. LSwarm calculates collision avoiding velocities that maximize the conformity of an agent to an optimal path given by a global coverage strategy and ensure sufficient resolution of the coverage data collected by each agent. Our algorithm is formulated based on optimal reciprocal collision avoidance and is scalable with respect to the size of the swarm. We evaluate the coverage performance of LSwarm in realistic simulations of a swarm of quadrotors in complex urban models. In practice, our approach can compute collision avoiding velocities for a swarm composed of tens to hundreds of agents in a few milliseconds on dense urban scenes consisting of tens of buildings.

OSSEC IDS Extension to Improve Log Analysis and Override False Positive or Negative Detections

Intrusion Detection Systems (IDS) are used to prevent attacks by detecting potential harmful intrusion attempts. Currently, there are a set of available Open Source IDS with different characteristics. The Open Source Host-based Intrusion Detection System (OSSEC) supports multiple features and its implementation consists of Agents that collect and send event logs to a Manager that analyzes and tests them against specific rules. In the Manager, if certain events match a specific rule, predefined actions are triggered in the Agents such as to block or unblock a particular IP address. However, once an action is triggered, the systems administrator is not able to centrally check and obtain detailed information of the past event logs. In addition, OSSEC may assume false positive or negative detections and their triggered actions: previously harmless but blocked IP addresses by OSSEC have to be unblocked in order to reestablish normal operation or potential harmful IP addresses not previously blocked by OSSEC should be blocked in order to increase protection levels. These operations to override OSSEC actions must be manually performed in every Agent, thus requiring time and human resources. Both these limitations have a higher impact on large scale OSSEC deployments assuming tens or hundreds of Agents. This paper proposes an extension to OSSEC that improves the administrator analysis capability by maintaining, organizing and presenting Agent logs in a central point, and it allows for blocking or unblocking IP addresses in order to override actions triggered by false detections. The proposed extension aims to increase efficiency of time and human resources management, mainly considering large scale OSSEC deployments.