Multi-agent Collision Avoidance Research Articles

Safe and Efficient Multi-Agent Collision Avoidance With Physics-Informed Reinforcement Learning

Safe and Efficient Multi-Agent Collision Avoidance With Physics-Informed Reinforcement Learning

Maximum-Entropy Multi-Agent Dynamic Games: Forward and Inverse Solutions

In this article, we study the problem of multiple stochastic agents interacting in a dynamic game scenario with continuous state and action spaces. We define a new notion of stochastic Nash equilibrium for boundedly rational agents, which we call the entropic cost equilibrium (ECE). We show that ECE is a natural extension to multiple agents of maximum entropy optimality for a single agent. We solve both the “forward” and “inverse” problems for the multi-agent ECE game. For the forward problem, we provide a Riccati algorithm to compute closed-form ECE feedback policies for the agents, which are exact in the linear-quadratic-gaussian case. We give an iterative variant to find locally ECE feedback policies for the nonlinear case. For the inverse problem, we present an algorithm to infer the cost functions of the multiple interacting agents given noisy, boundedly rational input and state trajectory examples from agents acting in an ECE. The effectiveness of our algorithms is demonstrated in a simulated multi-agent collision avoidance scenario, and with data from the INTERACTION traffic dataset. In both cases, we show that, by taking into account the agents' game theoretic interactions using our algorithm, a more accurate model of agents' costs can be learned, compared with standard inverse optimal control methods.

Collision Avoidance Among Dense Heterogeneous Agents Using Deep Reinforcement Learning

Navigating in a complex congested social environment without collision is a crucial and challenging task. Recent studies have demonstrated the considerable success of Deep Reinforcement Learning (DRL) in multi-agent collision avoidance. However, the assumption of these studies that agents are homogeneous circles deviates from reality, leading to performance deterioration in congested scenarios. The current work extends the DRL-based approaches to develop a collision avoidance method for congested scenarios wherein the heterogeneity of agents can no longer be disregarded. Considering shape heterogeneity, we use the Orientated Bounding Capsule (OBC) to model the agents and transform the interactive state space of Robot-Obstacle agent pair. For speed heterogeneity, we design a velocity-related collision risk function to shape the behavior of the robot. Experimental results demonstrate that our proposed method outperforms state-of-the-art DRL-based approaches in terms of success rate and safety. It also exhibits desired collision avoidance behavior.

Multi-Agent Collision Avoidance System Based on Centralization and Decentralization Control for UAV Applications

A collision avoidance method for multi-agent systems based on the centralization and decentralization effects for cooperative control is presented in this article. In this context, a matrix called Anti-Laplacian is proposed to control the agents when the UAVs are on a conflicting route. The matrix adjustment occurs through proportionality relations with the Laplacian Matrix from an interaction graph between the agents. The adjustment method aims a balance between centering and decentering to avoid collisions. Controlled quadcopters follow the trajectory indicated by virtual agents that act as guides for the real ones. The tests are performed via simulation for the most critical cases, with protocols involving flock centralization. As a virtual agent, a first-order model is used in the simulation, the method efficiency is observed by varying the number of agents involved. The proposed method presents allows different forms to control the collision avoidance parameter, such as deterministic and machine learning methods.

Distributed multi-agent collision avoidance using robust differential game

This paper proposes a novel robust differential game scheme to solve the collision avoidance problem for networked multi-agent systems (MASs), subject to linear dynamics, external disturbances and limited observation capabilities. Compared with the existing differential game approaches only considering obstacle avoidance objectives, we explicitly incorporate the trajectory optimization target by penalizing the deviation from reference trajectories, based on the artificial potential field (APF) concept. It is proved that the strategies of each agent defined by individual optimization problems will converge to a local robust Nash equilibrium (R-NE), which further, with a fixed strong connection topology, will converge to the global R-NE. Additionally, to cope with the limited observation for MASs, local robust feedback control strategies are constructed based on the best approximate cost function and distributed robust Hamilton–Jacobi–Isaacs (DR-HJI) equations, which does not require global information of agents as in the traditional Riccati equation form. The feedback gains of the control strategies are found via the ant colony optimization (ACO) algorithm with a non-dominant sorting structure with convergence guarantees. Finally, simulation results are provided to verify the efficacy and robustness of the novel scheme. The agents arrived at the targeted position collision-free with a reduced arrival time, and reached the targeted positions under disturbance.

A Distributed Multi-Agent Formation Control Method Based on Deep Q Learning.

Distributed control method plays an important role in the formation of a multi-agent system (MAS), which is the prerequisite for an MAS to complete its missions. However, the lack of considering the collision risk between agents makes many distributed formation control methods lose practicability. In this article, a distributed formation control method that takes collision avoidance into account is proposed. At first, the MAS formation control problem can be divided into pair-wise unit formation problems where each agent moves to the expected position and only needs to avoid one obstacle. Then, a deep Q network (DQN) is applied to model the agent's unit controller for this pair-wise unit formation. The DQN controller is trained by using reshaped reward function and prioritized experience replay. The agents in MAS formation share the same unit DQN controller but get different commands due to various observations. Finally, through the min-max fusion of value functions of the DQN controller, the agent can always respond to the most dangerous avoidance. In this way, we get an easy-to-train multi-agent collision avoidance formation control method. In the end, unit formation simulation and multi-agent formation simulation results are presented to verify our method.

Multi-Agent Planning with Mixed-Integer Programming and Adaptive Interaction Constraint Generation (Extended Abstract)

We consider multi-agent planning in which the agents' optimal plans are solutions to mixed-integer programs (MIP) that are coupled via integer constraints. While in principle, one could find the joint solution by combining the separate problems into one large joint centralized MIP, this approach rapidly becomes intractable for growing numbers of agents and large problem domains. To address this issue, we propose an iterative approach that combines conflict detection with constraint-generation whereby the agents plan repeatedly until all conflicts are resolved. In each planning iteration, the agents plan with as few other agents and interaction-constraints as possible. This yields an optimal method that can reduce computation markedly. We test our approach in the context of multi-agent collision avoidance in graphs with indivisible flows. Our initial simulations on randomized graph routing problems confirm predicted optimality and reduced computational effort.

Control barrier functions for stochastic systems

Control Barrier Functions (CBFs) aim to ensure safety by constraining the control input at each time step so that the system state remains within a desired safe region. This paper presents a framework for CBFs in stochastic systems in the presence of Gaussian process and measurement noise. We first consider the case where the system state is known at each time step, and present reciprocal and zero CBF constructions that guarantee safety with probability 1. We extend our results to high relative degree systems and present both general constructions and the special case of linear dynamics and affine safety constraints. We then develop CBFs for incomplete state information environments, in which the state must be estimated using sensors that are corrupted by Gaussian noise. We prove that our proposed CBF ensures safety with probability 1 when the state estimate is within a given bound of the true state, which can be achieved using an Extended Kalman Filter when the system is linear or the process and measurement noise are sufficiently small. We propose control policies that combine these CBFs with Control Lyapunov Functions in order to jointly ensure safety and stochastic stability. Our results are validated via numerical study on a multi-agent collision avoidance scenario.

Adaptation to Other Agent’s Behavior Using Meta-Strategy Learning by Collision Avoidance Simulation

In human’s cooperative behavior, there are two strategies: a passive behavioral strategy based on others’ behaviors and an active behavioral strategy based on the objective-first. However, it is not clear how to acquire a meta-strategy to switch those strategies. The purpose of the proposed study is to create agents with the meta-strategy and to enable complex behavioral choices with a high degree of coordination. In this study, we have experimented by using multi-agent collision avoidance simulations as an example of cooperative tasks. In the experiments, we have used reinforcement learning to obtain an active strategy and a passive strategy by rewarding the interaction with agents facing each other. Furthermore, we have examined and verified the meta-strategy in situations with opponent’s strategy switched.

A Hybrid Controller for Multi-Agent Collision Avoidance via a Differential Game Formulation

We consider the multi-agent collision avoidance problem for a team of wheeled mobile robots. Recently, a local solution to this problem, based on a game-theoretic formulation, has been provided and validated via numerical simulations. Due to its local nature, the result is not well-suited for online applications. In this article, we propose a novel hybrid implementation of the control inputs that yields a control strategy suited for the online navigation of mobile robots. Moreover, subject to a certain dwell time condition, the resulting trajectories are globally convergent. The control design is demonstrated both via simulations and experiments.

Interval Analysis Technique for Versatile and Parallel Multi-Agent Collision Detection and Avoidance

Collision detection and avoidance are vital sub-tasks in any autonomous robotic task. This work presents a technique that guarantees collision avoidance in an environment containing multiple agents. These agents can be of different types such as static, dynamic obstacles and other robots. The technique supports parallel implementation and is appropriate for real-time applications. In this work, the theory of interval arithmetic is used to represent the pose of agents as intervals in a fixed time period. Geometrically, the intervals correspond to finite-length arcs and line-segments. Theoretical results on the inclusion of one interval, in another interval, in terms of sub-intervals are derived. Breaking down the problem in sub-intervals supports parallelism in performing multiple interval inclusion tests and in handling multiple agents. The proposed interval-arithmetic based framework leads directly to a hardware-efficient collision detection scheme. In particular, the proposed strategy admits a solution even for a dynamic environment using just shift and add capability, an important aspect for embedded implementation. The solution of interval inclusion is also used to find a set of solutions for guaranteed collision avoidance with multiple agents with known or unknown trajectories. Simulation results in MATLAB and experiments with an FPGA-driven differential drive mobile robot demonstrate the versatility of the proposed approach.

Velocity Obstacle Approaches for Multi-Agent Collision Avoidance

This paper presents a critical analysis of some of the most promising approaches to geometric collision avoidance in multi-agent systems, namely, the velocity obstacle (VO), reciprocal velocity obstacle (RVO), hybrid-reciprocal velocity obstacle (HRVO) and optimal reciprocal collision avoidance (ORCA) approaches. Each approach is evaluated with respect to increasing agent populations and variable sensing assumptions. In implementing the localized avoidance problem, the author notes a problem of symmetry not considered in the literature. An intensive 1000-cycle Monte Carlo analysis is used to assess the performance of the selected algorithms in the presented conditions. The ORCA method is shown to yield the most scalable computation times and collision likelihood in the presented cases. The HRVO method is shown to be superior than the other methods in dealing with obstacle trajectory uncertainty for the purposes of collision avoidance. The respective features and limitations of each algorithm are discussed and presented through examples.

On the relationship between dynamics and complexity in multi-agent collision avoidance

This work examines how dynamics and complexity are related in multi-agent collision avoidance. Motivated particularly by work in the field of automated driving, this work considers a variant of the reciprocal n-body collision avoidance problem. In this problem, agents must avoid collision while moving according to individual reward functions in a crowded environment. The main contribution of this work is the result that there is a quantifiable relationship between system dynamics and the requirement for agent coordination, and that this requirement can change the complexity class of the problem dramatically: from P to NEXP or even $$\hbox {NEXP}^{\text {NP}}$$ . A constructive proof is provided that demonstrates the relationship, and potential practical applications are discussed.

Space-based Collision Avoidance Framework for Autonomous Vehicles

High confidence in the safe operation of autonomous systems remains a critical hurdle on their path to becoming ubiquitous. Recent accidents of Uber and Google driverless cars illustrate the difficulty ahead. Leading collision avoidance framework for autonomous systems fail to properly capture and account for the high variability of geometries, shapes, and sizes of the agents (e.g., 18 wheels truck vs. 4 doors sedan), capabilities that are critical in situations with high risk of accident (e.g., intersection crossing). We introduce a simple and efficient multi-agent collision avoidance framework for Autonomous Vehicles (AV) in various collision configurations (i.e., glancing, away, clipping). Machine learning techniques are proposed to properly train the autonomous systems involved. Vehicle-to-Vehicle (V2V) communication technologies and shape-based spatial-temporal collision avoidance algorithms are leveraged to ensure the accurate prediction of the collision and correct decision on the appropriate steps to avoid its occurrence. A prototype implementation and simulation is currently under development for a clipping collision problem at a lightless intersection crossing using the AirSim platform.

Autonomous collision avoidance for wheeled mobile robots using a differential game approach

A multi-agent system consisting of N wheeled mobile robots is considered. The robots are modeled by unicycle dynamics and the multi-agent collision avoidance problem, which lies in steering each robot from its initial position to a desired target position while avoiding collisions with obstacles and other agents is considered. The problem is solved in two steps. First, exploiting a differential game formulation, collision-free trajectories are generated for virtual agents satisfying single-integrator dynamics. Second, the previous step is used to construct dynamic feedback strategies for the wheeled mobile robots satisfying unicycle dynamics which ensure each of the robots reaches its target without collisions occurring. A numerical study of the proposed methodology is provided through a series of simulations.

A Differential Game Approach to Multi-agent Collision Avoidance

A multi-agent system consisting of $N$ agents is considered. The problem of steering each agent from its initial position to a desired goal while avoiding collisions with obstacles and other agents is studied. This problem, referred to as the multi-agent collision avoidance problem , is formulated as a differential game. Dynamic feedback strategies that approximate the feedback Nash equilibrium solutions of the differential game are constructed and it is shown that, provided certain assumptions are satisfied, these guarantee that the agents reach their targets while avoiding collisions.

Deep-Learned Collision Avoidance Policy for Distributed Multiagent Navigation

High-speed, low-latency obstacle avoidance that is insensitive to sensor noise is essential for enabling multiple decentralized robots to function reliably in cluttered and dynamic environments. While other distributed multiagent collision avoidance systems exist, these systems require online geometric optimization where tedious parameter tuning and perfect sensing are necessary. We present a novel end-to-end framework to generate reactive collision avoidance policy for efficient distributed multiagent navigation. Our method formulates an agent's navigation strategy as a deep neural network mapping from the observed noisy sensor measurements to the agent's steering commands in terms of movement velocity. We train the network on a large number of frames of collision avoidance data collected by repeatedly running a multiagent simulator with different parameter settings. We validate the learned deep neural network policy in a set of simulated and real scenarios with noisy measurements and demonstrate that our method is able to generate a robust navigation strategy that is insensitive to imperfect sensing and works reliably in all situations. We also show that our method can be well generalized to scenarios that do not appear in our training data, including scenes with static obstacles and agents with different sizes. Videos are available at https://sites.google.com/view/deepmaca.