Optimal Reciprocal Collision Avoidance Research Articles

USV formation navigation decision-making through hybrid deep reinforcement learning using self-attention mechanism

To address the challenging of balancing Unmanned Surface Vessel (USV) autonomous collision avoidance and formation maintenance in uncertain environments, a formation construction and navigation decision-making strategy based on Hybrid Deep Reinforcement Learning (HDRL) is proposed in this study. The novelty of this study is that: (1) A HDRL training approach is proposed, incorporating diverse DRL for virtual leader and followers, thereby significantly enhancing the decision-making adaptability of USVs formation. (2) The multi-head attention mechanism of decentralized Critic strategy is used to enhance the attentional focus of the HDRL algorithm towards different agents, thereby effectively improving convergence speed. (3) The method employs a meticulously designed hybrid reward function and incorporates the Optimal Reciprocal Collision Avoidance (ORCA) for speed selection, thereby providing optimal speed recommendations based on the current situation. It is worth emphasizing that the method exhibits exceptional performance in terms of success rate, navigation time, average reward value, and other relevant indicators within the simulation. Finally, the method is validated through the utilization of real-time navigation data in the Panama Canal, thereby substantiating the potential engineering applicability.

Relaxing the Limitations of the Optimal Reciprocal Collision Avoidance Algorithm for Mobile Robots in Crowds

Relaxing the Limitations of the Optimal Reciprocal Collision Avoidance Algorithm for Mobile Robots in Crowds

Collision avoidance control for limited perception unmanned surface vehicle swarm based on proximal policy optimization

In order to ensure the safe and coordinated operation of unmanned surface vehicle (USV) swarm in complex marine environments, the primary problem is collision avoidance control (CAC). However, the limited perception, environmental uncertainty and multi-source complexity bring significant challenges to the efficient collaboration and CAC of the USV swarm. To overcome the above challenges, this paper aims to propose a distributed CAC method for USVs based on the proximal policy optimization (PPO). This method does not necessitate precise system models and is capable of autonomous learning, effectively adapting to unknown environments. In terms of CAC, unlike designing reward functions based solely on the distance from obstacles, we additionally consider the velocity of obstacles, and combine optimal reciprocal collision avoidance (ORCA) to design a reward function. We further consider the limited perception range of USVs and construct a bidirectional gated recurrent unit (BiGRU) network to extract features of variable length observation data, effectively overcoming the problem of dimensionality in observable data. Moreover, we construct a high-quality USV swarm simulation environment using the Gazebo 3D physics engine, which is used for testing the generalization capability of collision avoidance policy. Finally, to verify the effectiveness of the policy learning and optimization, a series of experiments are conducted in various scenarios, network architectures, and control methods. The experimental results indicate that our approach has remarkable superiority in terms of travel time, average velocity, average reward, and success rate.

Collision-Free Formation Control With Global Network Integrity Maintenance via Preserving Induced-Subgraph Connectivity

Network connectivity is a necessary prerequisite to realizing distributed formation control. However, it is challenging to maintain the global network connectivity of a formation with local information because the communications between topological agents may be disconnected as formation maneuvers due to the range-limited sensing capabilities of agents. In this article, we propose a novel connectivity-preserving formation strategy to address the challenge. The core idea is to decompose global connectivity constraint into the induced-subgraph-preserving problems of each agent. Different from the state-of-the-art methods, our strategy can acquire the local algebraic connectivity directly at the level of maximal cliques, instead of estimating the global algebraic connectivity by hundreds of iterations. Consequently, it yields more flexibility in formation maneuvers and requires lower computing costs. Moreover, we treat the entire formation as a single rigid body and employ a variant of the optimal reciprocal collision avoidance algorithm to optimize the overall movement of the formation so that agents can maintain the desired shape while avoiding obstacles. Here, the position and orientation of the desired formation are negotiated by the agents using only local peer-to-peer information. Simulation results verify that the proposed strategy can drive a group of agents to implement connectivity-preserving formations in obstacle-rich spaces.

A deep multi-agent reinforcement learning framework for autonomous aerial navigation to grasping points on loads

Deep reinforcement learning, by taking advantage of neural networks, has made great strides in the continuous control of robots. However, in scenarios where multiple robots are required to collaborate with each other to accomplish a task, it is still challenging to build an efficient and scalable multi-agent control system due to increasing complexity. In this paper, we regard each unmanned aerial vehicle (UAV) with its manipulator as one agent, and leverage the power of multi-agent deep deterministic policy gradient (MADDPG) for the cooperative navigation and manipulation of a load. We propose solutions for addressing navigation to grasping point problem in targeted and flexible scenarios, and mainly focus on how to develop model-free policies for the UAVs without relying on a trajectory planner. To overcome the challenges of learning in scenarios with an increasing number of grasping points, we incorporate the demonstrations from an Optimal Reciprocal Collision Avoidance (ORCA) algorithm into our framework to guide the policy training and adapt two novel techniques into the architecture of MADDPG. Furthermore, curriculum learning with the attention mechanism is utilized by reusing knowledge from fewer grasping points to facilitate the training of a load with more points. Our experiments were validated by a load with three, four and six grasping points respectively in Coppeliasim simulator and then transferred into the real world with Crazyflie quadrotors. Our results show that the average tracking deviations from the desirable grasping point to the final position of the UAV can be less than 10 cm in some real-world experiments. Compared with state-of-the-art model-free reinforcement learning and swarm optimization algorithms, results show that our proposed methods outperform other baselines with a reasonable success rate especially in the scenarios with more grasping points. Furthermore, the learned optimal policies enable UAVs to reach and hover over all the grasping points before manipulation without any collision. We conducted a comprehensive analysis of both targeted and flexible navigation, highlighting their respective advantages and disadvantages.

FREE-RDW: A Multi-user Redirected Walking Method for Supporting Non-forward Steps.

Multi-user redirected walking (RDW) is widely used in large-scale virtual scenes because it allows more users to move synchronously in both virtual and physical environments. To ensure the freedom of virtual roaming, which can be used in various situations, some redirected algorithms have been dedicated to non-forward movements, such as vertical movement and jumping. However, the existing RDW methods still mainly focus on forward steps, ignoring sideward and backward steps, which are also common and necessary in virtual reality. RDW algorithms for non-forward steps can enrich the movement direction of users' virtual roaming and improve the realism of VR roaming. In addition, the non-forward motions have a larger curvature gain, which can be used to better reduce resets in RDW. Therefore, this paper presents a new method of multi-user redirected walking for supporting non-forward steps (FREE-RDW), which adds the options of sideward and backward steps to extend the VR locomotion. Our method adopts a user collision avoidance strategy based on optimal reciprocal collision avoidance (ORCA) and optimizes it into a linear programming problem to obtain the optimal velocity for users. Furthermore, our method uses APF to expose the user to repulsive forces from other users and walls, thus further reducing potential collisions and improving the utilization of physical space. The experiments show that our method performs well in virtual scenes with forward and non-forward steps. In addition, our method can significantly reduce the number of resets compared with reactive RDW algorithms such as DDB-RDW and APF-RDW in multi-user forward-step virtual scenes.

Dual-Horizon Reciprocal Collision Avoidance for Aircraft and Unmanned Aerial Systems

The aircraft conflict detection and resolution problem has been addressed with a wide range of centralised methods in the past few decades, e.g. constraint programming, mathematical programming or metaheuristics. In the context of autonomous, decentralized collision avoidance without explicit coordination, geometric methods provide an elegant, cost-effective approach to avoid collisions between mobile agents, provided they all share a same logic and a same view of the traffic. The Optimal Reciprocal Collision Avoidance (ORCA) algorithm is a state-of-the art geometric method for robot collision avoidance, which can be used as a Detect & Avoid logic on-board aircraft or Unmanned Aerial Vehicles. However, ORCA does not handle well some degenerate situations where agents operate at constant or near-constant speeds, which is a widespread feature of commercial aircraft or fixed-winged Unmanned Airborne Systems. In such degenerate situations, pairs of aircraft could end up flying parallel tracks without ever crossing paths to reach their respective destination. The Constant Speed ORCA (CS-ORCA) was proposed in 2018 to better handle these situations. In this paper, we discuss the limitations of both ORCA and CS-ORCA, and introduce the Dual-Horizon ORCA (DH-ORCA) algorithm, where two time horizons are used respectively for short-term collision avoidance and medium-term path-crossing. We show that this new approach mitigates the main issues of ORCA and CS-ORCA and yields better performances with dense traffic scenarios.

Data driven analysis of the desired speed in ordinary differential equation based pedestrian simulation models

In this paper, the desired speed is figured out as an important parameter for ordinary differential equation (ODE) based pedestrian simulation models. However, there lacks a thorough study on the optimal way of assigning the desired speed for ODE based pedestrian simulation models to achieve reliable performance. To gain a deep understanding of the role of the desired speed and its influence on the performances of the ODE based pedestrian simulation models, a total of nine assigning strategies of the desired speed are conducted a comparison study. Specifically, the performances of the ODE based models equipped with such desired speed assigning strategies are compared with regard to reference data from both the uni- and bi-directional pedestrian flow scenarios using a fair model comparison framework. Then, the performances are compared across several dimensions, including different assigning strategies of the desired speed, the different pedestrian simulation models, different experiments for the same motion base, and different motion base scenarios. Based on these results, four findings are summarized. Firstly, the desired speed is quantitatively verified to be an important role in the performance of ODE based pedestrian simulation models. Secondly, pedestrian density level is verified to be an effective indicator for the complexity of pedestrian crowd dynamics. Thirdly, the performance of the ODE based models is dependent on the extent to which the heterogeneity of desired speed is considered. Lastly, the performance of the social force model (SF) is suggested to be more predictable than optimal reciprocal collision avoidance model (ORCA) regarding the stability and optimality. Finally, based on the above findings, recommendations on assigning the desired speed for ODE based pedestrian simulation models are given.

COLREGs-Compliant Multi-Ship Collision Avoidance Based on Multi-Agent Reinforcement Learning Technique

The congestion of waterways can easily lead to traffic hazards. Moreover, according to the data, the majority of sea collisions are caused by human error and the failure to comply with the Convention on the International Regulation for the preventing Collision at Sea (COLREGs). To avoid this situation, ship automatic collision avoidance has become one of the most important research issues in the field of marine engineering. In this study, an efficient method is proposed to solve multi-ship collision avoidance problems based on the multi-agent reinforcement learning (MARL) algorithm. Firstly, the COLREGs and ship maneuverability are considered for achieving multi-ship collision avoidance. Subsequently, the Optimal Reciprocal Collision Avoidance (ORCA) algorithm is utilized to detect and reduce the risk of collision. Ships can operate at the safe velocity computed by the ORCA algorithm to avoid collisions. Finally, the Nomoto three-degrees-of-freedom (3-DOF) model is used to simulate the maneuvers of ships. According to the above information and algorithms, this study designs and improves the state space, action space and reward function. For validating the effectiveness of the method, this study designs various simulation scenarios with thorough performance evaluations. The simulation results indicate that the proposed method is flexible and scalable in solving multi-ship collision avoidance, complying with COLREGs in various scenarios.

A Fast Markov Decision Process-Based Algorithm for Collision Avoidance in Urban Air Mobility

Multiple aircraft collision avoidance is a challenging problem due to a stochastic environment and uncertainty in the intent of other aircraft. Traditionally a layered approach to collision avoidance has been employed using a centralized air traffic control system, established rules of the road, separation assurance, and last minute pairwise collision avoidance. With the advent of Urban Air Mobility (air taxis), the expected increase in traffic density in urban environments, short time scales, and small distances between aircraft favor decentralized decision making on-board the aircraft. In this paper, we present a Markov Decision Process (MDP) based method, named FastMDP, which can solve a certain subclass of MDPs quickly, and demonstrate using the algorithm online to safely maintain separation and avoid collisions with multiple aircraft (1-on-n) while remaining computationally efficient. We compare the FastMDP algorithm’s performance against two online collision avoidance algorithms that have been shown to be both efficient and scale to large numbers of aircraft: Optimal Reciprocal Collision Avoidance (ORCA) and Monte Carlo Tree Search (MCTS). Our simulation results show that under the assumption that aircraft do not have perfect knowledge of other aircraft intent FastMDP outperforms ORCA and MCTS in collision avoidance behavior in terms of loss of separation and near mid-air collisions while being more computationally efficient. We further show that in our simulation FastMDP behaves nearly as well as MCTS with perfect knowledge of other aircraft intent. Our results show that FastMDP is a promising algorithm for collision avoidance that is also computationally efficient.

ORCANet: Differentiable multi‐parameter learning for crowd simulation

AbstractRealistic crowd simulation has always been an important research field in computer graphics. While both agent‐based motion models and data‐driven behavior models have made some progress, they are still suffering from either huge effort of multi‐parameter tuning or limited realistic motion. In this article, we propose a novel and differentiable multi‐parameter learning method for crowd simulation, which is called ORCANet. The main idea is to learn from real data and inverse evaluating the multi‐parameter for subsequent simulation. ORCANet uses classic optimal reciprocal collision avoidance (ORCA) as a basic motion model which is integrated into the deep learning framework. Addressing the feature of linear programming and non‐differentiable operation, a Gaussian kernel is added to approximate the role of neighbor distance in collision avoidance, which turns the original discrete operation into a fully differentiable forward simulation. Furthermore, we leverage ORCANet to optimize the multi‐parameter combination in synthetic and real‐world datasets. ORCANet is proved to rapidly converge to correct parameter values and regenerate the input synthetic sequence. Moreover, experiments on real‐world datasets by the metric of pedestrian trajectories verified that a more realistic crowd simulation has been generated through ORCANet.

Neighbor Reward with Optimal Reciprocal Collision Avoidance for Swarm Agents*

Navigating in an unknown area safely is counted as the underlying work which can support swarm agents for more complex tasks. When available information of search regions are lacking, agents make real-time action decisions according to surrounding environments they have perceived. For swarm agent system, connectivity maintenance and collision avoidance are both essential. Based on optimal Reciprocal Collision Avoidance (ORCA) algorithm, we proposed a method that agents can provide assistances to surrounding agents by spreading the status information of themselves, which is the neighbor reward method (NRM). This kind of status information contains ambient information and perceptions of the task which are transferred to reward data for convenient and uniform distributions. In other words, individuals utilize inter-neighbor interactions to achieve the same high-level goal, as well as result in an intelligent independent swarm agents system.This method solves the velocity selection problem of ORCA and optimizes the obstacle avoidance of the original NRM. The algorithm has been integrated in ROS framework and simulated on GAZEBO. In the tested scenario, our method is efficient for swarm agents collision avoidance in decentralized way.

Volcanic Ash Region Path Planning Based on Improved A-Star Algorithm

In civil aviation flight path planning, in order to effectively reduce the safety threat caused by the volcanic ash area to the civil aviation flight, factors such as the speed and acceleration of the aircraft in the volcanic ash area must be considered. In this paper, we propose an improved A-star algorithm by adopting the concept of potential collision set and using the velocity vector idea of optimal reciprocal collision avoidance (ORCA). The improved A-star algorithm selects the optimal speed range of the aircraft from the limited elements, obtains the speed and acceleration set of the aircraft in the volcanic ash area, calculates the flight path through the displacement increment, continuously refreshes the global starting point to the target point, and finally obtains the final path planning results by calculating the total cost value. The improved A-star algorithm is used to plan two flight paths from Madrid to Cairo and Algiers to Rome in volcanic ash areas. The verification results show that the improved A-star algorithm optimizes the flight path planning in the volcanic ash area and has the advantages of less search nodes, a small search range, and short computing time.

VR-ORCA: Variable Responsibility Optimal Reciprocal Collision Avoidance

As one of the most popular multi-agent path planning approaches, the optimal reciprocal collision avoidance (ORCA) algorithm assumes that each agent takes half the responsibility for collision avoidance. However, due to the asymmetric situation faced by adjacent agents, they are expected to take different responsibilities for collision avoidance to improve the entire crowd's navigation performance. Thus, in this letter, we propose the variable responsibility optimal reciprocal collision avoidance (VR-ORCA) algorithm, which relaxes the original assumption in ORCA and only requires the responsibilities of a pair of agents sum to one. In particular, the responsibility division between a pair of nearby agents is determined independently for each agent by minimizing a cost function involving their common neighbors. We validate our approach on a variety of simulated benchmarks and the results demonstrate that our novel method is beneficial in reducing the collision probability, travel time and distance of ORCA.

Discrete Pigeon-Inspired Optimization-Simulated Annealing Algorithm and Optimal Reciprocal Collision Avoidance Scheme for Fixed-Wing UAV Formation Assembly

Multi-UAV system is an important part of unmanned system, which plays an indispensable role in military field and civil agriculture. First, task assignment model including complex constraints is established and Discrete Pigeon-Inspired Optimization-Simulated Annealing algorithm (DPIO-SA) is proposed to solve it, which updates the speed and position of pigeons through exchange and cross operations. Then, Genetic Algorithm (GA) is adopted to optimize the UAVs’ locations in formation. The experimental results show that the average fitness value of DPIO-SA is 13.5% higher than DPIO; After running the algorithm for 30 times, the number of times that DPIO-SA algorithm reaches the global optimum is 15, while DPIO is 2. Both mean DPIO-SA is easier to jump out of local extremum. To describe the fixed-wing UAV, the Unicycle model is adopted. PID control is used to control the fixed-wing’s heading and speed. Aiming at the collision avoidance, Optimal Reciprocal Collision Avoidance (ORCA) algorithm is proposed, which allows fixed-wings to avoid collisions without having to communicate with each other. In the algorithm, the velocity region is divided by the definition of velocity obstacle, and the optimal velocity is obtained by linear planning algorithm. This enables the fixed-wings’ formation to find the right velocity in real time and effectively to avoid collision. Experiments show that 24 fixed-wings completed the formation assembly after running 16.4 s. Finally, according to the Contract Network Algorithm (CNA), the task scheduling problem is solved by the interaction among fixed-wings.

Autonomous Free Flight Operations in Urban Air Mobility With Computational Guidance and Collision Avoidance

The use of electrical vertical takeoff and landing (eVTOL) aircraft to provide efficient, high-speed, on-demand air transportation within a metropolitan area is a topic of increasing interest, which is expected to bring fundamental changes to the city infrastructures and daily commutes. NASA, Uber, and Airbus have been exploring this exciting concept of Urban Air Mobility (UAM), which has the potential to provide meaningful door-to-door trip time savings compared with automobiles. However, the ability to manage many of these eVTOL aircraft safely in a congested urban area presents a challenge unprecedented in air traffic management. In order to enable safe and efficient autonomous on-demand free flight operations in UAM, a computational guidance algorithm with collision avoidance capability is designed and analyzed. The approach proposed in this paper is to formulate this problem as a Markov Decision Process (MDP) and solve it using an online algorithm Monte Carlo Tree Search (MCTS). For illustration, a high-density free flight airspace simulator is created to test the proposed algorithm's performance. Numerical experiment results show that this proposed algorithm has fewer conflicts and near mid-air collisions than Optimal Reciprocal Collision Avoidance (ORCA), a state-of-the-art collision avoidance strategy.

Directional optimal reciprocal collision avoidance

A great amount of effort has been devoted to the study on self-separation assurance approach for civil aviation in the airspace with increasing density. In this article, the Optimal Reciprocal Collision Avoidance (ORCA) algorithm is modified to make it work for autonomous and decentralized collision avoidance for civil aircraft. Without considering the direction selectivity of collision-free maneuver, aircraft may select the relative parallel trajectories by deploying the ORCA algorithm in both decentralized and centralized way. As a result, the collision tends to be postponed to the next time horizon because civil aircraft need to return to original trajectories. Simultaneously, the unified rules can hardly be integrated into the approach due to the lack of direction selectivity for collision-free navigation. The process of separation assurance will be disorderly when multiple aircraft are involved. To solve the problem mentioned above, a new algorithm called Directional Optimal Reciprocal Collision Avoidance (DORCA) is proposed. The DORCA algorithm employs a vector rotation mode to construct the forbidden Velocity Obstacle (VO) set in order to improve the computation efficiency. In addition, the direction selectivity of maneuver is achieved through constructing the direction-constrained VO set according to the direction of relative motion in velocity space. Direction selectivity of the algorithm enables the process of collision avoidance to comply with the unified rules. A number of encounter scenarios are conducted to confirm the validity and feasibility of the proposed DORCA algorithm. In all scenarios tested, the direction selectivity of collision-free maneuver can be successfully integrated into the DORCA algorithm, and the algorithm is more efficient than the ORCA algorithm for collision avoidance in decentralized way.

Route choice in the pedestrian evacuation: Microscopic formulation based on visual information

A pedestrian’s visual field constantly changes as they move around a room containing structural components or large furniture, which can block the pedestrian from perceiving visual information. Changes in position simultaneously update optional route sets and the perception of how crowded these paths are. In this study, a microscopic pedestrian–simulation model with continuous-space representation is developed to investigate the route-choice behavior of pedestrians in a space where obstacles block the visual field. Instead of potential-based navigation, the desired direction of pedestrians during the motion is expressed by an oriented network and delivered to the navigation layer: an optimal reciprocal collision avoidance (ORCA) system, which models pedestrians’ local maneuvers and movement decisions in detail. We quantify the characteristics of the model by comparing the model results with two groups of experiments conducted under conditions of good, limited, and zero visibility. The performance results with two different layouts are compared through numerical simulation, illustrating the model’s capability to accurately represent the observed crowd dynamics in the facility. Results from the sensitivity analysis and simulation scenario can benefit the evacuation guidance arrangement and internal component design in large public spaces.

Coordinating multiple mobile robots for obstacle avoidance using cloud computing

AbstractThis study presents a motion planning method for coordinating multiple mobile robots for collision‐free navigation. A control architecture that uses cloud computing is proposed for the acquisition of real‐time robotic data and to coordinate multiple robots in an unstructured environment. The system consists of collision‐free path planning for multiple robots, obstacle avoidance, and navigation control. The path planning method was developed based on the optimal reciprocal collision avoidance (ORCA) algorithm to generate a velocity set of each robot in the system. A laser scanner is used to detect obstacles for each robot. Appropriate obstacle avoidance behaviors are generated using sensory information. A behavior‐fusion control scheme combines obstacle avoidance and collision‐free path planning to coordinate multiple robots. The proposed anti‐collision motion planning method for multiple robots prevents collisions with unexpected obstacles and with other robots in the environment. The experimental results show that multiple mobile robots can navigate to targets that are assigned by the cloud server without colliding with other robots, regardless of whether there are unexpected static or dynamic obstacles in the environment.

A decentralized method for collision detection and avoidance applied to civil aircraft

With the increasing density level of airspace, the flawed logic of resolution in air conflict has become a potential hazard to keep flight safety for civil aviation. A powerful decision-support system is needed to identify and resolve potential conflicts on planned trajectory in advance. Existing studies on this subject mainly focus on the centralized means, but seldom consider the decentralized approaches. In this paper, a decentralized method is proposed so that each aircraft can generate the collision-free Reference Business Trajectory (RBT) autonomously, and resolve potential conflicts while conforming to the unified rules. Firstly, a Synchronous Discrete-Time-Discrete-Space trajectory modeling is developed to divide the continuous planned trajectory into multiple trajectory segments according to motion state. Thus, the collision can be accurately located at one certain risky segment, and the corresponding collision time can be acquired precisely. Through a weight analysis of collision time, the critical trajectory segment is determined to implement the task of conflict resolution. Then, the Optimal Reciprocal Collision Avoidance (ORCA) algorithm is adopted and extended to determine the collision-free maneuver with the consideration of direction selectivity. At last, the Trajectory Change Points (TCPs) are achieved by the quadratic program for each aircraft. The proposed method can help aircraft generate collision-free RBT in decentralized way successfully. Several simulations are conducted to confirm the validity and efficiency of the proposed approach.