Sampling-based Planning Research Articles

Sampling-Based Robot Motion Planning: A Review

Motion planning is a fundamental research area in robotics. Sampling-based methods offer an efficient solution for what is otherwise a rather challenging dilemma of path planning. Consequently, these methods have been extended further away from basic robot planning into further difficult scenarios and diverse applications. A comprehensive survey of the growing body of work in sampling-based planning is given here. Simulations are executed to evaluate some of the proposed planners and highlight some of the implementation details that are often left unspecified. An emphasis is placed on contemporary research directions in this field. We address planners that tackle current issues in robotics. For instance, real-life kinodynamic planning, optimal planning, replanning in dynamic environments, and planning under uncertainty are discussed. The aim of this paper is to survey the state of the art in motion planning and to assess selected planners, examine implementation details and above all shed a light on the current challenges in motion planning and the promising approaches that will potentially overcome those problems.

A Survey on Randomized Sampling-Based Path Optimization Methods

Randomized sampling-based motion planners could efficiently generate collision-free motion paths. However, these paths have some quality problems (with respect to quality measures, such as path length, clearance and smoothness), especially in high dimensional configuration spaces. Thus, researchers studied some path optimization methods to improve path quality and to make paths suitable for practical applications. This paper reviews some of the most influential path optimization methods and gives an overall perspective on the most widely used ideas in the field.

Open-Loop Planning in Large-Scale Stochastic Domains

We focus on effective sample-based planning in the face of underactuation, high-dimensionality, drift, discrete system changes, and stochasticity. These are hallmark challenges for important problems, such as humanoid locomotion. In order to ensure broad applicability, we assume domain expertise is minimal and limited to a generative model. In order to make the method responsive, computational costs that scale linearly with the amount of samples taken from the generative model are required. We bring to bear a concrete method that satisfies all these requirements; it is a receding-horizon open-loop planner that employs cross-entropy optimization for policy construction. In simulation, we empirically demonstrate near-optimal decisions in a small domain and effective locomotion in several challenging humanoid control tasks.

An Efficient Path Replanning Technique for EST-Based Robot Navigation in Dynamic Environments

Expansive-Spaces Trees (EST) is a single-query sampling-based path planner. When EST is applied to robot navigation in dynamic environments, EST confronts the risk of a collision with dynamic obstacles since it has generated a path without any consideration for dynamic obstacles. This paper proposes an efficient path replanning technique for EST-based robot navigation in dynamic environments. The proposed technique replans a collision-free and efficient path instead of the original EST path which may cause a collision with dynamic obstacles. Besides, the replanned path can be easily merged into the original EST path because it preserves the property of EST. Simulation results in various dynamic environments reveal that the proposed technique can successfully replan a collision-free and efficient path for EST-based navigation.

A Low Dispersion Probabilistic Roadmaps (LD-PRM) Algorithm for Fast and Efficient Sampling-Based Motion Planning

In this paper, we propose a new learning strategy for a probabilistic roadmap (PRM) algorithm. The proposed strategy is based on reducing the dispersion of the generated set of samples. We defined a forbidden range around each selected sample and ignored this region in further sampling. The resultant planner, called low dispersion-PRM, is an effective multi-query sampling-based planner that is able to solve motion planning queries with smaller graphs. Simulation results indicated that the proposed planner improved the performance of the original PRM and other low-dispersion variants of PRM. Furthermore, the proposed planner is able to solve difficult motion planning instances, including narrow passages and bug traps, which represent particularly difficult tasks for classic sampling-based algorithms. For measuring the uniformity of the generated samples, a new algorithm was created to measure the dispersion of a set of samples based on a predetermined resolution.

TEXPLORE: real-time sample-efficient reinforcement learning for robots

The use of robots in society could be expanded by using reinforcement learning (RL) to allow robots to learn and adapt to new situations online. RL is a paradigm for learning sequential decision making tasks, usually formulated as a Markov Decision Process (MDP). For an RL algorithm to be practical for robotic control tasks, it must learn in very few samples, while continually taking actions in real-time. In addition, the algorithm must learn efficiently in the face of noise, sensor/actuator delays, and continuous state features. In this article, we present texplore, the first algorithm to address all of these challenges together. texplore is a model-based RL method that learns a random forest model of the domain which generalizes dynamics to unseen states. The agent explores states that are promising for the final policy, while ignoring states that do not appear promising. With sample-based planning and a novel parallel architecture, texplore can select actions continually in real-time whenever necessary. We empirically evaluate the importance of each component of texplore in isolation and then demonstrate the complete algorithm learning to control the velocity of an autonomous vehicle in real-time.

Grafting: A Path Replanning Technique for Rapidly-Exploring Random Trees in Dynamic Environments

The rapidly-exploring random trees (RRT) is a sampling-based path planner which utilizes simultaneously kinematics and dynamics of a robot. However, since the RRT has produced a robot path without taking the existence of dynamic obstacles into consideration, RRT-based navigation has the risk of a collision with dynamic obstacles. We proposed a path replanning technique for the RRT applied to robot navigation in dynamic environments, which is named grafting. The proposed technique replans a safe and efficient path in real time instead of the original path which may cause a collision with dynamic obstacles. Moreover, the replanned path can be easily merged into the original RRT path because the grafting technique preserves the property of the RRT. The grafting technique was tested by simulations in various dynamic environments, which revealed that the grafting technique was capable of replanning a safe and efficient path for RRT-based navigation in real time.

Adaptive sampling for generalized probabilistic roadmaps

In this paper, an adaptive sampling strategy is presented for the generalized sampling-based motion planner, generalized probabilistic roadmap (GPRM). These planners are designed to account for stochastic map and model uncertainty and provide a feedback solution to the motion planning problem. Intelligently sampling in this framework can result in large speedups when compared to naive uniform sampling. By using the information of transition probabilities, encoded in these generalized planners, the proposed strategy biases sampling to improve the efficiency of sampling, and increase the overall success probability of GPRM. The strategy is used to solve the motion planning problem of a fully actuated point robot and a 3-DOF fixed-base manipulator on several maps of varying difficulty levels, and results show that the strategy helps solve the problem efficiently, while simultaneously increasing the success probability of the solution. Results also indicate that these rewards increase with an increase in map complexity.

Task Space Regions

We present a manipulation planning framework that allows robots to plan in the presence of constraints on end-effector pose, as well as other common constraints. The framework has three main components: constraint representation, constraint-satisfaction strategies, and a general planning algorithm. These components come together to create an efficient and probabilistically complete manipulation planning algorithm called the Constrained BiDirectional Rapidly-exploring Random Tree (RRT) – CBiRRT2. The underpinning of our framework for pose constraints is our Task Space Regions (TSRs) representation. TSRs are intuitive to specify, can be efficiently sampled, and the distance to a TSR can be evaluated very quickly, making them ideal for sampling-based planning. Most importantly, TSRs are a general representation of pose constraints that can fully describe many practical tasks. For more complex tasks, such as manipulating articulated objects, TSRs can be chained together to create more complex end-effector pose constraints. TSRs can also be intersected, a property that we use to plan with pose uncertainty. We provide a detailed description of our framework, prove probabilistic completeness for our planning approach, and describe several real-world example problems that illustrate the efficiency and versatility of the TSR framework.

Generalized Sampling-Based Motion Planners

In this paper, generalized versions of the probabilistic sampling-based planners, i.e., probabilistic roadmaps and rapidly exploring random tree, are presented. The generalized planners, i.e., generalized probabilistic roadmap and the generalized rapidly exploring random tree, result in hybrid hierarchical feedback planners that are robust to the uncertainties in the robot motion model and in the robot map or workspace. The proposed planners are analyzed and shown to probabilistically be complete. The algorithms are tested on fully actuated and underactuated robots on several maps of varying degrees of difficulty, and the results show that the generalized methods have a significant advantage over the traditional methods when planning under uncertainty.

Aplicación de Muestreo basado en Modelos de Control Predictivo a un Vehículo Autónomo Subacuático

Unmanned Underwater Vehicles (UUVs) can be utilized to perform difficult tasks in cluttered environments such as harbor and port protection. However, since UUVs have nonlinear and highly coupled dynamics, motion planning and control can be difficult when completing complex tasks. Introducing models into the motion planning process can produce paths the vehicle can feasibly traverse. As a result, Sampling-Based Model Predictive Control (SBMPC) is proposed to simultaneously generate control inputs and system trajectories for an autonomous underwater vehicle (AUV). The algorithm combines the benefits of sampling-based motion planning with model predictive control (MPC) while avoiding some of the major pitfalls facing both traditional sampling-based planning algorithms and traditional MPC. The method is based on sampling (i.e., discretizing) the input space at each sample period and implementing a goal-directed optimization (e.g., A*) in place of standard numerical optimization. This formulation of MPC readily applies to nonlinear systems and avoids the local minima which can cause a vehicle to become immobilized behind obstacles. The SBMPC algorithm is applied to an AUV in a 2D cluttered environment and an AUV in a common local minima problem. The algorithm is then used on a full kinematic model to demonstrate the benefits.

Integrating Sample-Based Planning and Model-Based Reinforcement Learning

Recent advancements in model-based reinforcement learning have shown that the dynamics of many structured domains (e.g. DBNs) can be learned with tractable sample complexity, despite their exponentially large state spaces. Unfortunately, these algorithms all require access to a planner that computes a near optimal policy, and while many traditional MDP algorithms make this guarantee, their computation time grows with the number of states. We show how to replace these over-matched planners with a class of sample-based planners — whose computation time is independent of the number of states — without sacrificing the sample-efficiency guarantees of the overall learning algorithms. To do so, we define sufficient criteria for a sample-based planner to be used in such a learning system and analyze two popular sample-based approaches from the literature. We also introduce our own sample-based planner, which combines the strategies from these algorithms and still meets the criteria for integration into our learning system. In doing so, we define the first complete RL solution for compactly represented (exponentially sized) state spaces with efficiently learnable dynamics that is both sample efficient and whose computation time does not grow rapidly with the number of states.

Reachable Distance Space: Efficient Sampling-Based Planning for Spatially Constrained Systems

Motion planning for spatially constrained robots is difficult due to additional constraints placed on the robot, such as closure constraints for closed chains or requirements on end-effector placement for articulated linkages. It is usually computationally too expensive to apply sampling-based planners to these problems since it is difficult to generate valid configurations. We overcome this challenge by redefining the robot’s degrees of freedom and constraints into a new set of parameters, called reachable distance space (RD-space), in which all configurations lie in the set of constraint-satisfying subspaces. This enables us to directly sample the constrained subspaces with complexity linear in the number of the robot’s degrees of freedom. In addition to supporting efficient sampling of configurations, we show that the RD-space formulation naturally supports planning and, in particular, we design a local planner suitable for use by sampling-based planners. We demonstrate the effectiveness and efficiency of our approach for several systems including closed chain planning with multiple loops, restricted end-effector sampling, and on-line planning for drawing/sculpting. We can sample single-loop closed chain systems with 1,000 links in time comparable to open chain sampling, and we can generate samples for 1,000-link multi-loop systems of varying topologies in less than a second.

Lower Bounding Klondike Solitaire with Monte-Carlo Planning

Despite its ubiquitous presence, very little is known about the odds of winning the simple card game of Klondike Solitaire. The main goal of this paper is to investigate the use of probabilistic planning to shed light on this issue. Unfortunatley, most probabilistic planning techniques are not well suited for Klondike due to the difficulties of representing the domain in standard planning languages and the complexity of the required search. Klondike thus serves as an interesting addition to the complement of probabilistic planning domains. In this paper, we study Klondike using several sampling-based planning approaches including UCT, hindsight optimization, and sparse sampling, and establish lower bounds on their performance. We also introduce novel combinations of these approaches and evaluate them in Klondike. We provide a theoretical bound on the sample complexity of a method that naturally combines sparse sampling and UCT. Our results demonstrate that there is a policy that within tight confidence intervals wins over 35% of Klondike games. This result is the first reported lower bound of an optimal Klondike policy.

Pseudospectral Motion Planning for Autonomous Vehicles

A COMMON task for autonomous vehicles is motion planning. Discipline-based design of motion planning algorithms have led to the development and evolution of different techniques to solve specific problems. For instance, the artificial–potential–function technique [1] is a popular method for the motion planning of unmanned ground vehicles (UGV) and robotic manipulators [1–3]. Although this technique has been used for over 30 years, it suffers from the possibility of the vehicle not achieving its goal and difficulties in accommodating various environmental constraints [4]. To overcome such issues, recent developments in motion planning algorithms place heavy emphasis on so-called sampling-based planning techniques such as probabilistic road maps, rapidly exploring random trees, and expansive space trees to name a few [5– 9]. These methods use probabilistic means of connecting the initial configuration to thefinal configuration thereby enabling an improved capacity to achieve the goal and a capability to generate initial feasible paths. In all these techniques, the initial feasible paths do not automatically incorporate vehicle dynamics; hence, path-following control techniques are needed to satisfy the physics of themotion [9]. This is one reason why nonholonomic constraints play such a crucial role in constrained control techniques that are designed to serve pathfollowing systems. There is no doubt that optimal control theory is the most natural framework for solving motion planning problems; however, solving optimal control problems has historically been considered difficult due to the twin curses of dimensionality and complexity. These difficulties are exacerbated in the presence of state constraints; hence, an obstacle-cluttered environment becomes a substantially more difficult problem under the framework of optimal control theory [10,11]. Nonetheless, it is possible to solve simplified motion planning problems wherein the cost function is quadratic or the environment is obstacle free. Because the motion planning techniques developed in robotics applications are unsuitable for flight vehicles, such as launch and reentry, for example, aerospace problems have motivated the development of efficient optimal control algorithms. In aerospace applications, satisfaction of the dynamical constraints is exceedingly important; hence, trajectory and control become fundamentally intertwined. On the other hand, aerospace problems do not have the vast number of path constraints that are common in an obstacle-cluttered environment. In recent years, paradigm-changing advancements have taken place in computational optimal control that challenge conventional wisdom. For instance, onboard the International Space Station, Bedrossian et al. [12] discovered and implemented a revolutionary momentum-dumping approach that they call a zero-propellant maneuver. This discovery was made possible by an application of pseudospectral methods to solve a real-life challenging optimal control problem [13]. Other applications of such advancements are discussed in [14,15] and include the ground test of a revolutionary attitude control concept for the NPSAT1 spacecraft. Motivated by these advancements, we apply pseudospectral methods to develop motion planning algorithms for autonomous vehicles characterized by nonlinear dynamical constraints, an obstacle-cluttered environment, and a need to generate solutions in real time. We consider the problem of generating optimal trajectories for generic autonomous vehicles. Shapes of arbitrary number, size, and configuration are modeled in the form of path constraints in the resulting optimal control problem. The method is tested under various obstacle environments on different platforms such as sea surface vehicles, ground vehicles, and aerial vehicles. The optimality of the computed trajectories is verified by way of the necessary conditions. We show that it is possible to do motion planning for different problems under the unified framework of optimal control and pseudospectral methods [16–19].

Motion Planning for Legged Robots on Varied Terrain

In this paper we study the quasi-static motion of large legged robots that have many degrees of freedom. While gaited walking may suffice on easy ground, rough and steep terrain requires unique sequences of footsteps and postural adjustments specifically adapted to the terrain's local geometric and physical properties. In this paper we present a planner that computes these motions by combining graph searching to generate a sequence of candidate footfalls with probabilistic sample-based planning to generate continuous motions that reach these footfalls. To improve motion quality, the probabilistic planner derives its sampling strategy from a small set of motion primitives that have been generated offline. The viability of this approach is demonstrated in simulation for the six-legged Lunar vehicle ATHLETE and the humanoid HRP-2 on several example terrains, including one that requires both hand and foot contacts and another that requires rappelling.

A General Deterministic Sequence for Sampling d-Dimensional Configuration Spaces

Previous works have already demonstrated that deterministic sampling can be competitive with respect to probabilistic sampling in sampling-based path planners. Nevertheless, the definition of a general sampling sequence for any d-dimensional configuration space satisfying the requirements needed for path planning is not a trivial issue. This paper makes a proposal of a simple and yet efficient deterministic sampling sequence based on the recursive use, over a multi-grid cell decomposition, of the ordering of the 2 d descendant cells of any parent cell. This ordering is generated by the digital construction method using a d×d matrix T d . A general expression of this matrix (i.e. for any d) is introduced and its performance analyzed in terms of the mutual distance. The paper ends with a performance evaluation of the use of the proposed deterministic sampling sequence in different well known path planners.

Reachability-based analysis for Probabilistic Roadmap planners

In the last fifteen years, sampling-based planners like the Probabilistic Roadmap Method ( prm) have proved to be successful in solving complex motion planning problems. While theoretically, the complexity of the motion planning problem is exponential in the number of degrees of freedom, sampling-based planners can successfully handle this curse of dimensionality in practice. We give a reachability-based analysis for these planners which leads to a better understanding of the success of the approach. This analysis compares the techniques based on coverage and connectivity of the free configuration space. The experiments show, contrary to general belief, that the main challenge is not getting the free space covered but getting the nodes connected, especially when the problems get more complicated, e.g. when a narrow passage is present. By using this knowledge, we can tackle the narrow passage problem by incorporating a refined neighbor selection strategy, a hybrid sampling strategy, and a more powerful local planner, leading to a considerable speed-up.

Sampling-based planning, control and verification of hybrid systems

A sampling-based approach to planning, control and verification inspired by robotics motion planning algorithms such as rapidly exploring random trees (RRTs) and probabilistic roadmaps (PRMs) is surveyed. With the focus on RRTs, how to adapt them to solve standard non-linear control problems is demonstrated. RRTs are extended to purely discrete spaces (replacing distance metrics with cost-to-go heuristic estimates and substituting local planners for straight-line connectivity) and computational experiments comparing them to conventional methods, such as A* are provided. Finally, RRTs are extended to the case of hybrid systems and our modifications to LaValle's motion strategy library to allow for hybrid planning and verification are described. The work on the coverage and optimality properties of sampling-based techniques is also reviewed.