Probabilistic Planner Research Articles

Multiple UAVs on a shared tether: Use cases, modeling, and probabilistic path planning

We analyze the concept of multiple unmanned aerial vehicles (UAVs) on a shared tether (MUST), where the UAVs act as control nodes for the shape of the tether, enabling the system to maneuver around obstacles without tangling or colliding for increased flexibility compared with single-tethered UAVs. MUST use cases are gathered from stakeholders in the military, public safety, and commercial domains. We present a model for the tether shape to use in collision checking and a model for the interactions among tether weight, size, and power, which we exercise to determine the maximum tether segment sizes. We apply three probabilistic path-planning algorithms from the literature to MUSTs, using a novel local planner and constraint set. In simulation, we show that probabilistic planners are a feasible approach to path planning for MUSTs with curved tether segments. We also show the first manual piloting modality of MUSTs.

A Linking Invariant for Truss Robot Motion Planning

In this paper, we introduce a new invariant of C-space components for truss robots: the link-augmented graph. This invariant uses techniques from knot, link, and spatial graph theory to encode the linking information between different closed chains in the robot. For robots with a disconnected free configuration space, this invariant serves as a tool to distinguish robot configurations that lie in different connected components of C-space from each other. This can be used to eliminate goal positions that are unreachable by any collision-free motion, without needing to perform any probabilistic planning. This invariant can also be used to find appropriate assignments of node labels in a specified goal position. We demonstrate the advantages of using this invariant in conjunction with a probabilistic planner, and introduce a variant of RRT-Connect to simultaneously search for all valid goal labelings.

Kinodynamic planning for an energy-efficient autonomous ornithopter

This paper presents a novel algorithm to plan energy-efficient trajectories for autonomous ornithopters. In general, trajectory optimization is quite a relevant problem for practical applications with Unmanned Aerial Vehicles (UAVs). Even though the problem has been well studied for fixed and rotatory-wing vehicles, there are far fewer works exploring it for flapping-wing UAVs, like ornithopters. These are of interest for many applications where long-flight endurance, but also hovering capabilities, are required. We propose an efficient approach to plan ornithopter trajectories that minimize energy consumption by combining gliding and flapping maneuvers. Our algorithm builds a tree of dynamically feasible trajectories and it applies heuristic search for efficient online planning, using reference curves to guide the search and prune states. We present computational experiments to analyze and tune the key parameters, as well as a comparison against a recent alternative probabilistic planner, showing best performance. Finally, we demonstrate how our algorithm can be used for planning perching maneuvers online.

Learning and Reasoning for Robot Sequential Decision Making under Uncertainty

Robots frequently face complex tasks that require more than one action, where sequential decision-making (sdm) capabilities become necessary. The key contribution of this work is a robot sdm framework, called lcorpp, that supports the simultaneous capabilities of supervised learning for passive state estimation, automated reasoning with declarative human knowledge, and planning under uncertainty toward achieving long-term goals. In particular, we use a hybrid reasoning paradigm to refine the state estimator, and provide informative priors for the probabilistic planner. In experiments, a mobile robot is tasked with estimating human intentions using their motion trajectories, declarative contextual knowledge, and human-robot interaction (dialog-based and motion-based). Results suggest that, in efficiency and accuracy, our framework performs better than its no-learning and no-reasoning counterparts in office environment.

GNSS/LiDAR-Based Navigation of an Aerial Robot in Sparse Forests

Autonomous navigation of unmanned vehicles in forests is a challenging task. In such environments, due to the canopies of the trees, information from Global Navigation Satellite Systems (GNSS) can be degraded or even unavailable. Also, because of the large number of obstacles, a previous detailed map of the environment is not practical. In this paper, we solve the complete navigation problem of an aerial robot in a sparse forest, where there is enough space for the flight and the GNSS signals can be sporadically detected. For localization, we propose a state estimator that merges information from GNSS, Attitude and Heading Reference Systems (AHRS), and odometry based on Light Detection and Ranging (LiDAR) sensors. In our LiDAR-based odometry solution, the trunks of the trees are used in a feature-based scan matching algorithm to estimate the relative movement of the vehicle. Our method employs a robust adaptive fusion algorithm based on the unscented Kalman filter. For motion control, we adopt a strategy that integrates a vector field, used to impose the main direction of the movement for the robot, with an optimal probabilistic planner, which is responsible for obstacle avoidance. Experiments with a quadrotor equipped with a planar LiDAR in an actual forest environment is used to illustrate the effectiveness of our approach.

Bridging Commonsense Reasoning and Probabilistic Planning via a Probabilistic Action Language

AbstractTo be responsive to dynamically changing real-world environments, an intelligent agent needs to perform complex sequential decision-making tasks that are often guided by commonsense knowledge. The previous work on this line of research led to the framework calledinterleaved commonsense reasoning and probabilistic planning(icorpp), which used P-log for representing commmonsense knowledge and Markov Decision Processes (MDPs) or Partially Observable MDPs (POMDPs) for planning under uncertainty. A main limitation of icorppis that its implementation requires non-trivial engineering efforts to bridge the commonsense reasoning and probabilistic planning formalisms. In this paper, we present a unified framework to integrate icorpp’s reasoning and planning components. In particular, we extend probabilistic action languagepBC+ to express utility, belief states, and observation as in POMDP models. Inheriting the advantages of action languages, the new action language provides an elaboration tolerant representation of POMDP that reflects commonsense knowledge. The idea led to the design of the systempbcplus2pomdp, which compiles apBC+ action description into a POMDP model that can be directly processed by off-the-shelf POMDP solvers to compute an optimal policy of thepBC+ action description. Our experiments show that it retains the advantages of icorppwhile avoiding the manual efforts in bridging the commonsense reasoner and the probabilistic planner.

Training Deep Reactive Policies for Probabilistic Planning Problems

State-of-the-art probabilistic planners typically apply look- ahead search and reasoning at each step to make a decision. While this approach can enable high-quality decisions, it can be computationally expensive for problems that require fast decision making. In this paper, we investigate the potential for deep learning to replace search by fast reactive policies. We focus on supervised learning of deep reactive policies for probabilistic planning problems described in RDDL. A key challenge is to explore the large design space of network architectures and training methods, which was critical to prior deep learning successes. We investigate a number of choices in this space and conduct experiments across a set of benchmark problems. Our results show that effective deep reactive policies can be learned for many benchmark problems and that leveraging the planning problem description to define the network structure can be beneficial.

Deterministic sampling-based motion planning: Optimality, complexity, and performance

Probabilistic sampling-based algorithms, such as the probabilistic roadmap (PRM) and the rapidly exploring random tree (RRT) algorithms, represent one of the most successful approaches to robotic motion planning, due to their strong theoretical properties (in terms of probabilistic completeness or even asymptotic optimality) and remarkable practical performance. Such algorithms are probabilistic in that they compute a path by connecting independently and identically distributed (i.i.d.) random points in the configuration space. Their randomization aspect, however, makes several tasks challenging, including certification for safety-critical applications and use of offline computation to improve real-time execution. Hence, an important open question is whether similar (or better) theoretical guarantees and practical performance could be obtained by considering deterministic, as opposed to random, sampling sequences. The objective of this paper is to provide a rigorous answer to this question. Specifically, we first show that PRM, for a certain selection of tuning parameters and deterministic low-dispersion sampling sequences, is deterministically asymptotically optimal, in other words, it returns a path whose cost converges deterministically to the optimal one as the number of points goes to infinity. Second, we characterize the convergence rate, and we find that the factor of sub-optimality can be very explicitly upper-bounded in terms of the-dispersion of the sampling sequence and the connection radius of PRM. Third, we show that an asymptotically optimal version of PRM exists with computational and space complexity arbitrarily close to (the theoretical lower bound), where n is the number of points in the sequence. This is in contrast to the complexity results for existing asymptotically optimal probabilistic planners. Fourth, we discuss extending our theoretical results and insights to other batch-processing algorithms such as FMT*, to non-uniform sampling strategies, to k-nearest-neighbor implementations, and to differentially constrained problems. Importantly, our main theoretical tool is the -dispersion, an interesting consequence of which is that all our theoretical results also hold for low--dispersion random sampling (which i.i.d. sampling does not satisfy). In other words, achieving deterministic guarantees is really a matter of i.i.d. sampling versus non-i.i.d. low-dispersion sampling (with deterministic sampling as a prominent case), as opposed to random versus deterministic. Finally, through numerical experiments, we show that planning with deterministic (or random) low-dispersion sampling generally provides superior performance in terms of path cost and success rate.

Learning Relational Dynamics of Stochastic Domains for Planning

Probabilistic planners are very flexible tools that can provide good solutions for difficult tasks. However, they rely on a model of the domain, which may be costly to either hand code or automatically learn for complex tasks. We propose a new learning approach that (a) requires only a set of state transitions to learn the model; (b) can cope with uncertainty in the effects; (c) uses a relational representation to generalize over different objects; and (d) in addition to action effects, it can also learn exogenous effects that are not related to any action, e.g., moving objects, endogenous growth and natural development. The proposed learning approach combines a multi-valued variant of inductive logic programming for the generation of candidate models, with an optimization method to select the best set of planning operators to model a problem. Finally, experimental validation is provided that shows improvements over previous work.

From FOND to Robust Probabilistic Planning: Computing Compact Policies that Bypass Avoidable Deadends

We address the class of probabilistic planning problems where the objective is to maximize the probability of reaching a prescribed goal. The complexity of probabilistic planning problems makes it difficult to compute high quality solutions for large instances, and existing algorithms either do not scale, or do so at the expense of the solution quality. We leverage core similarities between probabilistic and fully observable non-deterministic (FOND) planning to construct a sound, offline probabilistic planner, ProbPRP, that exploits algorithmic advances from state-of-the-art FOND planner, PRP, to compute compact policies that are guaranteed to bypass avoidable deadends. We evaluate ProbPRP on a selection of benchmarks used in past probabilistic planning competitions. The results show that ProbPRP, in many cases, outperforms the state of the art, computing substantially more robust policies and at times doing so orders of magnitude faster.

Learning Uncertainty in Ocean Current Predictions for Safe and Reliable Navigation of Underwater Vehicles

Operating autonomous underwater vehicles AUVs near shore is challenging-heavy shipping traffic and other hazards threaten AUV safety at the surface, and strong ocean currents impede navigation when underwater. Predictive models of ocean currents have been shown to improve navigation accuracy, but these forecasts are typically noisy, making it challenging to use them effectively. Prior work has explored the use of probabilistic planners, such as Markov decision processes MDPs, for planning in these scenarios, but prior methods have lacked a principled way of modeling the uncertainty in ocean model predictions, which limits applicability to cases in which high fidelity models are available. To overcome this limitation, we propose using Gaussian processes GPs augmented with interpolation variance to provide confidence measures on predictions. This paper describes two novel planners that incorporate these confidence measures: 1 a stationary risk-aware GPMDP for low-variability currents, and 2 a nonstationary risk-aware NS-GPMDP for faster and high-variability currents. Extensive simulations indicate that the learned confidence measures allow for safe and reliable operation with uncertain ocean current models. Field tests of the planners on Slocum gliders over several weeks in the ocean demonstrate the practical efficacy of our approach.

Depth-based short-sighted stochastic shortest path problems

Stochastic Shortest Path Problems (SSPs) are a common representation for probabilistic planning problems. Two approaches can be used to solve SSPs: (i) consider all probabilistically reachable states and (ii) plan only for a subset of these reachable states. Closed policies, the solutions obtained in the former approach, require significant computational effort, and they do not require replanning, i.e., the planner is never re-invoked. The second approach, employed by replanners, computes open policies, i.e., policies for a subset of the probabilistically reachable states. Therefore, when a state is reached in which the open policy is not defined, the replanner is reinvoked to compute a new open policy. In this article, we introduce a special case of SSPs, the depth-based short-sighted SSPs, in which every state has a nonzero probability of being reached using at most t actions. We also introduce the novel algorithm Short-Sighted Probabilistic Planner (SSiPP), which solves SSPs through depth-based short-sighted SSPs and guarantees that at least t actions can be executed without replanning. Therefore, SSiPP can compute both open and closed policies: as t increases, the returned policy approaches the behavior of a closed policy, and for t large enough, the returned policy is closed. Moreover, we present two extensions to SSiPP: Labeled-SSiPP and SSiPP-FF. The former extension incorporates a labeling mechanism to avoid revisiting states that have already converged. The latter extension combines SSiPP and determinizations to improve the performance of SSiPP in problems without dead ends. We also performed an extensive empirical evaluation of SSiPP and its extensions in several problems against state-of-the-art planners. The results show that (i) Labeled-SSiPP outperforms SSiPP and the considered planners in the task of finding the optimal solution when the problems have a low percentage of relevant states; and (ii) SSiPP-FF outperforms SSiPP in the task of quickly finding suboptimal solutions to problems without dead ends while performing similarly in problems with dead ends.

INTEGRATING PLANNING, EXECUTION, AND LEARNING TO IMPROVE PLAN EXECUTION

Algorithms for planning under uncertainty require accurate action models that explicitly capture the uncertainty of the environment. Unfortunately, obtaining these models is usually complex. In environments with uncertainty, actions may produce countless outcomes and hence, specifying them and their probability is a hard task. As a consequence, when implementing agents with planning capabilities, practitioners frequently opt for architectures that interleave classical planning and execution monitoring following a replanning when failure paradigm. Though this approach is more practical, it may produce fragile plans that need continuous replanning episodes or even worse, that result in execution dead‐ends. In this paper, we propose a new architecture to relieve these shortcomings. The architecture is based on the integration of a relational learning component and the traditional planning and execution monitoring components. The new component allows the architecture to learn probabilistic rules of the success of actions from the execution of plans and to automatically upgrade the planning model with these rules. The upgraded models can be used by any classical planner that handles metric functions or, alternatively, by any probabilistic planner. This architecture proposal is designed to integrate off‐the‐shelf interchangeable planning and learning components so it can profit from the last advances in both fields without modifying the architecture.

Short-Sighted Stochastic Shortest Path Problems

Two extreme approaches can be applied to solve a probabilistic planning problem, namely closed loop algorithms and open loop (a.k.a. replanning) algorithms. While closed loop algorithms invest significant computational effort to generate a closed form solution, open loop algorithms compute open form solutions and interact with the environment in order to refine the computed solution. In this paper, we introduce short-sighted Stochastic Shortest Path (SSP), a new model in which solutions computed based on it can be executed for at least t steps as a closed form solution. Using short-sighted SSPs, we present a novel probabilistic planner called Short-sighted Open Loop Planner (SOLP) that bridges the gap between open and closed loop planners by varying the parameter t: as t increases, more actions can be executed without replanning and, for t sufficiently large, a closed form solution is obtained. We prove that SOLP is asymptotically optimal. To the best of our knowledge, SOLP is the unique probabilistic planner that at the same time provides both replanning and optimality guarantees. We empirically compare SOLP with the winners of the previous probabilistic planning competitions and SOLP outperforms all of them in 33.3% of the problems and ties with the best planner in 48.3% of the problems.

Nonprobabilistic anytime algorithm for high-quality trajectories in high-dimensional spaces

SUMMARYThis paper proposes a feasible variation of approximate grid-based path-planning methods, which have been replaced with probabilistic methods nowadays, due mainly to their impracticality in high-dimensional spaces. Our aim is to demonstrate that, with an incremental exploration of the CSpace by means of Interpolated walk primitives and ANY-Time algorithms, we can generate – online – high-quality solutions that can be compared with probabilistic methods and can even improve some aspects, such as the completeness. Computational time, path smoothness, path clearance, and path distance are the qualifiers used to evaluate the path planner. These quality factors are critical in robotics. In fact, in both mobile and industrial robots, the computational time is a primary requirement to work online, the clearance gives security to the movements of the robot, and the smoothness can prolong the life of the mechanical components. Our method has been compared to probabilistic path planners, with the feasibility and benefits of our algorithm being proved in terms of the quality factors aforementioned.

Linear dimensionality reduction in random motion planning

In this paper we present a method to control random sampling in motion planning algorithms. The principle of the method is to use online the results of a probabilistic planner to describe the free space in which the planning takes place, by computing a principal component analysis (PCA). This method identifies the locally free directions of the free space. Given that description, our algorithm accelerates the progression along these favored directions. In this way, if the free space appears as a small volume around a sub-manifold of a high-dimensional configuration space, the method overcomes the usual limitations of probabilistic motion planning algorithms and finds a solution quickly. The presented method is theoretically analyzed and experimentally compared with known motion planners.

SixthSense: Fast and Reliable Recognition of Dead Ends in MDPs

The results of the latest International Probabilistic Planning Competition (IPPC-2008) indicate that the presence of dead ends, states with no trajectory to the goal, makes MDPs hard for modern probabilistic planners. Implicit dead ends, states with executable actions but no path to the goal, are particularly challenging; existing MDP solvers spend much time and memory identifying these states. As a first attempt to address this issue, we propose a machine learning algorithm called SIXTHSENSE. SIXTHSENSE helps existing MDP solvers by finding nogoods, conjunctions of literals whose truth in a state implies that the state is a dead end. Importantly, our learned nogoods are sound, and hence the states they identify are true dead ends. SIXTHSENSE is very fast, needs little training data, and takes only a small fraction of total planning time. While IPPC problems may have millions of dead ends, they may typically be represented with only a dozen or two no-goods. Thus, nogood learning efficiently produces a quick and reliable means for dead-end recognition. Our experiments show that the nogoods found by SIXTHSENSE routinely reduce planning space and time on IPPC domains, enabling some planners to solve problems they could not previously handle.

Systematic Tele-operation with Augmented Reality Path Planned Navigation Cues in Cluttered Environments

A novel method of providing human operators with Augmented Reality (AR) cues for collision-free navigation of robot arms in cluttered environments is described. The cues are graphic objects that aid the operator to rotate and translate the end-effector of the robot along AR collision-free paths pre-computed by a probabilistic path planner. Manual control of robot arms is a common mode of tele-manipulation on the International Space Station (ISS). Limited depth perception of the remote robot's 3D worksite can result in collisions with severe consequences. This paper combines the qualities of a planning algorithm to search for ideal paths, and the benefits of augmented reality for enhanced visualization. Presenting spatial path information to the user in an intuitive manner results in collision-free robot navigation along with performance repeatability. We present findings from the implementation of such a system with human operators and show that the system is beneficial.

Motion Planning for Legged Robots on Varied Terrain

In this paper we study the quasi-static motion of large legged robots that have many degrees of freedom. While gaited walking may suffice on easy ground, rough and steep terrain requires unique sequences of footsteps and postural adjustments specifically adapted to the terrain's local geometric and physical properties. In this paper we present a planner that computes these motions by combining graph searching to generate a sequence of candidate footfalls with probabilistic sample-based planning to generate continuous motions that reach these footfalls. To improve motion quality, the probabilistic planner derives its sampling strategy from a small set of motion primitives that have been generated offline. The viability of this approach is demonstrated in simulation for the six-legged Lunar vehicle ATHLETE and the humanoid HRP-2 on several example terrains, including one that requires both hand and foot contacts and another that requires rappelling.

Sequential Monte Carlo in reachability heuristics for probabilistic planning

Some of the current best conformant probabilistic planners focus on finding a fixed length plan with maximal probability. While these approaches can find optimal solutions, they often do not scale for large problems or plan lengths. As has been shown in classical planning, heuristic search outperforms bounded length search (especially when an appropriate plan length is not given a priori). The problem with applying heuristic search in probabilistic planning is that effective heuristics are as yet lacking. In this work, we apply heuristic search to conformant probabilistic planning by adapting planning graph heuristics developed for non-deterministic planning. We evaluate a straight-forward application of these planning graph techniques, which amounts to exactly computing a distribution over many relaxed planning graphs (one planning graph for each joint outcome of uncertain actions at each time step). Computing this distribution is costly, so we apply Sequential Monte Carlo (SMC) to approximate it. One important issue that we explore in this work is how to automatically determine the number of samples required for effective heuristic computation. We empirically demonstrate on several domains how our efficient, but sometimes suboptimal, approach enables our planner to solve much larger problems than an existing optimal bounded length probabilistic planner and still find reasonable quality solutions.