Belief Space Hierarchical Planning in the Now for Unmanned Aerial Vehicles<br />

Abstract

Planning long duration missions for autonomous unmanned aerial vehicles (UAVs) in dynamic environments has proven to be a challenging problem. Autonomous UAVs must be able to reason about how to best accomplish mission objectives in the face of evolving mission conditions and environmental uncertainty. For example, tactical UAV missions consist of executing multiple interdependent tasks such as: locating, identifying, and prosecuting targets; responding to dynamic (i.e. pop-up) threats; motion planning with kinematic and dynamic constraints; and/or acting as a communication relay. The resulting planning problem is then one over a large and stochastic state space due to the size of the mission environment and the number of (un)known objects within that environment. The world state is also only partially observable due to the inherent limitations and stochastic nature of sensing and actuation. This requires us to plan in the belief space, which is a probability distribution over all possible states. Some a priori contextual world knowledge, like terrain maps, target locations, and threats, is available via satellite imagery based maps. But it is likely threat and target information will be “old” data by execution time in a rapidly changing mission environment. This makes classic approaches to a priori task, or symbolic, planning a poor choice of tool in that task dependent decisions are made without full knowledge of the mission environment. In addition, task planners traditionally do not have methods for tightly coupling geometric planning problems with the high level task planning, as most approaches treat these as separate planning problems. However, modern belief space geometric planning tools, like Partially Observable Markov Decision Problem (POMDP) based formulations, become intractable for large state spaces, such as the tactical UAV mission discussed in this paper. Simply developing a set of ordered tasks, as those produced by purely symbolic approaches like hybrid hierarchical task networks, provides a means of generating high-level plans in domains where many different types of actions are possible, as in our domain. But, those methods do not take into account the geometric constraints that may arise while trying to perform a mission task (e.g. fly a path from point A to point B subject to our knowledge regarding potential threats in the world). Unstructured and partially observable mission environments with unexpected events create the need for a tactical UAV to reason in both the domain geometry as well as the task space. For example, if a UAV is tasked with locating a target, it must ensure that there are no threats preventing the completion of this task and monitor this condition throughout the flight. If a threat does pop-up, it must be determined if its location endangers the mission with some probability. If it does, then the threat must be avoided or additional actions must be planned to neutralize the danger with some certainty. All of these potential responses, or additional actions, require motion planning, thus requiring both probabilistic symbolic and geometric reasoning. Geometric planning quickly becomes intractable in partially observable domains, or belief spaces, with long planning horizons. However, recent tools in the domain of robotic manipulation have approached