Hierarchical Task Network Planning Research Articles

Comparison of HTN Planning and OR-based Approaches for Solving Problems in Logistics Domains

Logistics is a day-to-day real-life situation leading to a very large domain. Among the various kinds of logistics problems we encounter are those pertaining to transportation, product inventories and location. Traditionally, Operations Research (OR) techniques are used most often and are popular candidates for solving logistics problems. With the development of Hierarchical Task Network (HTN) planning algorithms, there has been a recent trend to use HTN planners to address planning problems in general. It is also envisaged that HTN planners could solve logistics problems; yet, no proof or evidence could be found as to whether this approach is more efficient or suitable to handle logistics problems. In this paper, a comparison is done to ascertain whether HTN planning could be more efficient than OR-based approaches in solving logistics problems. Results revealed that HTN planning outperforms the traditional OR-based approach, especially in solving moderate-sized and more difficult harder problems. Furthermore, HTN planning is capable of solving very large scale problems, whereas OR based approach was unable to handle them within the same computational resource limitations.Keywords: Logistics, Operations research, Hierarchical task network, Transportation simplex method, HyperTensioN

A novel hierarchical task network planning approach for multi-objective optimization

In this paper, we focus on how to generate the multi-objective optimal plan in hierarchical task network (HTN) planning. Many practical HTN planning problems not only require feasible plans, but also require these plans to achieve the best performance on multiple criteria, which is the multi-objective optimization problem in HTN planning. Aiming to generate the multi-objective optimal plan in HTN planning, a novel HTN planning approach is proposed. Our initial step is to express multiple criteria by extending the concept of the HTN planning domain. The primary search process of our proposed HTN planning approach has two components. The first one is heuristic search, which assesses the quality of methods/operators and organizes them according to their dominance relations. The second one is anytime search, which reduces the search space on the basis of the dominance relations between the partial plan and the non-dominated plan set, which is updated during the planning process. After the termination of the HTN planning search, a non-dominated plan set is returned for the decision-maker to choose from. We adopt Zeno Travel domain problems and an emergency evacuation planning problem in an experimental study to demonstrate how effective and practicable the proposed approach is.

On the Computational Complexity of Plan Verification, (Bounded) Plan-Optimality Verification, and Bounded Plan Existence

In this paper we study the computational complexity of several reasoning tasks centered around the bounded plan existence problem. We do this for standard classical planning and hierarchical task network (HTN) planning and each for a grounded and a lifted representation. Whereas bounded plan existence complexity is known for classical planning, it has not yet been studied for HTN planning. For plan verification, results were available for both formalisms except for the lifted HTN planning. We will present lower and upper bounds of the complexity of plan verification in lifted HTN planning and provide novel insights into its grounded counterpart, in which we show that verification is not just NP-complete in the general case, but already for a severely restricted special case. Finally, we show the complexity concerning verifying the optimality of a given plan and discuss its connection to the bounded plan existence problem.

Flexible FOND HTN Planning: A Complexity Analysis

Hierarchical Task Network (HTN) planning is an expressive planning formalism that has often been advocated to address real-world problems. Yet few extensions exist that can deal with the many challenges encountered in the real world, one being the capability to express uncertainty. Recently, a new HTN formalism for fully observable nondeterministic problems was proposed and studied theoretically. In this paper, we lay out limitations of that formalism and propose an alternative definition, which addresses and resolves such limitations. We also study its complexity for certain problems.

HTN Replanning from the Middle

When an actor executes a plan, action failures and exogenous events may lead to unexpected states that require replanning from the middle of plan execution. In Hierarchical Task Network (HTN) planning, unless the HTN methods have been carefully written to work well in unexpected states, replanning may either fail or produce plans that perform poorly.
 To overcome this problem, we introduce IPyHOP, a reentrant version of GTPyhop (a SHOP-like HTN planner), and Run-Lazy-Refineahead, a modification of the Run-Lazy-Lookahead actor that utilizes IPyHOP's reentrant replanning capability to replan during plan execution. In our experiments, Run-Lazy-Refineahead and IPyHOP expend less search effort (fewer decompositions and fewer iterations), find revised plans with fewer actions and lower total action cost, and finish execution with fewer failures.

SquadSmart — Hierarchical Planning and Coordinated Plan Execution for Squads of Characters

This paper presents an application of Hierarchical Task Network (HTN) planning to a squad-based military simulation. The hierarchical planner produces collaborative plans for the whole squad in real time, generating the type of highly coordinated behaviours typical for armed combat situations involving trained professionals. Here, we detail the extensions to HTN planning necessary to provide real-time planning and subsequent collaborative plan execution. To make full hierarchical planning feasible in a game context we employ a planner compilation technique that saves memory allocations and speeds up symbol access. Additionally, our planner can be paused and resumed, making it possible to impose a hard limit on its computation time during any single frame. For collaborative plan execution we describe several synchronization extensions to the HTN framework, allowing agents to participate in several plans at once and to act in parallel or in sequence during single plans. Overall, we demonstrate that HTN planning can be used as an expressive and powerful real-time planning framework for tightly coupled groups of in-game characters.

Hierarchical Plan Representations for Encoding Strategic Game AI

In this paper we explore the use of Hierarchical-Task-Network (HTN) representations to model strategic game AI. We will present two case studies. The first one reports on an experiment using HTNs to model strategies for Unreal Tournament® (UT) bots. We will argue that it is possible to encode strategies that coordinate teams of bots in first-person shooter games using HTNs. The second one compares an alternative to HTNs called Task-Method-Knowledge (TMK) process models. TMK models are of interest to game AI because, as we will show, they are as expressive as HTNs but have more convenient syntax. Therefore, HTN planners can be used to generate correct plans for coordinated team AI behavior modeled with TMK representations.

Tight Bounds for HTN Planning with Task Insertion (Extended Abstract)

Hierarchical Task Network (HTN) planning with task insertion (TIHTN planning) is a variant of HTN planning. In HTN planning, the only means to alter task networks is to decompose compound tasks. In TIHTN planning, tasks may also be inserted directly. In this paper we provide tight complexity bounds for TIHTN planning along two axis: whether variables are allowed and whether methods must be totally ordered.

Applying Monte-Carlo Tree Search in HTN Planning

Search methods are useful in hierarchical task network (HTN) planning to make performance less dependent on the domain knowledge provided, and to minimize plan costs. Here we investigate Monte-Carlo tree search (MCTS) as a new algorithmic alternative in HTN planning. We implement combinations of MCTS with heuristic search in PANDA. We furthermore investigate MCTS in JSHOP, to address lifted (non-grounded) planning, leveraging the fact that, in contrast to other search methods, MCTS does not require a grounded task representation. Our new methods yield coverage performance on par with the state of the art, but in addition can effectively minimize plan cost over time.

Incrementally Solving STNs by Enforcing Partial Path Consistency

Efficient management and propagation of temporal constraints is important for temporal planning as well as for scheduling. During plan development, new events and temporal constraints are added and existing constraints may be tightened; the consistency of the whole temporal network is frequently checked; and results of constraint propagation guide further search. Recent work shows that enforcing partial path consistency provides an efficient means of propagating temporal information for the popular Simple Temporal Network (STN). We show that partial path consistency can be enforced incrementally, thus exploiting the similarities of the constraint network between subsequent edge tightenings. We prove that the worst-case time complexity of our algorithm can be bounded both by the number of edges in the chordal graph (which is better than the previous bound of the number of vertices squared), and by the degree of the chordal graph times the number of vertices incident on updated edges. We show that for many sparse graphs, the latter bound is better than that of the previously best-known approaches. In addition, our algorithm requires space only linear in the number of edges of the chordal graph, whereas earlier work uses space quadratic in the number of vertices. Finally, empirical results show that when incrementally solving sparse STNs, stemming from problems such as Hierarchical Task Network planning, our approach outperforms extant algorithms.

Symbolic Search for Optimal Total-Order HTN Planning

Symbolic search has proven to be a useful approach to optimal classical planning. In Hierarchical Task Network (HTN) planning, however, there is little work on optimal planning. One reason for this is that in HTN planning, most algorithms are based on heuristic search, and admissible heuristics have to incorporate the structure of the task network in order to be informative. In this paper, we present a novel approach to optimal (totally-ordered) HTN planning, which is based on symbolic search. An empirical analysis shows that our symbolic approach outperforms the current state of the art for optimal totally-ordered HTN planning.

Landmark Generation in HTN Planning

Landmarks (LMs) are state features that need to be made true or tasks that need to be contained in every solution of a planning problem. They are a valuable source of information in planning and can be exploited in various ways. LMs have been used both in classical and hierarchical planning, but while there is much work in classical planning, the techniques in hierarchical planning are less evolved. We introduce a novel LM generation method for Hierarchical Task Network (HTN) planning and show that it is sound and incomplete. We show that every complete approach is as hard as the co-class of the underlying HTN problem, i.e. coNP-hard for our setting (while our approach is in P). On a widely used benchmark set, our approach finds more than twice the number of landmarks than the approach from the literature. Though our focus is on LM generation, we show that the newly discovered landmarks bear information beneficial for solvers.

Revealing Hidden Preconditions and Effects of Compound HTN Planning Tasks – A Complexity Analysis

In Hierarchical Task Network (HTN) planning, compound tasks need to be refined into executable (primitive) action sequences. In contrast to their primitive counterparts, compound tasks do not specify preconditions or effects. Thus, their implications on the states in which they are applied are not explicitly known: they are "hidden" in and depending on the decomposition structure. We formalize several kinds of preconditions and effects that can be inferred for compound tasks in totally ordered HTN domains. As relevant special case we introduce a problem relaxation which admits reasoning about preconditions and effects in polynomial time. We provide procedures for doing so, thereby extending previous work, which could only deal with acyclic models. We prove our procedures to be correct and complete for any totally ordered input domain. These results are embedded into an encompassing complexity analysis of the inference of preconditions and effects of compound tasks, an investigation that has not been made so far.

Loop Detection in the PANDA Planning System

The International Planning Competition (IPC) in 2020 was the first one for a long time to host tracks on Hierarchical Task Network (HTN) planning. HyperTensioN, the winner of the tack on totally-ordered problems, comes with an interesting technique: it stores parts of the decomposition path in the state to mark expanded tasks and forces its depth first search to leave recursive structures in the hierarchy. This can be seen as a form of loop detection (LD) – a technique that is not very common in HTN planning. This might be due to the spirit of encoding enough advice in the model to find plans (so that loop detection is simply not necessary), or because it becomes a computationally hard task in the general (i.e. partially-ordered) setting. We integrated several approximate and exact techniques for LD into the progression search of the HTN planner PANDA. We test our techniques on the benchmark set of the IPC 2020. Both in the partial ordered and total ordered track, PANDA with LD performs better than the respective winner of the competition.

The hierarchical task network planning method based on Monte Carlo Tree Search

Since the hierarchical task network (HTN) planning depends on the domain knowledge of the problem, the planning result relies on the writing order of the decomposition method. Besides, the solution obtained by planning is usually a general feasible solution, which means there are shortcomings in the ability of finding the optimal solution. In order to reduce the dependence of HTN planning on domain knowledge and obtain a better planning solution, Pyhop-m, an HTN planning algorithm based on Monte Carlo Tree Search(MCTS) is proposed. In the planning process, a planning tree is built by MCTS to guide the HTN planner to choose the best decomposition method. Experiments illustrates that whether in the static or dynamic environment, Pyhop-m is superior to the existing Pyhop and heuristic-based Pyhop-h in plan length, planning success rate and optimal solution rate. Under the 95% confidence level, the confidence intervals of Pyhop-m algorithm to achieve the planning success rate and the optimal solution rate in the dynamic environment are [75.82%,89.18%] and [88.67%,93.95%], which are significantly higher than those of Pyhop-h with [58.19%,77.81%] and [69.91%,80.69%], respectively. Moreover, it can solve the planning problem with uncertain action executions by repeatedly simulating and evaluating the leaf nodes of the planning tree. It can be concluded that Pyhop-m can not only make the planning result independent of the writing order of the decomposition methods, but also search out the global optimal solution.

HTN Learning via Transfer Learning of Domain Landmarks

Hierarchical Task Network (HTN) planning uses task-subtask relationships to break complex problems into more manageable subtasks, similar to how human problem-solvers plan. However, one limitation of HTN planning is that it requires domain knowledge in the form of planning methods to perform this task decomposition. Recent work has partially alleviated this knowledge engineering requirement by learning HTN methods from traces of observed behavior. Although this greatly reduces the amount of knowledge that must be encoded by a domain expert, it requires a large collection of traces in order to infer important landmark states that are used during trace segmentation and method learning. In this paper we present a novel method for landmark inference that transfers knowledge of landmarks from previously encountered environments to new environments without requiring any traces from the new environment. We evaluate our work in a logistics planning domain and show that our approach performs comparably to the existing landmark inference method but requires far fewer traces.

Lilotane: A Lifted SAT-based Approach to Hierarchical Planning

One of the oldest and most popular approaches to automated planning is to encode the problem at hand into a propositional formula and use a Satisfiability (SAT) solver to find a solution. In all established SAT-based approaches for Hierarchical Task Network (HTN) planning, grounding the problem is necessary and oftentimes introduces a combinatorial blowup in terms of the number of actions and reductions to encode. Our contribution named Lilotane (Lifted Logic for Task Networks) eliminates this issue for Totally Ordered HTN planning by directly encoding the lifted representation of the problem at hand. We lazily instantiate the problem hierarchy layer by layer and use a novel SAT encoding which allows us to defer decisions regarding method arguments to the stage of SAT solving. We show the correctness of our encoding and compare it to the best performing prior SAT encoding in a worst-case analysis. Empirical evaluations confirm that Lilotane outperforms established SAT-based approaches, often by orders of magnitude, produces much smaller formulae on average, and compares favorably to other state-of-the-art HTN planners regarding robustness and plan quality. In the International Planning Competition (IPC) 2020, a preliminary version of Lilotane scored the second place. We expect these considerable improvements to SAT-based HTN planning to open up new perspectives for SAT-based approaches in related problem classes.

Exploring Context-Free Languages via Planning: The Case for Automating Machine Learning

While Machine Learning has achieved considerable success in recent years, this success crucially relies on human experts to select appropriate features, workflows, algorithms with their hyper-parameters, etc. Automating the role of the human expert has seen some attention from the Machine Learning community, with a dedicated workshop running since 2014 at one of the top Machine Learning conferences. % In this work, we propose to exploit multiple AI Planning tools for automating the human expert, generating Machine Learning pipelines automatically. We start from a knowledge about possible valid pipelines encoded as Context-Free Grammar, translate the problem of generating the corresponding language into Hierarchical Task Network (HTN) Planning model, further translate the HTN Planning model into a classical planning model. We use existing planners to produce multiple plans for the classical planning task, translate these plans into Machine Learning pipelines, train and evaluate these pipelines. Based on pipelines' accuracy feedback we update the classical planning model to improve the quality of pipelines obtained in next iterations. Using planning tools allows us to exploit the flexibility of model update instead of solution modification. We present an application that helps users to focus pipelines' exploration process by allowing to encode additional constraints on desired pipelines. Our experimental evaluation shows the feasibility of using planning techniques in this context.

Refining HTN Methods via Task Insertion with Preferences

Hierarchical Task Network (HTN) planning is showing its power in real-world planning. Although domain experts have partial hierarchical domain knowledge, it is time-consuming to specify all HTN methods, leaving them incomplete. On the other hand, traditional HTN learning approaches focus only on declarative goals, omitting the hierarchical domain knowledge. In this paper, we propose a novel learning framework to refine HTN methods via task insertion with completely preserving the original methods. As it is difficult to identify incomplete methods without designating declarative goals for compound tasks, we introduce the notion of prioritized preference to capture the incompleteness possibility of methods. Specifically, the framework first computes the preferred completion profile w.r.t. the prioritized preference to refine the incomplete methods. Then it finds the minimal set of refined methods via a method substitution operation. Experimental analysis demonstrates that our approach is effective, especially in solving new HTN planning instances.

Improving AI Planning using Map Reduce

Today, the Landmark concept is adapted from the classical planning to work in hierarchical task network planning. It was shown how it is used to extracts landmark literals from a given hierarchical planning domain and problem description and then use these literals to update the the planning domain by ruling out the irrelevant tasks and methods before the actual planning is performed. In this paper, we compine the landmark concept with the Map-reduce framework to increase the performance of the planning process. Our empirical evaluation shows that the combination between landmark and Map-Reduce framework dramatically improves performance of the planning process.