Non-deterministic Planning Research Articles

PRP Rebooted: Advancing the State of the Art in FOND Planning

Fully Observable Non-Deterministic (FOND) planning is a variant of classical symbolic planning in which actions are nondeterministic, with an action's outcome known only upon execution. It is a popular planning paradigm with applications ranging from robot planning to dialogue-agent design and reactive synthesis. Over the last 20 years, a number of approaches to FOND planning have emerged. In this work, we establish a new state of the art, following in the footsteps of some of the most powerful FOND planners to date. Our planner, PR2, decisively outperforms the four leading FOND planners, at times by a large margin, in 17 of 18 domains that represent a comprehensive benchmark suite. Ablation studies demonstrate the impact of various techniques we introduce, with the largest improvement coming from our novel FOND-aware heuristic.

A Best-First Search Algorithm for FOND Planning and Heuristic Functions to Optimize Decompressed Solution Size

In this work, we study fully-observable non-deterministic (FOND) planning, which models uncertainty through actions with non-deterministic effects. We present a best-first heuristic search algorithm called AND* that searches the policy-space of the FOND task to find a solution policy. We generalize the concepts of optimality, admissibility, and goal-awareness for FOND. Using these new concepts, we formalize the concept of heuristic functions that can guide a policy-space search. We analyze different aspects of the general structure of FOND solutions to introduce and characterize a set of FOND heuristics that estimate how far a policy is from becoming a solution. One of these heuristics applies a novel insight. Guided by them AND* returns only solutions with the minimal possible number of mapped states. We systematically study these FOND heuristics theoretically and empirically. We observe that our best heuristic makes AND* much more effective than the straightforward heuristics. We believe that our work allows a better understanding of how to design algorithms and heuristics to solve FOND tasks.

Solution Diversity in Planning

Diverse planning consists of generating multiple different solutions for the same planning problem. I explore solution diversity, based on quantitative (domain-independent) and qualitative (domain-dependent) distance metrics, in deterministic and nondeterministic planning domains.

Numeric Planning via Search Space Abstraction (Extended Abstract)

Many real-world planning problems are best modeled as infinite search space problems, using numeric fluents. Unfortunately, most planners and planning heuristics do not directly support such fluents. We propose a search space abstraction technique that compiles a planning problem with numeric fluents into a finite state propositional planning problem. To account for the loss of precision resulting from the abstraction, we leverage a policy repair technique used for non-deterministic planning. We evaluate our approach on a set of benchmarks and compare it to state-of-the-art planners that deal with numeric fluents.

Automated Generation of Diverse NPC-Controlling FSMs Using Nondeterministic Planning Techniques

We study the problem of generating a set of Finite State Machines (FSMs) modeling the behavior of multiple, distinct NPCs. We observe that nondeterministic planning techniques can be used to generate FSMs by following conventions typically used when manually creating FSMs modeling NPC behavior. We implement our ideas in DivNDP, the first algorithm for automated diverse FSM generation.

Fully Observable Nondeterministic HTN Planning -- Formalisation and Complexity Results

Much progress has been made in advancing the state of the art of HTN planning theory in recent years. However, scarce studies have been made with regards to the theory and complexity of HTN problems on nondeterministic domains. In this paper we provide a novel formalisation for fully observable nondeterministic HTN planning. We propose and study different solution criteria which differ in when nondeterministic action outcomes are considered: at plan generation or at plan execution. We integrate our solution criteria with notions of weak and strong plans canonical in nondeterministic planning and identify similarities and differences with plans in other fields of AI planning. We also provide completeness results for a majority of HTN problem subclasses and show the significant result that problems are not made any harder under nondeterminism for certain solution criteria by using compilation techniques to deterministic HTN planning. This supports and justifies the practicality and scalability of extending HTN problems over nondeterministic domains to deal with real world scenarios.

Computing Contingent Plan Graphs using Online Planning

In contingent planning under partial observability with sensing actions, agents actively use sensing to discover meaningful facts about the world. Recent successful approaches translate the partially observable contingent problem into a non-deterministic fully observable problem, and then use a planner for non-deterministic planning. However, the translation may become very large, encumbering the task of the non-deterministic planner. We suggest a different approach—using an online contingent solver repeatedly to construct a plan tree. We execute the plan returned by the online solver until the next observation action, and then branch on the possible observed values, and replan for every branch independently. In many cases a plan tree can have an exponential width in the number of state variables, but the tree may have a structure that allows us to compactly represent it using a directed graph. We suggest a mechanism for tailoring such a graph that reduces both the computational effort and the storage space. Our method also handles non-deterministic domains, by identifying cycles in the plans. We present a set of experiments, showing our approach to scale better than state-of-the-art offline planners.

Multi-Tier Automated Planning for Adaptive Behavior

A planning domain, as any model, is never “complete” and inevitably makes assumptions on the environment's dynamic. By allowing the specification of just one domain model, the knowledge engineer is only able to make one set of assumptions, and to specify a single objective-goal. Borrowing from work in Software Engineering, we propose a multi-tier framework for planning that allows the specification of different sets of assumptions, and of different corresponding objectives. The framework aims to support the synthesis of adaptive behavior so as to mitigate the intrinsic risk in any planning modeling task. After defining the multi-tier planning task and its solution concept, we show how to solve problem instances by a succinct compilation to a form of non-deterministic planning. In doing so, our technique justifies the applicability of planning with both fair and unfair actions, and the need for more efforts in developing planning systems supporting dual fairness assumptions.

Implicit Coordination Using FOND Planning

Epistemic planning can be used to achieve implicit coordination in cooperative multi-agent settings where knowledge and capabilities are distributed between the agents. In these scenarios, agents plan and act on their own without having to agree on a common plan or protocol beforehand. However, epistemic planning is undecidable in general. In this paper, we show how implicit coordination can be achieved in a simpler, propositional setting by using nondeterminism as a means to allow the agents to take the other agents' perspectives. We identify a decidable fragment of epistemic planning that allows for arbitrary initial state uncertainty and non-determinism, but where actions can never increase the uncertainty of the agents. We show that in this fragment, planning for implicit coordination can be reduced to a version of fully observable nondeterministic (FOND) planning and that it thus has the same computational complexity as FOND planning. We provide a small case study, modeling the problem of multi-agent path finding with destination uncertainty in FOND, to show that our approach can be successfully applied in practice.

Generalized Planning via Abstraction: Arbitrary Numbers of Objects

We consider a class of generalized planning problems based on the idea of quantifying over sets of similar objects. We show how we can adapt fully observable nondeterministic planning techniques to produce generalized solutions that are easy to instantiate over particular problem instances. We also describe how we can reformulate a classical planning problem into a quantified one. The reformulation allows us to solve the original planning task without grounding every action with respect to all objects in the problem, and a single solution can be applied to a possibly infinite set of related classical planning tasks. We report experimental results that show our approach is a practical and promising technique for solving an interesting class of problems.

Compact Policies for Fully Observable Non-Deterministic Planning as SAT

Fully observable non-deterministic (FOND) planning is becoming increasingly important as an approach for computing proper policies in probabilistic planning, extended temporal plans in LTL planning, and general plans in generalized planning. In this work, we introduce a SAT encoding for FOND planning that is compact and can produce compact strong cyclic policies. Simple variations of the encodings are also introduced for strong planning and for what we call, dual FOND planning, where some non-deterministic actions are assumed to be fair (e.g., probabilistic) and others unfair (e.g., adversarial). The resulting FOND planners are compared empirically with existing planners over existing and new benchmarks. The notion of ``probabilistic interesting problems'' is also revisited to yield a more comprehensive picture of the strengths and limitations of current FOND planners and the proposed SAT approach.

Fully Observable Non-deterministic Planning as Assumption-Based Reactive Synthesis

We contribute to recent efforts in relating two approaches to automatic synthesis, namely, automated planning and discrete reactive synthesis. First, we develop a declarative characterization of the standard "fairness" assumption on environments in non-deterministic planning, and show that strong-cyclic plans are correct solution concepts for fair environments. This complements, and arguably completes, the existing foundational work on non-deterministic planning, which focuses on characterizing (and computing) plans enjoying special "structural" properties, namely loopy but closed policy structures. Second, we provide an encoding suitable for reactive synthesis that avoids the naive exponential state space blowup. To do so, special care has to be taken to specify the fairness assumption on the environment in a succinct manner.

Uncertain Composition of Web Services via Non-Deterministic Planning

Web service composition (WSC) is the task of combining a set of single Web services together to create a more complex and cross-organizational composite service. Recently, many researchers have been done on WSC. However, most of these approaches did not take into account the inherent uncertainty of Web services that is the most important nature characteristic due to service deployment and invocation within a real and dynamic Internet environment. Therefore, this paper focuses on uncertain Web service composition (U-WSC) problem by three different non-deterministic effects of Web services. We proposed a comprehensive framework that models a U-WSC problem to a fully observable non-deterministic planning (FOND) problem using our automatic planning transition mechanism. The transformed uncertain planning problem can be solved by a highly efficient off-the-shelf non-deterministic planner which finds a plan to satisfy a composition request. We conducted some experiments based on a case study in an e-commerce application via an off-the-shelf non-deterministic planner called myND. The results of empirical experiments validate the feasibility of our proposed approach for dynamic composition of Web services with functional uncertainty.

Video-based interactive storytelling using real-time video compositing techniques

Interactive storytelling systems usually adopt computer graphics to represent virtual story worlds, which facilitates the dynamic generation of visual content. However, the quality of the images and motion produced by these systems is still inferior compared to the high quality experience found in live-action films. Interactive rates in photorealistic rendering for the film industry will not be possible for decades to come. A promising alternative is the replacement of 3D virtual characters with video sequences with real actors. In this paper, we propose a new method for video-based interactive narratives that uses video compositing algorithms that run at truly interactive frame rates. The proposed method is consistent with plots that are generated by nondeterministic planning algorithms. Moreover, we propose a system of artificial intelligent agents that perform the same roles played by filmmaking professionals. A user evaluation of the proposed method is presented. We believe that future improvements of the techniques proposed in this paper represent an important contribution to the quest for new and more immersive forms of interactive cinema.

Fast strong planning for fully observable nondeterministic planning problems

The ability to find strong solutions to fully observable nondeterministic (FOND) planning problems, if they exist, is desirable because unlike weak and strong-cyclic solutions, strong solutions are guaranteed to achieve the goal. However, only limited work has been done on FOND planning with strong solutions. In this paper, we present a sound and complete strong planning algorithm and incorporate it into our planner, FIP, which has achieved outstanding performance on strong cyclic planning problems. This new strong planning approach enables FIP to first search for strong solutions, and then search for strong-cyclic solutions only if strong solutions do not exist. We conduct extensive experiments to evaluate the new strong planning approach to (1) find a strong solution if one exists and (2) determine the non-existence of a strong solution. Experimental results demonstrate the superior performance of our planner to Gamer and MBP, the two best-known planners capable of solving strong FOND planning problems, on a variety of benchmark problems. Not only is our planner on average three orders of magnitude faster than Gamer and MBP, but it also scales up to larger problems.

Computing Contingent Plans Using Online Replanning

In contingent planning under partial observability with sensing actions, agents actively use sensing to discover meaningful facts about the world. For this class of problems the solution can be represented as a plan tree, branching on various possible observations. Recent successful approaches translate the partially observable contingent problem into a non-deterministic fully observable problem, and then use a planner for non-deterministic planning. While this approach has been successful in many domains, the translation may become very large, encumbering the task of the non-deterministic planner. In this paper we suggest a different approach - using an online contingent solver repeatedly to construct a plan tree. We execute the plan returned by the online solver until the next observation action, and then branch on the possible observed values, and replan for every branch independently. In many cases a plan tree can be exponential in the number of state variables, but still, the tree has a structure that allows us to compactly represent it using a directed graph. We suggest a mechanism for tailoring such a graph that reduces both the computational effort and the storage space. Furthermore, unlike recent state of the art offline planners, our approach is not bounded to a specific class of contingent problems, such as limited problem width, or simple contingent problems. We present a set of experiments, showing our approach to scale better than state of the art offline planners.

Agent planning programs

This work proposes a novel high-level paradigm, agent planning programs, for modeling agents behavior, which suitably mixes automated planning with agent-oriented programming. Agent planning programs are finite-state programs, possibly containing loops, whose atomic instructions consist of a guard, a maintenance goal, and an achievement goal, which act as precondition-invariance-postcondition assertions in program specification. Such programs are to be executed in possibly nondeterministic planning domains and their execution requires generating plans that meet the goals specified in the atomic instructions, while respecting the program control flow. In this paper, we define the problem of automatically synthesizing the required plans to execute an agent planning program, propose a solution technique based on model checking of two-player game structures, and use it to characterize the worst-case computational complexity of the problem as EXPTIME-complete. Then, we consider the case of deterministic domains and propose a different technique to solve agent planning programs, which is based on iteratively solving classical planning problems and on exploiting goal preferences and plan adaptation methods. Finally, we study the effectiveness of this approach for deterministic domains through an experimental analysis on well-known planning domains.

Non-deterministic planning methods for automated web service composition

Web service composition (WSC) is the task of generating new composite web services that exhibit functionalities not supported by any single web service. In its simplest form this is achieved by linking existing web services in sequence. More complex forms link services in parallel or use alternative paths. WSC can be considered a planning task, with the web services being the planning operators and the initial state and the goals being provided by the user. Particularly, since web services operate in a stochastic environment, their output is not predictable, and the problem is formulated as a non-deterministic planning one. This article presents a critical, comprehensive and up-to-date review of the literature concerning alternative non-deterministic planning methods, including probabilistic planning, determinization methods, planning in the belief state space and translation based methods. Furthermore, the article reviews existing implementations of WSC systems, employing a variety of planning approaches, and discusses the degree in which the current achievements from the non-deterministic planning field have been adopted successfully. To the best of our knowledge, this is the first review of its kind, one that provides a thorough introduction to the vast area of automated web service composition.

Search Complexities for HTN Planning

Abstract : Hierarchical Task Network (HTN) planning is the problem of decomposing an initial task into a sequence of executable steps. Often viewed as just a way to encode human knowledge to solve classical planning problems faster, HTN planning is more expressive than classical planning, even to the point of being undecidable in the general case. However, HTN planning is not just a way to solve planning problems faster, but is itself a search problem that can benefit from its own distinct search algorithms and heuristics. The dissertation examines the complexities of various HTN planning problem classes in order to motivate the development of heuristic search algorithms for HTN planning which are guaranteed to terminate on a large class of syntactically identifiable problems, as well as domain independent heuristics for those algorithms to use. This will allow HTN planning to be used in a number of areas where problems may be unsolvable, including during the initial development of a domain and for use in policy generation in non-deterministic planning environments. In particular, this dissertation analyzes two commonly used algorithms for HTN planning and describes the subsets of HTN problems that these algorithms terminate on. This allows us to discuss the run-times of these algorithms and compare the expressivity of the classes of problems they decide. We provide two new HTN algorithms which terminate on a strictly broader and more expressive set of HTN problems. We also analyze the complexity of delete-free HTN planning, an analogue to delete-free classical planning which is the base of many classical planning heuristics. We show that delete-free HTN planning is NP-complete, putting the existence of strict-semantics delete-relaxation-based HTN heuristics out of reach for practical purposes.

Fast Strong Planning for FOND Problems with Multi-Root Directed Acyclic Graphs

Recently, the state-of-the-art AI planners have significantly improved planning efficiency on Fully Observable Nondeterministic planning (FOND) problems with strong cyclic solutions. These strong cyclic solutions are guaranteed to achieve the goal if they terminate, implying that there is a possibility that they may run into indefinite loops. In contrast, strong solutions are guaranteed to achieve the goal, but few planners can effectively handle FOND problems with strong solutions. In this study, we aim to address this difficult, yet under-investigated class of planning problems: FOND planning problems with strong solutions. We present a planner that employs a new data structure, MRDAG (multi-root directed acyclic graph), to define how the solution space should be expanded. Based on the characteristics of MRDAG, we develop heuristics to ensure planning towards the relevant search direction and design optimizations to prune the search space to further improve planning efficiency. We perform extensive experiments to evaluate MRDAG, the heuristics, and the optimizations for pruning the search space. Experimental results show that our strong algorithm achieves impressive performance on a variety of benchmark problems: on average it runs more than three orders of magnitude faster than the state-of-the-art planners, MBP and Gamer, while demonstrating significantly better scalability.