Nondeterministic Actions Research Articles

Reasoning about plan revision in BDI agent programs

Facilities for handling plan execution failures are essential for agents which must cope with the effects of nondeterministic actions, and some form of failure handling can be found in most mature agent programming languages and platforms. While such features simplify the development of more robust agents, they make it hard to reason about the execution of agent programs, e.g., to verify their correctness. In this paper, we present an approach to the verification of agent programs which admit exceptional executions. We consider executions of the BDI-based agent programming language 3APL in which plans containing non-executable actions can be revised using plan revision rules, and present a logic for reasoning about normal and exceptional executions of 3APL programs. We provide a complete axiomatization for the logic and, using a simple example, show how to express properties of 3APL programs as formulas of the logic.

조건부 계획수립을 위한 효과적인 그래프 기반의 휴리스틱

In order to derive domain-independent heuristics from the specification of a planning problem, it is required to relax the given problem and then solve the relaxed one. In this paper, we present a new planning graph, Merged Planning Graph(MPG), and GD heuristics for solving contingent planning problems with both uncertainty about the initial state and non-deterministic action effects. The merged planning graph is an extended one to be applied to the contingent planning problems from the relaxed planning graph, which is a common means to get effective heuristics for solving the classical planning problems. In order to get heuristics for solving the contingent planning problems with sensing actions and non-deterministic actions, the new graph utilizes additionally the effect-merge relaxations of these actions as well as the traditional delete relaxations. Proceeding parallel to the forward expansion of the merged planning graph, the computation of GD heuristic excludes the unnecessary redundant cost from estimating the minimal reachability cost to achieve the overall set of goals by analyzing interdependencies among goals or subgoals. Therefore, GD heuristics have the advantage that they usually require less computation time than the overlap heuristics, but are more informative than the max and the additive heuristics. In this paper, we explain the experimental analysis to show the accuracy and the search efficiency of the GD heuristics.

Optimizing coalition formation for tasks with dynamically evolving rewards and nondeterministic action effects

We consider a problem domain where coalitions of agents are formed in order to execute tasks. Each task is assigned at most one coalition of agents, and the coalition can be reorganized during execution. Executing a task means bringing it to one of the desired terminal states, which might take several time steps. The state of the task evolves even if no coalition is assigned to its execution and depends nondeterministically on the cumulative actions of the agents in the coalition. Furthermore, we assume that the reward obtained for executing a task evolves in time: the more the execution of the task is delayed, the lesser the reward. A representative example of this class of problems is the allocation of firefighters to fires in a disaster rescue environment. We describe a practical methodology through which a problem of this class can be encoded as a Markov Decision Process. Due to the three levels of factoring in the resulting MDP (the states, actions and rewards are composites of the original features of the problem) the resulting MDP can be directly solved only for small problem instances. We describe two methods for parallel decomposition of the MDP: the MDP RSUA approach for random sampling and uniform allocation and the MDP REUSE method which reuses the lower level MDP to allocate resources to the parallel subproblems. Through an experimental study which models the problem domain using the fire simulation components of the Robocup Rescue simulator, we show that both methods significantly outperform heuristic approaches and MDP REUSE provides an overall higher performance than MDP RSUA.

Learning Symbolic Models of Stochastic Domains

In this article, we work towards the goal of developing agents that can learn to act in complex worlds. We develop a probabilistic, relational planning rule representation that compactly models noisy, nondeterministic action effects, and show how such rules can be effectively learned. Through experiments in simple planning domains and a 3D simulated blocks world with realistic physics, we demonstrate that this learning algorithm allows agents to effectively model world dynamics.

Ramifications: An Extension and Correspondence Result for the Event Calculus

Classical logic Event Calculus, and the special purpose logical action language Formula, are both well established formalisms for describing actions and change. However, there is yet to be an account of ramifications in Event Calculus sufficiently general to represent the classes of domains expressible in Formula. Indeed, an adequately general ramification theory constructed in any general purpose logical language still awaits. Therefore, under the motivation of creating a flexible ramification theory in a universal language, suitable for integration into a rich action theory, a new enhanced version of classical logic Event Calculus named EC-R is proposed. EC-R supports representation and reasoning about domains containing ramifications for classes of domains more general than those possible under previous general purpose language formulations. This article makes two main contributions. The first, EC-R, is a narrative-based action formalism able to represent concurrent events, non-deterministic actions and indirect causal effects by virtue of an integrated solution to the frame and ramification problems. The formalism can reason about significant subclasses of domains containing both mutually interacting effects and cyclic causal dependencies. The formalism is elaboration tolerant and may be integrated with the standard variants of the Event Calculus. The second contribution is the definition of a semantic mapping between EC-R and Formula, and a proof of soundness and completeness of the EC-R theory with respect to Formula's model theoretic specification.

Strong planning under partial observability

Rarely planning domains are fully observable. For this reason, the ability to deal with partial observability is one of the most important challenges in planning. In this paper, we tackle the problem of strong planning under partial observability in nondeterministic domains: find a conditional plan that will result in a successful state, regardless of multiple initial states, nondeterministic action effects, and partial observability. We make the following contributions. First, we formally define the problem of strong planning within a general framework for modeling partially observable planning domains. Second, we propose an effective planning algorithm, based on and-or search in the space of beliefs. We prove that our algorithm always terminates, and is correct and complete. In order to achieve additional effectiveness, we leverage on a symbolic, bdd-based representation for the domain, and propose several search strategies. We provide a thorough experimental evaluation of our approach, based on a wide selection of benchmarks. We compare the performance of the proposed search strategies, and identify a uniform winner that combines heuristic distance measures with mechanisms that reduce runtime uncertainty. Then, we compare our planner mbp with other state-of-the art-systems. mbp is able to outperform its competitor systems, often by orders of magnitude.

Conformant planning via symbolic model checking and heuristic search

In this paper we tackle the problem of Conformant Planning: find a sequence of actions that guarantees goal achievement regardless of an uncertain initial condition and of nondeterministic action effects. Our approach, set in the framework of search in the belief space, is based on two main intuitions. First, symbolic model checking techniques, Binary Decision Diagrams in particular, can be used to represent and expand the search space, and to provide an efficient computational platform. Second, driving the search solely with reachability information is not effective, and the notion of knowledge has to be taken explicitly into account. We make the following contributions. First, we thoroughly analyze reachability heuristics: we formally prove their admissibility, we show how they can be implemented by means of symbolic techniques, and experimentally evaluate them. Second, we analyze the limitations of reachability heuristics, and propose more informed, yet admissible, heuristics, based on a formal notion of knowledge. Third, we present a practical conformant planning algorithm, where the search is explicitly based on the notion of target knowledge, i.e., the amount of information that has to be available in order to reach the goal. The search alternates between the “Acquire Knowledge” mode, where actions have the precise purpose of gathering necessary information, and the “Reach Goal” mode, where actions are directed towards the goal. Finally, we provide a thorough experimental comparison between several conformant planners. Our approach is sometimes able to outperform the competitor systems by orders of magnitude.

Profit Sharing Introducing the Judgement of Incomplete Perception

To apply reinforcement learning to difficult classes such as real-environment learning, we need to use a method robust to perceptual aliasing problem. The exploitation-oriented methods such as Profit Sharing can deal with the perceptual aliasing problem to a certain extent. However, when the agent needs to select different actions at the same sensory input, the learning efficiency worsens. To overcome the problem, several state partition methods using history information of state-action pairs are proposed. These methods try to convert a POMDP environment into an MDP environment, and thus they are sometimes very useful. However, their computation cost is very high especially in large state spaces. In contrast, memory-less approaches try to escape from the aliased states by outputting actions stochastically. However, these methods output actions stochastically even in unaliased states, and thus the learning efficiency is bad. If we desire to guarantee the rationality in POMDPs, it is efficient to output actions stochastically only in the aliased states and to output one action deterministically in the other unaliased states. Hence, to discriminate between aliased states and unaliased states, the utilization of χ² -goodness-of-fit test is proposed by Miyazaki et al. They point out that, in aliased states, the distributions of the state transitions by random search and a particular policy are different. This difference doesn't occur owing to non-deterministic actions. Hence, if the agent can collect enough samples to implement the test, the agent can distinguish between aliased states and unaliased states well. However, such a test needs a large amount of data, and it's a problem how the agent collects samples without worsening learning efficiency. If the agent uses random search in the course of learning, the learning efficiency worsens especially in unaliased states. Therefore, in this research, we propose a new method called Extended On-line Profit Sharing with Judgement (EOPSwJ) to detect important incomplete perception, which doesn't need large computation cost and numerous samples. We use two criterions for detecting important incomplete perceptions to attain a task. One is the rate of transitions to each state, and the other is the deterministic rate of actions. We confirm the availability of EOPSwJ using two simulations.

Nonmonotonic causal theories

The nonmonotonic causal logic defined in this paper can be used to represent properties of actions, including actions with conditional and indirect effects, nondeterministic actions, and concurrently executed actions. It has been applied to several challenge problems in the theory of commonsense knowledge. We study the relationship between this formalism and other work on nonmonotonic reasoning and knowledge representation, and discuss its implementation, called the Causal Calculator.

A Logic for Suspicious Players: Epistemic Actions and Belief–Updates in Games

This paper introduces a notion of ‘epistemic action’ to describe changes in the information states of the players in a game. For this, ideas are developed from earlier contributions. The ideas are enriched to cover not just purely epistemic actions, but also fact–changing actions (‘real moves’, e.g. choosing a card, exchanging cards etc.) and nondeterministic actions and strategies (conditional actions having knowledge tests as conditions). The author considers natural operations with epistemic actions and uses them to describe significant aspects of the interaction between beliefs and actions in a game. A logic is used that combines in a specific way a multiagent epistemic logic with a dynamic logic of ‘epistemic actions’. The author presents a complete and decidable proof system for this logic. As an application, the author analyses a specific example of a dialogue game (a version of the Muddy Children Puzzle, in which some of the children can ‘cheat’ by engaging in secret communication moves, while others may be punished for their credulity). Also presented is a sketch of a ‘rule–based’ approach to games with imperfect information (allowing ‘sneaky’ possibilities, such as cheating, being deceived and suspecting the others to be cheating).

Reasoning about actions in dynamic linear time temporal logic

In this paper we present a theory for reasoning about actions which is based on Dynamic Linear Time Temporal Logic (DLTL). DLTL is a simple extension of propositional temporal logic of linear time in which regular programs of propositional dynamic logic can be used for indexing temporal modalities. The action theory we define allows to reason with incomplete initial states, to do postdiction and to deal with ramifications and with nondeterministic actions, which are captured by possibly alternative extensions (temporal models). The expressiveness of temporal logic is exploited to enhance the action language by allowing the definition of general temporal constraints as well as complex actions in the specification of the domain description. We show that the temporal projection problem and the planning problem can be modelled as satisfiability problems in DLTL.

Knowledge and the action description language A

We introduce Ak, an extension of the action description language A (Gelfond and Lifschitz, 1993) to handle actions which affect knowledge. We use sensing actions to increase an agent's knowledge of the world and non-deterministic actions to remove knowledge. We include complex plans involving conditionals and loops in our query language for hypothetical reasoning. We also present a translation of Ak domain descriptions into epistemic logic programs.

Ramification and causality in a modal action logic

The paper presents a logic for action theory based on a modal language, where modalities represent actions. The frame problem is tackled by using a nonmonotonic formalism which maximizes persistency assumptions. The problem of ramification is tackled by introducing a modal causality operator which is used to represent causal rules. Assumptions on the value of fluents in the initial state allow reasoning with incomplete initial states and postdiction. The action theory can also deal with nonminimal change and nondeterministic actions.

Coordinating action systems

We develop an action systems-based approach that supports the separation of the design of the functional or computation aspects of a system under construction from the coordination and synchronization issues. The computation aspects are modeled as nondeterministic actions that work in parallel with the coordination actions, which impose some control on the nondeterministic part. We define a special form of action systems that models this type of coordination activity. Certain forms of real-time scheduling and coordination as well as exception handling are shown to be special cases of our approach. We show how the coordinators can be stepwise brought about from a high-level specification of the target system and how the reasoning about their behaviors is carried out separately from the computation aspects of the system within the refinement calculus.

Tackling the Qualification Problem using Fluent Dependency Constraints

In the area of formal reasoning about action and change, one of the fundamental representation problems is providing concise modular and incremental specifications of action types and world models, where instantiations of action types are invoked by agents such as mobile robots. Provided the preconditions to the action are true, their invocation results in changes to the world model concomitant with the goal‐directed behavior of the agent. One particularly difficult class of related problems, collectively called the qualification problem, deals with the need to find a concise incremental and modular means of characterizing the plethora of exceptional conditions that might qualify an action, but generally do not, without having to explicitly enumerate them in the preconditions to an action. We show how fluent dependency constraints together with the use of durational fluents can be used to deal with problems associated with action qualification using a temporal logic for action and change called TAL‐Q. We demonstrate the approach using action scenarios that combine solutions to the frame, ramification, and qualification problems in the context of actions with duration, concurrent actions, nondeterministic actions, and the use of both Boolean and non‐Boolean fluents. The circumscription policy used for the combined problems is reducible to the first‐order case.

On agents that have the ability to choose

We demonstrate ways to incorporate nondeterminism in a system designed to formalize the reasoning of agents concerning their abilities and the results of the actions that they may perform. We distinguish between two kinds of nondeterministic choice operators: one that expresses an internal choice, in which the agent decides what action to take, and one that expresses an external choice, which cannot be influenced by the agent. The presence of abilities in our system is the reason why the usual approaches towards nondeterminism cannot be used here. The semantics that we define for nondeterministic actions is based on the idea that composite actions are unravelled in the strings of atomic actions and tests that constitute them. The main notions used in defining this semantics are finite computation sequences and finite computation runs of actions. The results that we obtain meet our intuitions regarding events and abilities in the presence of nondeterminism.

Planning with graded nondeterministic actions: A possibilistic approach

This article proposes a framework for planning under uncertainty given a partially known initial state and a set of actions having nondeterministic (disjunctive) effects, some being more possible (normal) than the others. The problem, henceforth called possibilistic planning problem, is represented in an extension of the STRIPS formalism in which the initial state of the world and the graded nondeterministic effects of actions are described by possibility distributions. Two notions of solution plans are introduced: γ-acceptable plans that lead to a goal state with a certainty greater than a given threshold γ, and optimally safe plans that lead to a goal state with maximal certainty. It is shown that the search of a γ-acceptable plan amounts to solve a derived planning problem that has only pure (nongraded) nondeterministic actions. A sound and complete partial order planning algorithm, called NDP, has been developed for such classical nondeterministic planning problems. The generation of γ-acceptable and optimally safe plans is achieved by two sound and complete planning algorithms: POSPLAN that relies on NDP, and POSPLAN* that can be seen as a hierarchical version of POSPLAN. The possibilistic planning framework is illustrated throughout the article by an example in the agronomic domain. © 1997 John Wiley & Sons, Inc.

Planning with graded nondeterministic actions: A possibilistic approach

This article proposes a framework for planning under uncertainty given a partially known initial state and a set of actions having nondeterministic (disjunctive) effects, some being more possible (normal) than the others. The problem, henceforth called possibilistic planning problem, is represented in an extension of the STRIPS formalism in which the initial state of the world and the graded nondeterministic effects of actions are described by possibility distributions. Two notions of solution plans are introduced: γ-acceptable plans that lead to a goal state with a certainty greater than a given threshold γ, and optimally safe plans that lead to a goal state with maximal certainty. It is shown that the search of a γ-acceptable plan amounts to solve a derived planning problem that has only pure (nongraded) nondeterministic actions. A sound and complete partial order planning algorithm, called NDP, has been developed for such classical nondeterministic planning problems. The generation of γ-acceptable and optimally safe plans is achieved by two sound and complete planning algorithms: POSPLAN that relies on NDP, and POSPLAN* that can be seen as a hierarchical version of POSPLAN. The possibilistic planning framework is illustrated throughout the article by an example in the agronomic domain. © 1997 John Wiley & Sons, Inc.

Representing action: indeterminacy and ramifications

We define and study a high-level language for describing actions, more expressive than the action language A introduced by Gelfond and Lifschitz. The new language, AR , allows us to describe actions with indirect effects (ramifications), nondeterministic actions, and actions that may be impossible to execute. It has symbols for nonpropositional fluents and for the fluents that are exempt from the commonsense law of inertia. Temporal projection problems specified using the language AR can be represented as nested abnormality theories based on the situation calculus.

Explicit and implicit indeterminism reasoning about uncertain and contradictory specifications of dynamic systems

A high-level action semantics for specifying and reasoning about dynamic systems is presented which supports both uncertain knowledge (taken as explicit indeterminism) and contradictory information (taken as implicit indeterminism). We start by developing an action description language for intentionally representing nondeterministic actions in dynamic systems. We then study the different possibilities of interpreting contradictory specifications of concurrent actions. We argue that the most reasonable interpretation which allows for exploiting as much information as possible, is to take such conflicts as implicit indeterminism. As the second major contribution, we present a calculus for our resulting action semantics based on the logic programming paradigm including negation-as-failure and equational theories. Soundness and completeness of this encoding wrt. the notion of entailment in our action language is proved by taking the completion semantics for equational logic programs with negation.