Goal-driven Autonomy Research Articles

Learning from Demonstration for Goal-Driven Autonomy

Goal-driven autonomy (GDA) is a conceptual model for creating an autonomous agent that monitors a set of expectations during plan execution, detects when discrepancies occur, builds explanations for the cause of failures, and formulates new goals to pursue when planning failures arise. While this framework enables the development of agents that can operate in complex and dynamic environments, implementing the logic for each of the subtasks in the model requires substantial domain engineering. We present a method using case-based reasoning and intent recognition in order to build GDA agents that learn from demonstrations. Our approach reduces the amount of domain engineering necessary to implement GDA agents and learns expectations, explanations, and goals from expert demonstrations. We have applied this approach to build an agent for the real-time strategy game StarCraft. Our results show that integrating the GDA conceptual model into the agent greatly improves its win rate.

Ontological Knowledge for Goal-Driven Autonomy Agents in Starcraft

Starcraft, a commercial Real-Time Strategy (RTS) game that has enjoyed world-wide popularity (including televised professional matches), is a challenging domain for automated computer agents. Evidence of this difficulty comes not only from characteristics of the game (massive state space, stochastic actions, partial visibility, etc.) but also from three years of competitive entries in tournaments (i.e. AIIDE Annual Starcraft Competition) in which the best automated entry performs poorly against a human expert. We are interested in taking a new research direction: using semantic knowledge, such as description logic, to represent the game state with abstract concepts in order to perform high level actions.

Expectations for agents with goal-driven autonomy

ABSTRACT Goal-driven autonomy is an agent model for managing a dynamic environment by reasoning about current and potential goals while planning and acting. Since unexpected events and conditions may cause an agent’s goals and plans to become invalid or infeasible, an agent with goal-driven autonomy should monitor the environment against its expectations. Designed for dynamic, open, and partially observable environments, such an agent can create new goals or change its existing goals as needed. We present a formalisation of expectations for agents operating in these kinds of environments. Our formalisation includes situations where agents have the capability to sense the environment with some associated costs. We examine agent choices and behaviour in these domains and evaluate multiple approaches for selecting a subset of the agent’s sensing actions to execute. The contributions of this work are (1) a specification of different approaches to generating expectations; (2) a formalisation of the autonomy problem that minimises sensing costs; (3) a complexity analysis of the problem; (4) new algorithms for deciding which sensing actions to perform; and (5) empirical results demonstrating the benefit and cost of these approaches.

Intelligent active sonar via tuning of transmit waveform and detection threshold

We consider active sonar systems that translate high-level tasks to a set of parameter adaptations and arbitrate among competing demands, thereby incorporating cognitive processing in the system. We propose adopting the goal-driven autonomy (GDA) architecture presented in Klenk et al. [Computational Intelligence, May 2013] to realize intelligent active processing. Using the GDA architecture, the sonar system uses its observations to inform how system parameters are tuned to achieve a set of surveillance goals. In addition, the system is able to reason about its actions, identifying discrepancies between predicted and observed performance and modifying both parameters and goals accordingly. In this work, we focus on a tracking task and consider the transmit waveform and detection threshold as tunable parameters. We describe an active sonar system that tunes these two parameters based on observations of the physical environment and target characteristics, as well as current goals. The intelligent system uses its observations of the environment to generate estimates of target Doppler and of clutter density to tune waveform and detection threshold, respectively. Through simulation, we demonstrate the performance improvement achieved by the cognitive system when tracking a maneuvering target that is traveling through spatially non-uniform clutter.

GOAL‐DRIVEN AUTONOMY FOR RESPONDING TO UNEXPECTED EVENTS IN STRATEGY SIMULATIONS

To operate autonomously in complex environments, an agent must monitor its environment and determine how to respond to new situations. To be considered intelligent, an agent should select actions in pursuit of its goals, and adapt accordingly when its goals need revision. However, most agents assume that their goals are given to them; they cannot recognize when their goals should change. Thus, they have difficulty coping with the complex environments of strategy simulations that are continuous, partially observable, dynamic, and open with respect to new objects. To increase intelligent agent autonomy, we are investigating a conceptual model for goal reasoning called Goal‐Driven Autonomy (GDA), which allows agents to generate and reason about their goals in response to environment changes. Our hypothesis is that GDA enables an agent to respond more effectively to unexpected events in complex environments. We instantiate the GDA model in ARTUE (Autonomous Response to Unexpected Events), a domain‐independent autonomous agent. We evaluate ARTUE on scenarios from two complex strategy simulations, and report on its comparative benefits and limitations. By employing goal reasoning, ARTUE outperforms an off‐line planner and a discrepancy‐based replanner on scenarios requiring reasoning about unobserved objects and facts and on scenarios presenting opportunities outside the scope of its current mission.

Applying Goal-Driven Autonomy to StarCraft

One of the main challenges in game AI is building agents that can intelligently react to unforeseen game situations. In real-time strategy games, players create new strategies and tactics that were not anticipated during development. In order to build agents capable of adapting to these types of events, we advocate the development of agents that reason about their goals in response to unanticipated game events. This results in a decoupling between the goal selection and goal execution logic in an agent. We present a reactive planning implementation of the Goal-Driven Autonomy conceptual model and demonstrate its application in StarCraft. Our system achieves a win rate of 73% against the built-in AI and outranks 48% of human players on a competitive ladder server.

Goal-Driven Autonomy in a Navy Strategy Simulation

Modern complex games and simulations pose many challenges for an intelligent agent, including partial observability, continuous time and effects, hostile opponents, and exogenous events. We present ARTUE (Autonomous Response to Unexpected Events), a domain-independent autonomous agent that dynamically reasons about what goals to pursue in response to unexpected circumstances in these types of environments. ARTUE integrates AI research in planning, environment monitoring, explanation, goal generation, and goal management. To explain our conceptualization of the problem ARTUE addresses, we present a new conceptual framework, goal-driven autonomy, for agents that reason about their goals. We evaluate ARTUE on scenarios in the TAO Sandbox, a Navy training simulation, and demonstrate its novel architecture, which includes components for Hierarchical Task Network planning, explanation, and goal management. Our evaluation shows that ARTUE can perform well in a complex environment and that each component is necessary and contributes to the performance of the integrated system.