Behavior Trees Research Articles

Deliberative robotics – a novel interactive control framework enhancing human-robot collaboration

Deliberative robots adapt their behavior from cobot to industrial arms as a result of the interaction dynamic with the human teammates and production context. This powerful capability is instrumental to boost robots' adoption within typically manual manufacturing contexts, by enhancing productivity while preserving human safety and job quality. The deliberative feature relies upon a modular planning and control infrastructure linked to a behavioral tree framework determining the robot response to the operating context and the human physical and cognitive status. The benefits of the approach have been validated through an industrial use case demanding enhanced human-robot collaboration in the electronics assembly value chain.

Research on the auxiliary treatment system of childhood autism based on virtual reality

ABSTRACT Autism, is a representative disease of neurodevelopmental disorders. Numerous autistic individuals are drawn to familiar settings and regular routines because of their behavioural, information processing, and sensory characteristics. Traditional treatment methods are restricted by medical and educational resources . The emergence of virtual reality technology (VR) has provided new ideas for the rehabilitation of autism.Based on the cognitive theory of children with autism , this paper designs an auxiliary treatment system for children with autism based on virtual reality. Through multiple training scenarios and using behaviour tree decision technology to process the intelligent decision-making of the role, the child can interact with the role and complete tasks in the present situation. Through experiments, children with autism have improved their hands-on ability, social skills and physical coordination.This shows that the system has a significant effect on the treatment of autism patients,confirming the feasibility of introducing virtual reality technology into autism treatment.

Behavior Trees in Robot Control Systems

In this article, we provide a control-theoretic perspective on the research area of behavior trees in robotics. The key idea underlying behavior trees is to make use of modularity, hierarchies, and feedback in order to handle the complexity of a versatile robot control system. Modularity is a well-known tool to handle software complexity by enabling the development, debugging, and extension of separate modules without detailed knowledge of the entire system. A hierarchy of such modules is natural, since robot tasks can often be decomposed into a hierarchy of subtasks. Finally, feedback control is a fundamental tool for handling uncertainties and disturbances in any low-level control system, but in order to enable feedback control on the higher level, where one module decides what submodule to execute, information regarding the progress and applicability of each submodule needs to be shared in the module interfaces. We describe how these three concepts can be used in theoretical analysis, practical design, and extensions and combinations with other ideas from control theory and robotics.

Reinforcement Learning Methods to Evaluate the Impact of AI Changes in Game Design

Game development has become a long process that requires many professionals working on a project during several months or years. With this scenario the re-utilization of resources is crucial not only to alleviate the process but also to bring coherence into the final product. In this paper we focus on the reuse of NPCs and the problems it brings about. In particular it is common to have different breeds (or personalities) of NPCs that are placed on different levels on the game. The problem arises when their behaviors are fine-tuned to accommodate a specific level needs without taking into consideration that this change may alter their performance on previous already-tested levels. The paper presents the application of reinforcement learning together with behavior trees to automatically test if modifications to the AIs of a stealth game have an impact on the user experience. Our experiments reveal that this approach provides a way of diagnosing alterations in level gameplay that correspond to the effects observed by human testers.

Dynamic Expansion of Behaviour Trees

Artificial intelligence in games is typically used for creating player's opponents. Manual edition of intelligent behaviors for Non-Player Characters (NPCs) of games is a cumbersome task that needs experienced designers. Our research aims to assist designers in this task. Behaviours typically use recurring patterns, so that experience and reuse are crucial aspects for behavior design. The use of hierarchical state machines allows working on different abstraction levels, sharing transitions and reusing pieces from the more detailed levels. However, the static nature of the design process does not release the designer from the burden to completely specify each behaviour. Our approach applies Case-Based Reasoning (CBR) techniques to retrieve and reuse stored behaviors represented as hierarchical state machines (actually, behaviour trees). In this paper we focus on dynamic retrieval of behaviours taking into account the world state and the underlying goals to select the most appropriate state machine to guide the NPC behaviour. The global behaviour of the NPC is dynamically built in run time querying the CBR system. We exemplify our approach through a serious game, developed by our research group, with gameplay elements from First-Person Shooter (FPS) games.

Discovering optimal strategy in tactical combat scenarios through the evolution of behaviour trees

In this paper we address the problem of automatically discovering optimal tactics in a combat scenario in which two opposing sides control a number of fighting units. Our approach is based on the evolution of behaviour trees, combined with simulation-based evaluation of solutions to drive the evolution. Our behaviour trees use a small set of possible actions that can be assigned to a combat unit, along with standard behaviour tree constructs and a novel approach for selecting which action from the tree is performed. A set of test scenarios was designed for which an optimal strategy is known from the literature. These scenarios were used to explore and evaluate our approach. The results indicate that it is possible, from the small set of possible unit actions, for a complex strategy to emerge through evolution. Combat units with different capabilities were observed exhibiting coordinated team work and exploiting aspects of the environment.

Building the Foundation of Robot Explanation Generation Using Behavior Trees

As autonomous robots continue to be deployed near people, robots need to be able to explain their actions. In this article, we focus on organizing and representing complex tasks in a way that makes them readily explainable. Many actions consist of sub-actions, each of which may have several sub-actions of their own, and the robot must be able to represent these complex actions before it can explain them. To generate explanations for robot behavior, we propose using Behavior Trees (BTs), which are a powerful and rich tool for robot task specification and execution. However, for BTs to be used for robot explanations, their free-form, static structure must be adapted. In this work, we add structure to previously free-form BTs by framing them as a set of semantic sets {goal, subgoals, steps, actions} and subsequently build explanation generation algorithms that answer questions seeking causal information about robot behavior. We make BTs less static with an algorithm that inserts a subgoal that satisfies all dependencies. We evaluate our BTs for robot explanation generation in two domains: a kitting task to assemble a gearbox, and a taxi simulation. Code for the behavior trees (in XML) and all the algorithms is available at github.com/uml-robotics/robot-explanation-BTs.

On the Implementation of Behavior Trees in Robotics

There is a growing interest in Behavior Trees (BTs) as a tool to describe and implement robot behaviors. BTs were devised in the video game industry and their adoption in robotics resulted in the development of ad-hoc libraries to design and execute BTs that fit complex robotics software architectures. While there is broad consensus on how BTs work, some characteristics rely on the implementation choices done in the specific software library used. In this letter, we outline practical aspects in the adoption of BTs and the solutions devised by the robotics community to fully exploit the advantages of BTs in real robots. We also overview the solutions proposed in open-source libraries used in robotics, we show how BTs fit in robotic software architecture, and we present a use case example.

An Expressiveness Hierarchy of Behavior Trees and Related Architectures

In this letter, we provide a formal framework for comparing the expressive power of Behavior Trees (BTs) to other action selection architectures. Taking inspiration from the analogous comparisons of structural programming methodologies, we formalise the concept of `expressiveness'. This leads us to an expressiveness hierarchy of control architectures, which includes BTs, Decision Trees (DTs), Teleo-reactive Programs (TRs) and Finite State Machines (FSMs). By distinguishing between BTs with auxiliary variables and those without, we demonstrate the existence of a trade-off in BT design between readability and expressiveness. We discuss what this means for BTs in practice.

Model-Driven AI for Games: Research Plan

The field of game AI is largely industry driven, lacking an agreed upon formalism for AI representation. Ad-hoc scripting languages, simple finite state machines, behaviour trees, and planners are employed, but not in a fashion adhering to any standard. As a result, reuse is sparse between games and formal analysis techniques are undeveloped. As research for a Ph.D. thesis, we propose to show that a layered Statechart-based AI is a suitable formalism for Game AI, enabling the use of model-driven development techniques such as reuse and high-level analysis including model-checking. The fundamentally modular nature of this approach leads naturally to reuse as a fundamental component of the design process. Supported by a clearly defined formalism, useful behavioural analyses become possible, such as testing reactions to various inputs at design time. We also explore transformations at the modelling level to enable procedural generation, allowing rapid deployment of varying AIs. Additionally, such a model allows for the generation of efficient code that can be directly inserted into games. Tool support for reuse, generation, and analysis will be developed, then employed in creating an industrial scale AI, proving that this formalism is appropriate for industrial use.

Statechart-Based AI in Practice

Layered Statechart-based AI shows considerable promise by being a highly modular, reusable, and designer friendly approach to game AI. Here we demonstrate the viability of this approach by replicating the functionality of a full-featured and commercial-scale behaviour tree AI within a non-commercial game framework. As well as demonstrating that layered Statecharts are both usable and amply expressive, our experience highlights the value of several, previously unidentified design considerations, such as sensor patterns, the necessity of subsumption, and the utility of orthogonal regions. These observations point towards simplified, higher-level AI construction techniques that can reduce the complexity of AI design and further enhance reuse.

Adapting AI Behaviors To Players in Driver San Francisco: Hinted-Execution Behavior Trees

The creative nature of games makes trying new ideas desirable, but these changes are sometimes very risky. We need to find ways to minimize risks while we build innovative experiences. Driver San Francisco did this by using Hinted-execution Behavior Trees; this technique allows developers to modify existing AI behaviors dynamically with very low risk, and was used to adapt Driver’s getaway AI to players’ skills.

An AI System for Large Open Virtual World

In recent years, computer games have reached unprecedented level of graphical fidelity to the real world. As the non-player characters (NPCs) in the game world look more and more realistic, players expect them to manifest believable behavior as well. This is accented especially in games that feature large open worlds, which players may explore freely and it is thus not possible to explicitly account for all possible player interactions. In this paper we focus mainly on ambient AI - the logic behind day to day behaviors of NPCs as they sleep, work and entertain themselves in the virtual world. In this context, it is of great importance to build a system that handles many NPCs (up to several hundreds) quickly. In this paper we report on an implementation of a particular AI system that was approved for deployment in an upcoming high-budget game. The system features a hierarchy of control similar to the subsumption architecture and a visual agent-based language inspired by behavior trees. We describe the challenges involved in building such a system and specific design decisions we have made that let us achieve a level of behavioral fidelity unmatched by existing games. Finally we evaluate the performance of the system in a realistic setting.

A Proposal for a Unified Agent Behaviour Framework

Games have used different mechanisms along its history to provide agent behavior: FSMs, utility systems, behavior trees and planning methods. In this paper, we present an architecture that aims at incorporating all these approaches into trees of event handling nodes with behaviours as leaves, using rules for combining actions akin to utility systems. This formalism aims to develop hybrid systems in an easier way.

Capturing the Essence: Towards the Automated Generation of Transparent Behavior Models

Hand-coded finite-state machines and behavior trees are the go-to techniques for artificial intelligence (AI) developers that want full control over their character's bearing. However, manually crafting behaviors for computer-controlled agents is a tedious and parameter-dependent task. From a high-level view, the process of designing agent AI by hand usually starts with the determination of a suitable set of action sequences. Once the AI developer has identified these sequences he merges them into a complete behavior by specifying appropriate transitions between them. Automated techniques, such as learning, tree search and planning, are on the other end of the AI toolset's spectrum. They do not require the manual definition of action sequences and adapt to parameter changes automatically. Yet AI developers are reluctant to incorporate them in games because of their performance footprint and lack of immediate designer control. We propose a method that, given the symbolic definition of a problem domain, can automatically extract a transparent behavior model from Goal-Oriented Action Planning (GOAP). The method first observes the behavior exhibited by GOAP in a Monte-Carlo simulation and then evolves a suitable behavior tree using a genetic algorithm. The generated behavior trees are comprehensible, refinable and as performant as hand-crafted ones.

Semi-Autonomous Behaviour Tree-Based Framework for Sorting Electric Vehicle Batteries Components

The process of recycling electric vehicle (EV) batteries currently represents a significant challenge to the waste management automation industry. One example of it is the necessity of removing and sorting dismantled components from EV battery pack. This paper proposes a novel framework to semi-automate the process of removing and sorting different objects from an EV battery pack using a mobile manipulator. The work exploits the Behaviour Trees model for cognitive task execution and monitoring, which links different robot capabilities such as navigation, object tracking and motion planning in a modular fashion. The framework was tested in simulation, in both static and dynamic environments, and it was evaluated based on task time and the number of objects that the robot successfully placed in the respective containers. Results suggested that the robot’s success rate in accomplishing the task of sorting the battery components was 95% and 82% in static and dynamic environments, respectively.

Retrieval of behavior trees using map-and-reduce technique

There has been an increased interest in the creation of AI social agents who possess complex behaviors that allow them to perform social interactions. Behavior trees provide a plan model execution that has been widely used to build complex behaviors for AI social agents. Behavior trees can be represented in the form of a memory structure known as cognitive scripts, which would allow them to evolve through further development over multiple exposure to repeated enactment of a particular behavior or similar ones. Behavior trees that share the same context will then be able to learn from each other resulting in new behavior trees with richer experience. The main challenge appears in the expensive cost of retrieving contextually similar behavior trees (scripts) from a repertoire of scripts to allow for that learning process to occur. This paper introduces a novel application of map-and-reduce technique to retrieve cognitive with low computational time and memory allocation. The paper focuses on the design of a corpus of cognitive scripts, as a knowledge engineering key challenge, and the application of map-and-reduce with semantic information to retrieve contextually similar cognitive scripts. The results are compared to other techniques used to retrieve cognitive scripts in the literature, such as Pharaoh which uses the least common parent (LCP) technique in its core. The results show that the map-and-reduce technique can be successfully used to retrieve cognitive scripts with high retrieval accuracy of 92.6%, in addition to being cost effective.

BT Expansion: a Sound and Complete Algorithm for Behavior Planning of Intelligent Robots with Behavior Trees

Behavior Trees (BTs) have attracted much attention in the robotics field in recent years, which generalize existing control architectures and bring unique advantages for building robot systems. Automated synthesis of BTs can reduce human workload and build behavior models for complex tasks beyond the ability of human design, but theoretical studies are almost missing in existing methods because it is difficult to conduct formal analysis with the classic BT representations. As a result, they may fail in tasks that are actually solvable. This paper proposes BT expansion, an automated planning approach to building intelligent robot behaviors with BTs, and proves the soundness and completeness through the state-space formulation of BTs. The advantages of blended reactive planning and acting are formally discussed through the region of attraction of BTs, by which robots with BT expansion are robust to any resolvable external disturbances. Experiments with a mobile manipulator and test sets are simulated to validate the effectiveness and efficiency, where the proposed algorithm surpasses the baseline by virtue of its soundness and completeness. To the best of our knowledge, it is the first time to leverage the state-space formulation to synthesize BTs with a complete theoretical basis.

Extending Behavior Trees for Representing and Planning Robot Adjoint Actions in Partially Observable Environments

Extending Behavior Trees for Representing and Planning Robot Adjoint Actions in Partially Observable Environments

DSD - Dynamic Stack Decider

We present the Dynamic Stack Decider (DSD), a lightweight open-source control architecture. It combines different well-known approaches and is inspired by behavior trees as well as hierarchical state machines. The DSD allows to design and structure complex behavior of robots as well as software agents while providing easy maintainability. Challenges that often occur in robotics, i.e., a dynamic environment and situation uncertainty, remain well-manageable. Furthermore, it allows fast modifications of the control flow, while providing the state-fullness of a state machine. The approach allows developing software using a simple Domain Specific Language (DSL) which defines the control flow and two types of elements that contain the programmed parts. The framework takes care of executing the demanded portions of the code and gives, due to its stack-like internal representation, the ability to verify preconditions while maintaining a clear structure. The presented software was used in different robotic scenarios and showed great performance in terms of flexibility and structuredness.