Behavior Trees Research Articles

Coordination of NPCs in multi-agent systems based on behavior trees

Abstract. This paper addresses the design and coordination challenges of controlling non-player characters' (NPCs) behaviors in multi-agent systems (MAS) using behavior trees (BTs), which are preferred over finite state machines (FSMs) due to their hierarchical structure and ease of maintenance. While BTs effectively resolve the question of "What to do next?" for individual NPCs, their application in MAS, particularly with the integration of a blackboard system for central control, reveals limitations in efficiency, robustness, and heuristic capacity as system complexity increases. To explore solutions to these challenges, this study analyzes various algorithms that enhance the functionality of behavior trees within MAS. The research focuses on three primary areas: the optimization of behavior trees, the development of behavior tree search algorithms, and the improvement of communication algorithms within BTs. Methods involve a comparative analysis of existing and new algorithmic approaches to identify and address inefficiencies in NPC coordination. The findings indicate that advanced behavior tree configurations, when combined with innovative search and communication strategies, significantly improve the coordination, robustness, and operational efficiency of MAS. These enhancements allow for more dynamic and responsive NPC interactions in complex gaming environments.

Autonomous Robot Task Execution in Flexible Manufacturing: Integrating PDDL and Behavior Trees in ARIAC 2023.

The Agile Robotics for Industrial Automation Competition (ARIAC) was established to advance flexible manufacturing, aiming to increase the agility of robotic assembly systems in unstructured and dynamic industrial environments. ARIAC 2023 introduced eight agility challenges involving faulty parts, flipped parts, faulty grippers, robot malfunctions, sensor blackouts, high-priority orders, insufficient parts, and human safety. Given the unpredictability of these scenarios, it is impractical to develop a specific strategy for each possible situation. To address these issues, this paper presents a hierarchical framework for autonomous robotic task generation and execution in dynamic scenarios. The framework is divided into a task level and an execution level. Initially, an immediate task management strategy is adopted at the task level, which reasonably decomposes dynamic tasks and allocates short-term tasks to the floor robot and ceiling robot. Later, at the execution level, each robot is designed with an agent architecture that combines PDDL planning with the quick response of behavior trees. Finally, the effectiveness and practicality of the proposed framework were thoroughly validated in ARIAC 2023.

Evaluating behavior trees

Behavior trees (BTs) are increasingly popular in the robotics community. Yet in the growing body of published work on this topic, there is a lack of consensus on what to measure and how to quantify BTs when reporting results. This is not only due to the lack of standardized measures, but due to the sometimes ambiguous use of definitions to describe BT properties. This work provides a comprehensive overview of BT properties the community is interested in, how they relate to each other, the metrics currently used to measure BTs, and whether the metrics appropriately quantify those properties of interest. Finally, we provide the practitioner with a set of metrics to measure, as well as insights into the properties that can be derived from those metrics.By providing this holistic view of properties and their corresponding evaluation metrics, we hope to improve clarity when using BTs in robotics. This more systematic approach will make reported results more consistent and comparable when evaluating BTs.

Enhancing Autonomous Underwater Vehicle Decision Making through Intelligent Task Planning and Behavior Tree Optimization

The expansion of underwater scenarios and missions highlights the crucial need for autonomous underwater vehicles (AUVs) to make informed decisions. Therefore, developing an efficient decision-making framework is vital to enhance productivity in executing complex tasks within tight time constraints. This paper delves into task planning and reconstruction within the AUV control decision system to enable intelligent completion of intricate underwater tasks. Behavior trees (BTs) offer a structured approach to organizing the switching structure of a hybrid dynamical system (HDS), originally introduced in the computer game programming community. In this research, an intelligent search algorithm, MCTS-QPSO (Monte Carlo tree search and quantum particle swarm optimization), is proposed to bolster the AUV’s capacity in planning complex task decision control systems. This algorithm tackles the issue of the time-consuming manual design of control systems by effectively integrating BTs. By assessing a predefined set of subtasks and actions in tandem with the complex task scenario, a reward function is formulated for MCTS to pinpoint the optimal subtree set. The QPSO algorithm is then leveraged for subtree integration, treating it as an optimal path search problem from the root node to the leaf node. This process optimizes the search subtree, thereby enhancing the robustness and security of the control architecture. To expedite search speed and algorithm convergence, this paper recommends reducing the search space by pre-grouping conditions and states within the behavior tree. The efficacy and superiority of the proposed algorithm are validated through security and timeliness evaluations of the BT, along with comparisons with other algorithms for automatic AUV decision control behavior tree design. Ultimately, the effectiveness and superiority of the proposed algorithm are corroborated through simulations on a multi-AUV complex task platform, showcasing its practical applicability and efficiency in real-world underwater scenarios.

The differential impact of climate interventions along the political divide in 60 countries

A major barrier to climate change mitigation is the political polarization of climate change beliefs. In a global experiment conducted in 60 countries (N = 51,224), we assess the differential impact of eleven climate interventions across the ideological divide. At baseline, we find political polarization of climate change beliefs and policy support globally, with people who reported being liberal believing and supporting climate policy more than those who reported being conservative (Cohen’s d = 0.35 and 0.27, respectively). However, we find no evidence for a statistically significant difference between these groups in their engagement in a behavioral tree planting task. This conceptual-behavioral polarization incongruence results from self-identified conservatives acting despite not believing, rather than self-identified liberals not acting on their beliefs. We also find three interventions (emphasizing effective collective actions, writing a letter to a future generation member, and writing a letter from the future self) boost climate beliefs and policy support across the ideological spectrum, and one intervention (emphasizing scientific consensus) stimulates the climate action of people identifying as liberal. None of the interventions tested show evidence for a statistically significant boost in climate action for self-identified conservatives. We discuss implications for practitioners deploying targeted climate interventions.

Improving the Performance of Learned Controllers in Behavior Trees Using Value Function Estimates at Switching Boundaries

Improving the Performance of Learned Controllers in Behavior Trees Using Value Function Estimates at Switching Boundaries

Translating Words to Worlds: Zero-Shot Synthesis of 3D Terrain from Textual Descriptions Using Large Language Models

The current research on text-guided 3D synthesis predominantly utilizes complex diffusion models, posing significant challenges in tasks like terrain generation. This study ventures into the direct synthesis of text-to-3D terrain in a zero-shot fashion, circumventing the need for diffusion models. By exploiting the large language model’s inherent spatial awareness, we innovatively formulate a method to update existing 3D models through text, thereby enhancing their accuracy. Specifically, we introduce a Gaussian–Voronoi map data structure that converts simplistic map summaries into detailed terrain heightmaps. Employing a chain-of-thought behavior tree approach, which combines action chains and thought trees, the model is guided to analyze a variety of textual inputs and extract relevant terrain data, effectively bridging the gap between textual descriptions and 3D models. Furthermore, we develop a text–terrain re-editing technique utilizing multiagent reasoning, allowing for the dynamic update of the terrain’s representational structure. Our experimental results indicate that this method proficiently interprets the spatial information embedded in the text and generates controllable 3D terrains with superior visual quality.

Embedding multi-agent reinforcement learning into behavior trees with unexpected interruptions

Behavior trees have attracted great interest in computer games and robotic applications. However, it lacks the learning ability for dynamic environments. Previous works combining behavior trees with reinforcement learning either need to construct an independent sub-scenario or train the learning method over the whole game, which is not suited for complex multi-agent games. In this paper, a framework is proposed, named as MARL-BT, that embeds multi-agent reinforcement learning methods into behavior trees. Following the running mechanism of behavior trees, we design the way of collecting samples and the training procedure. Further, we point out a special phenomenon in MARL-BT, i.e., the unexpected interruption, and present an action masking technique to remove its harmful effect on learning performance. Finally, we make extensive experiments on the 11 versus 11 full game in Google Research Football. The introduced MARL-BT framework could get an 11.507% improvement compared to pure BT for certain scenarios. The action masking technique could greatly improve the performance of the learning method, i.e., the final reward is improved around 100% times for a sub-task.

Empathy-Aware Behavior Trees for Social Care Decision Systems

There is growing attention on the importance of building intelligent systems where humans and Artificial Intelligence-based systems (AIs) form teams exploiting the potentially synergistic relationships between humans and automation. In the last decade, the computational modeling of empathy has gained increasing attention. Empowering interactive agents with empathic capabilities leads, on the human’s side, to more trust, increases engagement, and thus interaction length, helps cope with stress. These findings suggest that agents endowed with empathy may enhance social interaction in educational applications, artificial companions, medical assistants, and gaming applications. This article focuses on modeling the empathic behavior of virtual agents interacting with humans. We propose a formal model that enables virtual agents to exhibit empathic, emotional behavior. Specifically, we extend the modeling of empathy via behavior trees with a new type of node allowing the specification of various kinds of empathy. Using the proposed extension, we show how different agents’ reactive behavior can be modeled.

Multi-modal human motion recognition based on behaviour tree

Multi-modal human motion recognition based on behaviour tree

Research on Virtual Character Behavior Simulation and Control Algorithm in Digital Movie Production

Abstract Aiming at the inefficient design of virtual character behavior in traditional digital movies, this paper proposes to construct a virtual character behavior and control model. The virtual character’s behavior is constructed using a behavior tree, and perception and decision-making behavior models are created to optimize the visual perception of the virtual character. After the behavioral modeling is completed, the action controller is used to generate the character’s behavioral actions, combined with the physical dynamics method to capture the character’s trajectory, and the feedback control theory is applied to output the character’s joint rotation data information to realize the behavioral control of the virtual character. In the digital movie scene design of firefighters fighting and rescuing, the acquisition and execution time of a single behavioral modeling sample of this paper’s virtual character behavioral modeling method is 0.01s~0.03s, and the maximum running iteration time is 63.83s, which is faster than the agent-based behavioral modeling method. Compared to the traditional virtual character behavioral control method, which has an offset of more than 4mm, the virtual character firefighter generated in this paper has an offset of less than 0.3mm, which provides better control accuracy.

Multi-modal human motion recognition based on Behaviour tree

Multi-modal human motion recognition based on Behaviour tree

Application of Automatic Driving Task Sequentialisation Monitoring for Band Operator Robots

Abstract Robotics is increasingly advancing across diverse sectors of industry and daily life, with particularly promising applications in autonomous driving. This research develops a robotic autopilot task serialization management system and conducts an in-depth analysis of how robot path planning influences serialization monitoring. The path planning system (PPS) has been enhanced through the integration of a behavioral tree path planning model and hierarchical decision-making processes. The performance of the newly developed PPS algorithm is evaluated through comparative simulation experiments against established path planning algorithms, such as Particle Swarm Optimization (PSO) and Artificial Bee Colony (ABC) algorithms. The findings reveal that the PPS algorithm consistently achieves superior outcomes, planning paths with the shortest average length (188.673) and the lowest standard deviation (1.1547). Furthermore, it demonstrates optimal path efficiency and minimum energy consumption compared to four other migration algorithms. Additionally, the study observes the migration rates for node radii of 60 and 120 mm, which are 26% and 70%, respectively. Notably, as the number of robots and nodes increases, the migration rate stabilizes, underscoring the scalability and effectiveness of the PPS algorithm in complex environments.

GOBT: A Synergistic Approach to Game AI Using Goal-Oriented and Utility-Based Planning in Behavior Trees

GOBT: A Synergistic Approach to Game AI Using Goal-Oriented and Utility-Based Planning in Behavior Trees

Analysis of the Role and Effect of Behavior Trees (DA) in Driving Virtual Character Behaviors in Role-Playing Games (RPG)

Rol yapma oyunları (RYO), bir bilgisayar oyununda sanal bir karakterin kontrol edilmesine dayanan bir oyun türüdür. Günümüzün dinamik oyun dünyasında, oyuncuların karakterlerin gelişimini şekillendirmesi, hikâyenin seyrini etkilemesi ve oyun evrenindeki kararları belirlemesi gibi unsurlar, RYO türünün temel özelliklerini oluşturur. Davranış ağacı (DA), bilgisayar oyunları gibi uygulamalarda karakterlerin veya nesnelerin davranışlarını yönlendirmek için kullanılan hiyerarşik ve yapılandırılmış bir programlama modelidir. Bu çalışmada, Sanal Gerçeklik (SG) ortamında RYO türünde DA'ların kullanımının sanal karakter gelişimindeki, hikâye ilerlemesindeki ve oyuncu kararlarının yönlendirilmesindeki rolünün ve etkisinin analiz edilmesi amaçlanmıştır. Çalışmanın temel motivasyonu, DA’ların RYO oyunlarında kullanımının, oyuncu deneyimini dinamik olarak şekillendirerek zenginleştireceği oluşturmaktadır. Bu motivasyon DA modelinin diğer oyun türlerinde olduğu gibi RYO türünde de oyuncuların karakterlerin davranışlarını etkileyerek daha kişisel bir deneyim yaşamalarını ve oyun deneyimini artırmalarını sağlayabileceğine dayanmaktadır. Bu bağlamda çalışmamızda UNITY 3D ortamında geliştirilen bir RYO oyunu üzerinde (VR-RPG-GAME) gerçekleştirilen deneylerle test edilmiş ve deneylerden elde edilen veriler analiz edilmiştir. Söz konusu deneylerde, oyuncuların karakterlerin karar verme yetenekleri, hikâye ilerlemesi, düşman etkileşimi ve karakter gelişimi üzerindeki etkilerinin analiz edilmesi amaçlanmıştır. Çalışmada DA'ların RYO oyun deneyimi üzerindeki gerçek etkisini anlamak için sistematik bir yaklaşım benimsenmiştir. VR-RPG-GAME’in geliştirilmesinde C# programlama dili, DA kütüphaneleri ve eklentileri, SG teknolojisi, UNITY Asset Store ve grafik tasarım araçları kullanılmıştır. Bu çalışmanın özellikle RYO türünün kullanıcılar tarafından DA tabanlı olarak geliştirilmesi açısından bilgisayar oyun teknolojisi çalışma alanına katkı sağlayacağı değerlendirilmektedir.

How to early integrate operational diagnosis objectives in model‐driven engineering processes: A methodological proposal based on fault and behavior trees

AbstractTo help operators perform their diagnosis tasks more efficiently, the authors put forward a novel methodology introducing a new type of model, dedicated to operations, co‐created in parallel with the design models, in the preliminary stages of system development, using the language semantics of behavior trees. In this paper, the authors present the need for this new model type as expressed by the industry and justify their choice for adopting behavior trees, while illustrating in detail the proposed methodology.

Research on the Innovative Path of College English Teaching Based on Deep Reinforcement Learning

Abstract College English is a very flexible and important subject that needs to be constantly innovated. For the shortcomings of traditional teaching methods, this paper constructs a teaching behavior tree based on reinforcement learning. By analyzing the direction of the behavior tree, using the path planning algorithm to propose better teaching strategies, combined with the Q learning algorithm, the teaching theme and the environment of the information are timely updated in the Q value table to update the teaching behavior selection strategy, and ultimately obtain the teaching behavior tree. The reinforcement learning algorithm based on Q -function interacts with the environment to directly train the method of generating the behavior tree to check whether the teaching behavior tree reaches the optimal standard. Combine reinforcement deep learning with English teaching to introduce a new approach to teaching English. The results show that on the diversified evaluation behavior, teachers of 985 colleges scored significantly higher than teachers of 211 colleges, with a score of 3.05 for teachers of 985 colleges and 2.68 for teachers of 211 colleges. Among the two teaching factors, the F-value of the diversified evaluation is 4.601, and the F-value of the personalized teaching is 3.596, which indicates that both of them have a greater influence on the students and they are able to improve the English proficiency of the students. This study provides a new reference direction for college English teaching, which can effectively improve students’ practical skills.

A Novel Control Architecture Based on Behavior Trees for an Omni-Directional Mobile Robot

Robotic systems are increasingly present in dynamic environments. This paper proposes a hierarchical control structure wherein a behavior tree (BT) is used to improve the flexibility and adaptability of an omni-directional mobile robot for point stabilization. Flexibility and adaptability are crucial at each level of the sense–plan–act loop to implement robust and effective robotic solutions in dynamic environments. The proposed BT combines high-level decision making and continuous execution monitoring while applying non-linear model predictive control (NMPC) for the point stabilization of an omni-directional mobile robot. The proposed control architecture can guide the mobile robot to any configuration within the workspace while satisfying state constraints (e.g., obstacle avoidance) and input constraints (e.g., motor limits). The effectiveness of the controller was validated through a set of realistic simulation scenarios and experiments in a real environment, where an industrial omni-directional mobile robot performed a point stabilization task with obstacle avoidance in a workspace.

Non-Playable Character (NPC) based on Behaviour Tree for Enhanced Immersive Experience in the Serious Game "Warik's Adventure"

Video games, or digital games, have transcended their traditional role of mere entertainment and are now being leveraged for more serious purposes, such as education and cultural preservation. Serious games offer the advantage of presenting content in an interactive and enjoyable manner. A crucial element in gameplay for creating interactivity and a pleasurable experience is the non-playable character (NPC). In serious games, replayability is pivotal for optimizing the comprehension of learning content, with the gameplay experience being a key factor influencing replayability. While games are known for their entertaining aspects, immersion design in serious games often falls short, resulting in a subpar experience. The NPC design in this study is grounded in appreciative learning, emphasizing positive outcomes such as possibilities, peak successes, exploration, and future optimism. The reward activity is structured across four levels: Discovery, Dream, Design, and Destiny. Behaviour Tree is employed to govern NPC behavior throughout all four stages. To evaluate the effectiveness of the immersive design, the Game Experience Questionnaire (GEQ) is employed, measuring three primary components—sensory, imaginative, and challenge-based immersion. The GEQ result indicates a score of 3.2, showcasing a slight improvement from the previous version's score of 3.07. This research contributes to the enhancement of serious game design by focusing on NPC behavior and immersive experiences, ultimately fostering more effective and engaging learning environments.

Towards Efficient Robotic Software Development by Reusing Behavior Tree Structures for Task Planning Paradigms

Towards Efficient Robotic Software Development by Reusing Behavior Tree Structures for Task Planning Paradigms