Modeling CGFs behavior by an extended option based learning behavior trees

Abstract

In military simulation for training, behavior modeling of computer generated forces (CGFs) can be problematic because of the constrained and adaptive model requirements. As an emerging AI scripting technique, behavior trees (BTs) have fantastic advantages of modularity and scalability over finite state machines (FSMs) to encode such CGFs, but still suffers from time-consuming, repetitive endeavor and lack of nuanced variations. In this paper, we propose an extended option based learning method to allow a flexible improvement for a predefined BT controller of CGFs. Based on the original option based learning behavior tree framework, we extend the original BT elements and propose bottom-up reward accumulation rules to allow flexible multiple selector policy optimization in the BT. Furthermore, the learned policy is reorganized transparently as condition nodes of original selected behavior to allow easier model validation. We apply our method to a predator-prey adversarial simulation scenario to improve a rough BT controller, and experimental results show that our method outperforms its competitors to facilitate BT design by achieving better final behavior performance.