Comparing real-time and incremental heuristic search for real-time situated agents

Abstract

Real-time situated agents, such as characters in real-time computer games, often do not know the terrain in advance but automatically observe it within a certain range around themselves. They have to interleave searches with action executions to make the searches tractable when moving autonomously to user-specified coordinates. The searches face real-time requirements since it is important that the agents be responsive to the commands of the users and move smoothly. In this article, we compare two classes of fast heuristic search methods for these navigation tasks that speed up A* searches in different ways, namely real-time heuristic search and incremental heuristic search, to understand their advantages and disadvantages and make recommendations about when each one should be used. We first develop a competitive real-time heuristic search method. LSS-LRTA* is a version of Learning Real-Time A* that uses A* to determine its local search spaces and learns quickly. We analyze the properties of LSS-LRTA* and then compare it experimentally against the state-of-the-art incremental heuristic search method D* Lite on our navigation tasks, for which D* Lite was specifically developed, resulting in the first comparison of real-time and incremental heuristic search in the literature. We characterize when to choose each one of the two heuristic search methods, depending on the search objective and the kind of terrain. Our experimental results show that LSS-LRTA* can outperform D* Lite under the right conditions, namely when there is time pressure or the user-supplied h-values are generally not misleading.