Learning situation-dependent costs: Improving planning from probabilistic robot execution

Abstract

Physical domains are notoriously hard to model completely and correctly, especially to capture the dynamics of the environment. In this article, we present Rogue, a robot that learns from its execution experiences. Since actions may have different costs under different conditions, we introduce the concept of situation-dependent rules, in which situational features are attached to the costs or probabilities, reflecting patterns and dynamics encountered in the environment. Rogue extracts learning opportunities from massive, continual, probabilistic execution traces. It then correlates these learning opportunities with environmental features, creating situation-dependent costs for its actions. We present the development and use of these rules for a robotic path planner. Our empirical results show that situation-dependent rules effectively improve the planner’s model of the environment, thus allowing the planner to predict and avoid failures, to create plans that are tailored to the real world, and to respond to a changing environment.