Human-Flow-Aware Long-Term Mobile Robot Task Planning Based on Hierarchical Reinforcement Learning

Abstract

The difficulty in finding long-term planning policies for a mobile robot increases when operating in crowded and dynamic environments. State-of-the-art approaches do not consider cues of human-robot-shared dynamic environments. Aiming to fill this gap, we present a novel Human-Flow-Aware Guided Hierarchical Dyna-Q (HA-GHDQ) algorithm, which solves long-term robot task planning problems by using human motion patterns encoded in Maps of Dynamics (MoDs). To tackle the complexity of long-term robot operation in dynamic environments, we propose a combination of symbolic planning and Hierarchical Reinforcement Learning (HRL) that generates robot policies considering cost information derived from MoDs. We evaluated HA-GHDQ in a factory environment with two simulation and one real-world datasets to complete a transportation-and-assembly task. Our approach outperforms the baselines with respect to sample efficiency and final plan quality. Moreover, we show that it is more adaptable and robust against environmental changes than the baselines.