Graph Relational Reinforcement Learning for Mobile Robot Navigation in Large-Scale Crowded Environments

Abstract

Mobile robot autonomous navigation in large-scale environments with crowded dynamic objects and static obstacles is still an essential yet challenging task. Recent works have demonstrated the potential of using deep reinforcement learning to enable autonomous navigation in crowds. However, only considering the human-robot interactions results in short-sighted and unsafe behaviors, and they typically use hand-crafted features and assume the global observation range, leading to large performance declines in large-scale crowded environments. Recent advances have shown the power of graph neural networks to learn local interactions among surrounding objects. In this paper, we consider autonomous navigation task in large-scale environments with crowded static and dynamic objects (such as humans). Particularly, local interactions among dynamic objects are learned for better-understanding their moving tendency and relational graph learning is introduced for aggregating both the object-object interactions and object-robot interactions. In addition, local observations are transformed into graphical inputs to achieve the scalability to various number of surrounding dynamic objects and various static obstacle patterns, and the globally guided reinforcement learning strategy is introduced to achieve the fixed-sized learning model even in large-scale complex environments. Simulation results validate our generalizability to various environments and advanced performance compared with existing works in large-scale crowded environments. In particular, our method with only local observations performs better than the benchmarks with global complete observability. Finally, physical robotic experiments demonstrate our effectiveness and practical applicability in real scenarios.