Generalization Ability of Deep Reinforcement Learning-based Navigation Based on Curriculum Learning

Abstract

In this paper, curriculum learning is studied as an approach to improve generalization ability in navigation tasks, that is, improve the agent’s ability of navigating in scenarios different from those used for training. The agent is trained based on the TD3 algorithm, and curriculum learning selects different stages (i.e. different curricula) of Empty/Sparse/Normal worlds for training. Via extensive numerical comparisons with agents trained under such curricula, it is shown that properly used curriculum learning improves the agent’s ability of generalization. Furthermore, an automatic curriculum learning (Auto-CL) approach is proposed. Auto-CL is shown to have even better generalization than the standard curriculum learning, since it makes the agent able to navigate in new environments with more than 6% shorter paths in more than 21% shorter time.