Autonomous navigation of mobile robots in unknown environments using off-policy reinforcement learning with curriculum learning

Abstract

Reinforcement learning (RL) is effective for autonomous navigation tasks without prior knowledge of the environment. However, traditional mobile robot navigation algorithms, based on off-policy RL, often face challenges such as low sample efficiency during training and lack of adequate safety mechanisms. In this paper, we present an off-policy RL navigation model named Soft Actor-Critic with Curriculum Prioritization and Fuzzy Logic (SCF). The model uses energy as a prioritized evaluation metric for experience replay. And through task-level curriculum, the agent’s learning sequence is formulated, thereby enhancing sampling efficiency and safety. We propose a Curriculum-based Energy Prioritization (CEP) approach. It selects a replay trajectory that matches the current agent’s capability based on trajectory energy. Our results show that robots using off-policy RL often have limitations in dynamic obstacle avoidance. To rectify this, our model uses a fuzzy logic controller to enhance real-time obstacle avoidance. The SCF approach enables mobile robots to navigate adeptly in unpredictable and dynamic environments, ensuring optimal planning control while being safe and robust. Experiments in Gazebo simulation environment and real world confirm the effectiveness of our proposed method. The comparison results show the superior performance of this method, especially in unknown and dynamic environments.