Robot Path Planning in Dynamic Environments Based on Deep Reinforcement Learning

Abstract

Path planning in dynamic environment has been the hot research direction. This paper considers a new dynamic environment—the obstacles are randomly distributed in the environment, and all of the obstacles will be distributed randomly again after robot’s movements. In the new dynamic environment, traditional path planning methods have some shortcomings when facing the dynamic environments. The traditional path planning algorithms need to re-calculate the path once the environments change, which is a very time consuming process. The deep reinforcement learning (DRL) model is a single-step algorithm, so the dynamic environments will not affect its running time consumption, which is superior to the traditional path planning algorithms in terms of running time consumption. However, the DRL model will face the problem of sparse rewards in the path planning problem due to the large state space of the environments. This paper uses DRL to solve the shortcomings of traditional path planning algorithms in dynamic environments and we propose a new framework to solve the problem of sparse reward in robot path planning. The framework uses a new strategy searching algorithm and a new shaped reward function. The improved framework can effectively solve the convergence problem in path planning. According to the simulation results, in the stochastic dynamic environments, the running time consumption of the new framework is less than the traditional path planning algorithm, and the new framework is better the classic DRL model in training results and planning results.