Collision Prediction Based Q-Learning for Mobile Robot Navigation in Unknown Dynamic Environments

Abstract

Q-learning (QL) approach is constantly used for mobile robot (MR) navigation in unknown dynamic environment because of its simplicity and well-developed theory. However, its salient downside is the curse of dimensionality problem, where it incurs a huge computational power and memory requirement. This problem is aggravated in complex environments. In this paper, a collision prediction based QL (CPQL) scheme is presented to MR navigation in a dynamic environment based on collision prediction between the robot and a group of static and dynamic obstacles. In the proposed scheme, a novel definition of environment states is presented to apply QL to unknown dynamic environments with compact state space, satisfactory robot turning angles, and adequate speed gradation. The key feature of the proposed CPQL scheme pertains to constructing a state—action pair based on two factors. The first factor is the region of predicting the position of collision between the robot and an obstacle, and the second is the region of the obstacle related to robot position. Simulation analysis and results show the superiority of CPQL in terms of learning convergence, obstacle avoidance, and smooth navigation path compared with state-of-the-art MR navigation schemes. Hence, CPQL proves its authenticity and suitability for real-time navigation in complex and dynamic environments.