Implementing Q-learning on Simple Vehicle: Learning Obstacle Avoidance in Different Environments

Abstract

Q-learning has been applied to many research projects in the past few decades. Its simple logic of mapping a corresponding action to each possible state has made it one of the most popular methods in reinforcement learning. In this implementation of Q-learning, a simple controlling scheme was used. The percept of the robot is taken as the state, and the actions give instruction to the actuating device of the robot. The goal is for this robot to pick up the obstacle avoidance skill after series of training. A sanity check for such an idea was provided in this essay. The basic disc robot from Robotics Playground was used, and the basic form of Q-learning was utilized. Our vehicle’s obstacle avoidance ability was assessed in environments with different levels of complexity using the Collision-Time graphs produced. The results showed very limited improvements in terms of reducing the number of collisions in all of the environments, and some fluctuations in the number of collisions were shown in the diagram. Combining the fluctuations with observations during the experiment process, we came to a conclusion that using Q-learning to navigate an arbitrary environment, the state space, and action space could be too large for any efficient training to be done.