Adaptive Reinforcement Learning in Box-Pushing Robots

Abstract

An adaptive state aggregation Q-Learning method, with the capability of multi-agent cooperation, is proposed to enhance the efficiency of reinforcement learning (RL) and is applied to box-pushing tasks for humanoid robots. First, a decision tree was applied to partition the state space according to temporary differences in reinforcement learning, so that a real valued action domain could be represented by a discrete space. Furthermore, adaptive state Q-Learning, which is the modification of estimating Q-value by tabular or function approximation, is proposed to demonstrate the efficiency of reinforcement learning in simulations of a humanoid robot pushing a box. The box moves in the direction in which the robot asserts force. To push the box to the target point, the robot needs to learn how to adjust angles, avoid obstacles, and keep balance. Simulation results show the proposed method outperforms Q-Learning without using adaptive states.