Deep Reinforcement Learning for the Improvement of Robot Manipulation Skills Under Sparse Reward

Abstract

Under sparse reward, the reward information is very sparse during the training process, which is difficult for the robot to obtain rewards and learn successful manipulation skills. Therefore, we utilize deep reinforcement learning to improve robot manipulation skills. In this work, we propose a fusion algorithm based on the Twin Delayed Deep Deterministic Policy Gradient algorithm (TD3) and Hindsight Experience Replay (HER). Taking the advantages of end action compression and expert data, we utilize them to optimize the performance of TD3+HER. A Fetch robot simulation environment is built in the Pybullet module to verify the fusion algorithm in the push and pick-and-place tasks. Experimental results show that the fusion algorithm reduces 30% learning time compared with TD3+HER in the push task. Moreover, the fusion algorithm makes a breakthrough in the pick-and-place task, which improves the success rate more than 70%. These results confirm the proposed algorithm can effectively solve the sparse reward problem.