A data-efficient goal-directed deep reinforcement learning method for robot visuomotor skill

Abstract

Deep reinforcement learning (DRL) has provided an effective end to end approach for autonomous robot skill learning. However, the task goal in most DRL approaches is always single and fixed which restricts the flexibility of policy. In addition, the data inefficiency also makes it impractical to applied on real-world robots. To address these problems, this paper proposes a data-efficient goal-directed DRL method for robotic skill learning. First, an asymmetric deep deterministic policy gradients algorithm is constructed as the basic framework of the method by taking advantage of the low-dimensional physical state accessible in simulations. Then, a Siamese representation learning network is designed to embed the RGB observation and the human intention at the same feature space to realize human-robot intention transfer. And, an auxiliary similarity evaluation network is added to the DRL algorithm to accelerate representation learning. Finally, a domain randomization method is employed to transfer the learned policies from simulation to reality. In experiments, two typical robotic tasks are set up to evaluate the proposed method. The experimental results validate the effectiveness of the proposed method. The trained robot can switch among multiple goals automatically according to human intentions. And, the Siamese representation learning network and auxiliary similarity evaluation network can improve data efficiency effectively.