A digital twin-based sim-to-real transfer for deep reinforcement learning-enabled industrial robot grasping

Abstract

• A digital twin-enabled approach for achieving effective transfer of DRL algorithms to a physical robot is proposed. • A digital twin system of the physical robotic system is established, which is used to correct the real grasping point. • Experimental results verify the effectiveness of the intelligent grasping algorithm and the digital twin-enabled sim-to-real transfer approach and mechanism. Deep reinforcement learning (DRL) has proven to be an effective framework for solving various complex control problems. In manufacturing, industrial robots can be trained to learn dexterous manipulation skills from raw pixels with DRL. However, training robots in the real world is a time-consuming, high-cost and of safety concerns process. A frequently adopted approach for easing this is to train robots through simulations first and then deploy algorithms (or policies) on physical robots. How to transfer policies of robot learning from simulation to the real world is a challenging issue. Digital twin that is able to create a dynamic, up-to-date representation of a physical robotic grasping system provides an effective approach for addressing this issue. In this paper, we focus on the scenario of DRL-based assembly-oriented industrial grasping and propose a digital twin-enabled approach for achieving effective transfer of DRL algorithms to a physical robot. Two parallel training systems, i.e., the physical robotic system and corresponding digital twin system, respectively, are established, which take virtual and real images as inputs. The output of the digital twin system is used to correct the real grasping point so that accurate grasping can be achieved. Experimental results verify the effectiveness of the intelligent grasping algorithm and the digital twin-enabled sim-to-real transfer approach and mechanism.