A digital twin system for Task-Replanning and Human-Robot control of robot manipulation

Abstract

In order to enhance the robustness of robots in scenarios with dynamic and complex manipulation tasks and respond quickly to the demands of personalized manufacturing, we propose a DT prototype system named “Alita” to achieve effectively task replanning and human-robot control, inspired by the real-time and closed-loop characteristics of DT. Alita constructs a DT representation with four layers, encoding the geometric, physical and visual dynamics of the work scene, and further obtains unified semantic expressions. Based on this DT representation, Alita establishes two accessible strategies, forming a two-way information feedback loop that endows it with the capability to optimize real robot manipulation. The first strategy presents a deep learning-based model that combines a graph network and a long short-term memory network, and introduces a specialized dataset to replan the manipulation task. The second strategy adopts a multi-virtual force constrained hybrid mapping method of joints and poses to achieve human-robot control. For performance evaluation, we study two cases involving multiple manipulation task replanning and human-robot collaborative grasping. Empirical results demonstrate that Alita accomplishes effectively the tasks of replanning and human-robot control, and mitigates the interferences of environment and task mutations.