Remote Robot Control with Human-in-the-Loop over Long Distances Using Digital Twins

Abstract

The sharing of skills over the Internet enables professionals to democratize their expertise and skills without exhausting their availability, e.g., through excessive traveling. To enable this Internet of Skills, we present a novel Digital Twin (DT) platform for the remote control of machines with human-in-the-loop. The DT of a remotely controlled machine acts effectively as an inter-layer between the operator and the controlled machine, e.g., robot arm. The DT can be optimized for a particular application to interact with the operator with an intuitive low-latency interface and, on other side, to control and monitor the quality of the remote task. Essentially, the human operator controls the DT, while the DT controls the remote robot. This paper introduces the DT framework for the remote control. The human-machine-human control loop is split into Virtual Reality (VR), remote control, and robot control loops. The proposed framework achieves low latency visual feedback and very short system reaction times for unexpected changes with arbitrary distances between operator and robot. Within the DT framework, this paper proposes a robot control algorithm for controlling time-critical robot applications over networks with considerable delays and jitter. The proposed framework has been implemented in a demonstrator with a robot arm and its DT in VR.