Online tangible robot programming: interactive automation method from teleoperation of manipulation task

Abstract

Teleoperated robots are introduced to hospitals, restaurants and factories and are expected to learn the task from human-teleoperated data. The automation methods from teleoperation, as known as Programming by demonstration (PbD) or imitation learning, are recently proposed increasingly, but they are focused on simple task, and training phase and execution phase should be separated. In this paper, we propose online and interactive automation method from teleoperation, online tangible robot programming, for object manipulation with multiple procedures and physical contacts. We focus on target-objects' states, split the task into motion sequences based on the states' change, and program the order of motion sequences with tangible user interface. We also implement the user interruption for mode switching, and a user can switch between autonomous and teleoperation modes and correct the programmed motion orders while continuing the task execution. For the evaluation, we did the real-world experiments with dual-arm robot and demonstrate the usefulness and efficiency of online tangible robot programming by switching from autonomous mode to teleoperation mode and immediately programming and modifing the task procedures of the box rotating and piling tasks with two arms.