A Novel Evaluation Framework for Teleoperation and a Case Study on Natural Human-Arm-Imitation Through Motion Capture

Abstract

Although tele-operation has a long history, when it comes to tuning, comparison, and evaluation of tele-operation systems, no standard framework exists which can fulfill desiderata such as: concisely modeling multiple aspects of the system as a whole, i.e. timing, accuracy, and event transitions, while also providing for separation of user-, feedback-, as well as learning-dependent components. On the other hand, real-time remote tele-operation of robotic arms, either industrial or humanoid, is highly suitable for a number of applications, especially in difficult or inaccessible environment, and thus such an evaluation framework would be desirable. Usually, teleoperation is driven by buttons, joysticks, haptic controllers, or slave-arms, providing an interface which can be quite cumbersome and unnatural, especially when operating robots with multiple degrees of freedom. Thus, in thus paper, we present a two-fold contribution: (a) a task-based teleoperation evaluation framework which can achieve the desiderata described above, as well as (b) a system for teleoperation of an industrial arm commanded through human-arm motion capture, which is used as a case study, and also serves to illustrate the effectiveness of the evaluation framework that we are introducing. In our system the desired trajectory of a remote robotic arm is easily and naturally controlled through imitation of simple movements of the operator’s physical arm, obtained through motion capture. Furthermore, an extensive real-world evaluation is provided, based on our proposed probabilistic framework, which contains an inter-subject quantitative study with 23 subjects, a longitudinal study with 6 subjects, as well as opinions and attitudes towards tele-operation study. The results provided illustrate the strengths of the proposed evaluation framework—by enabling the quick production of multiple task-, user-, system-, as well as learning-centric results, as well as the benefits of our natural imitation-based approach towards teleoperation. Furthermore, an interesting ordering of preferences towards different potential application areas of teleoperation is indicated by our data. Finally, after illustrating their effectiveness, we discuss how both our evaluation framework as well as teleoperation system presented are not only applicable in a wide variety of teleoperation domains, but are also directly extensible in many beneficial ways.