A method for the reproduction of cello bow kinematics using a robotic arm and motion capture

Abstract

Numerous experimental and theoretical methods have focused on the bow–string interaction in bowed string instruments, including several artificial bowing setups. The current research aims to present an experimental approach to reproduce bowing techniques using a robotic arm. First, optical motion capture is used to track the 3D kinematics of the bow. The cello bow and corpus are equipped with reflective markers. The cello is mounted on a playing platform. The recorded 3D trajectories of the bow markers are used to control the motion of the robotic arm. This process requires converting the 3D data between the coordinate frames of the two systems. This conversion is described in detail in this paper. To demonstrate the performance of the proposed method, an experienced cellist was asked to play an adapted piece on the cello, which was then repeated using the robotic arm. The robotic arm is capable of accurately reproducing the bow velocity, but even minimal variations in position can compromise proper bow–string contact. To illustrate this, the study compares two similar robotic situations and discusses the challenges of adapting the robot’s coordinates as a function of a given playing parameter or the sound produced.