Improving Teleoperation Through Human-Aware Haptic Feedback: A Distinguishable and Interpretable Physical Interaction Based on the Contact State

Abstract

Teleoperation with haptic feedback is especially useful for contact-rich manipulation. Humans can easily perceive physical interaction events and effects of hands-on manipulation. However, current measurement-based haptic rendering methods with distorting and confusing transmission limit teleoperation performance. The major problem is a mismatch between low-level modulated sensor signals and human-aware haptic stimuli. We explore a human-aware haptic feedback pipeline that renders a distinguishable and interpretable physical interaction based on the contact states. Manipulation tasks are modeled as time-series sequences of four contact states: 1) <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">noncontact</i> , 2) <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">contact</i> , 3) <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">stick</i> , and 4) <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">slip</i> . The temporal convolutional network model fuses force/torque sensor and velocity signals in real time to identify the four contact states with a 91.3% accuracy. Meanwhile, state-dependent haptic feedback, which combines transient and continuous feedback, brings more cues for physical interaction events and effects, corresponding to human fast- and slow-adapting receptors. We formulate a two-peak waveform for transient feedback based on a second-order superimposed exponentially decaying sinusoid model and adopt the orthogonal decomposition filter method for “inequable” continuous feedback. We demonstrate the effectiveness of our method through contact state and teleoperation experiments under different haptic conditions. The results indicate that the proposed method helps the operator to perceive and understand physical interaction and significantly improves teleoperation performance.