Learning Adaptive Grasping From Human Demonstrations

Abstract

This article studied a learning-based approach to learn grasping policies from teleoperated human demonstrations, which can achieve adaptive grasping using three different neural network (NN) structures. To transfer human grasping skills effectively, we used multisensing state within a sliding time window to learn the state–action mapping. By teleoperating an anthropomorphic robotic hand using human hand tracking, we collected training datasets from representative grasping of various objects, which were used to train grasping policies with three proposed NN structures. The learned policies can grasp objects with varying sizes, shapes, and stiffness. We benchmarked the grasping performance of all policies, and experimental validations showed significant advantages of using the sequential history states, compared to the instantaneous feedback. Based on the benchmark, we further validated the best NN structure to conduct extensive experiments of grasping hundreds of unseen objects with adaptive motions and grasping forces.