Combining Vision and Tactile Data for Cable Grasping

Abstract

In this paper, the problem of properly combining vision and tactile data to locate a deformable linear object, such as a cable, and grasp it according to a required position and orientation of the cable is considered. Tactile sensors suitably developed for this task are adopted in the experiments together with a vision algorithm based on deep learning for the detection of the cable shape from a 2D camera image. The vision system is initially adopted to locate the cable in the scene and execute the grasp, then the tactile sensor is used to estimate the cable shape and position after grasping. The capability of the systems of performing cable regrasp by correcting the grasp pose thanks to the tactile data acquired during the first grasp is considered to deal with the cases in which the vision system can’t be used because of occlusions. Experimental trials show the capability of improving significantly the quality of the grasp thanks to tactile-based regrasping. Finally, the fusion between the shape estimation provided by the vision system and the one provided by the tactile sensor is also presented.