Generalized Visual-Tactile Transformer Network for Slip Detection

Abstract

Slip detection plays a vital role in robotic dexterous grasping and manipulation, and it has long been a challenging problem in the robotic community. Different from traditional tactile perception-based methods, we propose a Generalized Visual-Tactile Transformer (GVT-Transformer) network to detect slip based on visual and tactile spatiotemporal sequences. The main novelty of GVT-Transformer is its ability to address unaligned vision and tactile data in various formats captured by various tactile sensors. Furthermore, we train and test our proposed network on a public and our visual-tactile grasping datasets. The experimental results show that our method is more suitable for sliding detection tasks than previous visual-tactile learning methods and more versatile.