3-D Tactile-Based Object Recognition for Robot Hands Using Force-Sensitive and Bend Sensor Arrays

Abstract

Tactile sensing is a particularly important and challenging task for a modern robot to safely manipulate objects, interact with humans in a shared space, and provide various services. This paper presents a 3D tactile glove for robots with the combination of a piezoresistive-based force sensor array (412 sensors) covering the full hand and a resistive bend sensor array (5 sensors) on the back of five fingers. Deep learning-based CNN (Convolutional Neural Network) methods using the designed tactile glove are proposed for object recognition. In the experiment for recognizing 15 objects with a dexterous robot hand, an average classification accuracy of 93.67% has been achieved. Comparison experiments with three other typical classifiers (the Quadratic SVM, Weighted KNN, and Bagged Trees) and our CNN methods show an average recognition accuracy of 91.67% with the 3D tactile glove, revealing an accuracy improvement of 4.17% over only using the force sensor array. We further apply our 3D tactile glove and the multi-modal CNN to identify three other objects and demonstrate their generalization ability of tactile object recognition with an average success accuracy of 78.33%. The proposed 3D tactile glove can be further used in human-robot interactions, the design of prosthetics and humanoid robots, and for improving the intelligence level in brain-computer collaborative systems.