Three-Dimensional Force Sensor Based on Deep Learning

Abstract

Human skin can accurately sense subtle changes of both normal and shear forces. However, tactile sensors applied to robots are challenging in decoupling 3D forces due to the inability to develop adaptive models for complex soft materials. Therefore, a new soft tactile sensor has been designed in this paper to detect shear and normal forces, including a soft probe and image acquisition device. First, to capture the deformation of the sensor, colored silicone squares were embedded in the soft probe. Capcamera movement of the colored squares under external forces. The image dataset collected at different 3D forces is then input into a deep learning model. Finally, a custom miniature image device is acquired and embedded in the soft probe to miniaturize the sensor. Computing results obtained from experimental datasets show that the proposed method can accurately decouple the 3D forces. Robots can grap vulnerable objects with sensors prepared at the robot’s tip. The tactile sensors studied in this paper are expected to be applied in robotics fields such as adaptive grasping, dexterous manipulation and human-computer interaction.