High Resolution Tactile Sensor Research Articles

Seamless-walk: natural and comfortable virtual reality locomotion method with a high-resolution tactile sensor

Seamless-walk: natural and comfortable virtual reality locomotion method with a high-resolution tactile sensor

Dataset with Tactile and Kinesthetic Information from a Human Forearm and Its Application to Deep Learning.

There are physical Human-Robot Interaction (pHRI) applications where the robot has to grab the human body, such as rescue or assistive robotics. Being able to precisely estimate the grasping location when grabbing a human limb is crucial to perform a safe manipulation of the human. Computer vision methods provide pre-grasp information with strong constraints imposed by the field environments. Force-based compliant control, after grasping, limits the amount of applied strength. On the other hand, valuable tactile and proprioceptive information can be obtained from the pHRI gripper, which can be used to better know the features of the human and the contact state between the human and the robot. This paper presents a novel dataset of tactile and kinesthetic data obtained from a robot gripper that grabs a human forearm. The dataset is collected with a three-fingered gripper with two underactuated fingers and a fixed finger with a high-resolution tactile sensor. A palpation procedure is performed to record the shape of the forearm and to recognize the bones and muscles in different sections. Moreover, an application for the use of the database is included. In particular, a fusion approach is used to estimate the actual grasped forearm section using both kinesthetic and tactile information on a regression deep-learning neural network. First, tactile and kinesthetic data are trained separately with Long Short-Term Memory (LSTM) neural networks, considering the data are sequential. Then, the outputs are fed to a Fusion neural network to enhance the estimation. The experiments conducted show good results in training both sources separately, with superior performance when the fusion approach is considered.

Vision-Based High-Resolution Tactile Sensor By Using Visual Light Ring

In this letter, a vision-based high-resolution pressure array sensor with sensitivity enhanced by using visual light ring was proposed. The sensor consists of a transparent flexible layer integrating micropillar array and an image sensor with circular array light emitting diodes (LEDs). When pressure is applied to the micropillar, the surface of the substrate around the micropillar produces a concave, and the edges of the concave reflect the light of the LED array to form visual light ring in the image sensor. A relationship between the pressure and the size of the visual light ring was established. The sensitivity using the light ring was increased by three times compared with using deformation of the micropillar. The sensor achieves a high resolution of 625 sensing points per cm², which was higher than that on the finger of a human. Thus, the sensor will be a competitive solution to artificial tactile system which is of great importance for intelligent robots.

Cable manipulation with a tactile-reactive gripper

Cables are complex, high-dimensional, and dynamic objects. Standard approaches to manipulate them often rely on conservative strategies that involve long series of very slow and incremental deformations, or various mechanical fixtures such as clamps, pins, or rings. We are interested in manipulating freely moving cables, in real time, with a pair of robotic grippers, and with no added mechanical constraints. The main contribution of this paper is a perception and control framework that moves in that direction, and uses real-time tactile feedback to accomplish the task of following a dangling cable. The approach relies on a vision-based tactile sensor, GelSight, that estimates the pose of the cable in the grip, and the friction forces during cable sliding. We achieve the behavior by combining two tactile-based controllers: (1) cable grip controller, where a PD controller combined with a leaky integrator regulates the gripping force to maintain the frictional sliding forces close to a suitable value; and (2) cable pose controller, where an linear–quadratic regulator controller based on a learned linear model of the cable sliding dynamics keeps the cable centered and aligned on the fingertips to prevent the cable from falling from the grip. This behavior is possible with the use of reactive gripper fitted with GelSight-based high-resolution tactile sensors. The robot can follow 1 m of cable in random configurations within two to three hand regrasps, adapting to cables of different materials and thicknesses. We demonstrate a robot grasping a headphone cable, sliding the fingers to the jack connector, and inserting it. To the best of the authors’ knowledge, this is the first implementation of real-time cable following without the aid of mechanical fixtures. Videos are available at http://gelsight.csail.mit.edu/cable/

Bayesian and Neural Inference on LSTM-Based Object Recognition From Tactile and Kinesthetic Information

Recent advances in the field of intelligent robotic manipulation pursue providing robotic hands with touch sensitivity. Haptic perception encompasses the sensing modalities encountered in the sense of touch (e.g., tactile and kinesthetic sensations). This letter focuses on multimodal object recognition and proposes analytical and data-driven methodologies to fuse tactile- and kinesthetic-based classification results. The procedure is as follows: a three-finger actuated gripper with an integrated high-resolution tactile sensor performs squeeze-and-release Exploratory Procedures (EPs). The tactile images and kinesthetic information acquired using angular sensors on the finger joints constitute the time-series datasets of interest. Each temporal dataset is fed to a Long Short-term Memory (LSTM) Neural Network, which is trained to classify in-hand objects. The LSTMs provide an estimation of the posterior probability of each object given the corresponding measurements, which after fusion allows to estimate the object through Bayesian and Neural inference approaches. An experiment with 36-classes is carried out to evaluate and compare the performance of the fused, tactile, and kinesthetic perception systems. The results show that the Bayesian-based classifiers improves capabilities for object recognition and outperforms the Neural-based approach.

DIGIT: A Novel Design for a Low-Cost Compact High-Resolution Tactile Sensor With Application to In-Hand Manipulation

Despite decades of research, general purpose in-hand manipulation remains one of the unsolved challenges of robotics. One of the contributing factors that limit current robotic manipulation systems is the difficulty of precisely sensing contact forces -- sensing and reasoning about contact forces are crucial to accurately control interactions with the environment. As a step towards enabling better robotic manipulation, we introduce DIGIT, an inexpensive, compact, and high-resolution tactile sensor geared towards in-hand manipulation. DIGIT improves upon past vision-based tactile sensors by miniaturizing the form factor to be mountable on multi-fingered hands, and by providing several design improvements that result in an easier, more repeatable manufacturing process, and enhanced reliability. We demonstrate the capabilities of the DIGIT sensor by training deep neural network model-based controllers to manipulate glass marbles in-hand with a multi-finger robotic hand. To provide the robotic community access to reliable and low-cost tactile sensors, we open-source the DIGIT design at https://digit.ml/.

(Invited) Post-CMOS Compatible Silicon MEMS Nano-Tactile Sensor for Touch Feeling Discrimination of Materials

Based on silicon post-CMOS compatible process, we have realized high performance silicon MEMS tactile sensor called “Nano-tactile sensor”, which is comparable to the performance of fingertip sense of touch. In this tactile sensor device, all the mechanical structures are made from single crystalline silicon which is the active layer of SOI wafers. No elastomer/polymer structures are used in the mechanical sensing structure, and softness and elasticity necessary to tactile sensing are realized by silicon micro mechanical structures. In the latest version of our tactile sensors, six contactors with fingerprint-like shape are integrated at a pitch of 500µm to get high spatial resolution tactile images. Each fingerprint-like contactor reproduces vertical motion (by micro roughness) and horizontal motion (by frictional force) of a fingerprint closely under sweeping motion of fingertip in measurement. The performances are enough high to analyze surface touch feelings of “Hair surface condition”, “Skin condition”, and “Touch feeling of various papers and clothes” like human fingertip. Combination of high-resolution tactile sensor and deep neural network is a strong approach to reproduce human fingertip sensation by state-of-the-art device electron device technology.

Using 3D Convolutional Neural Networks for Tactile Object Recognition with Robotic Palpation.

In this paper, a novel method of active tactile perception based on 3D neural networks and a high-resolution tactile sensor installed on a robot gripper is presented. A haptic exploratory procedure based on robotic palpation is performed to get pressure images at different grasping forces that provide information not only about the external shape of the object, but also about its internal features. The gripper consists of two underactuated fingers with a tactile sensor array in the thumb. A new representation of tactile information as 3D tactile tensors is described. During a squeeze-and-release process, the pressure images read from the tactile sensor are concatenated forming a tensor that contains information about the variation of pressure matrices along with the grasping forces. These tensors are used to feed a 3D Convolutional Neural Network (3D CNN) called 3D TactNet, which is able to classify the grasped object through active interaction. Results show that 3D CNN performs better, and provide better recognition rates with a lower number of training data.

CNN-Based Methods for Object Recognition With High-Resolution Tactile Sensors

Novel high-resolution pressure-sensor arrays allow treating pressure readings as standard images. Computer vision algorithms and methods such as convolutional neural networks (CNN) can be used to identify contact objects. In this paper, a high-resolution tactile sensor has been attached to a robotic end-effector to identify contacted objects. Two CNN-based approaches have been employed to classify pressure images. These methods include a transfer learning approach using a pre-trained CNN on an RGB-images dataset and a custom-made CNN (TactNet) trained from scratch with tactile information. The transfer learning approach can be carried out by retraining the classification layers of the network or replacing these layers with an SVM. Overall, 11 configurations based on these methods have been tested: eight transfer learning-based, and three TactNet-based. Moreover, a study of the performance of the methods and a comparative discussion with the current state-of-the-art on tactile object recognition is presented.

More Than a Feeling: Learning to Grasp and Regrasp Using Vision and Touch

For humans, the process of grasping an object relies heavily on rich tactile feedback. Most recent robotic grasping work, however, has been based only on visual input, and thus cannot easily benefit from feedback after initiating contact. In this paper, we investigate how a robot can learn to use tactile information to iteratively and efficiently adjust its grasp. To this end, we propose an end-to-end action-conditional model that learns regrasping policies from raw visuo-tactile data. This model -- a deep, multimodal convolutional network -- predicts the outcome of a candidate grasp adjustment, and then executes a grasp by iteratively selecting the most promising actions. Our approach requires neither calibration of the tactile sensors, nor any analytical modeling of contact forces, thus reducing the engineering effort required to obtain efficient grasping policies. We train our model with data from about 6,450 grasping trials on a two-finger gripper equipped with GelSight high-resolution tactile sensors on each finger. Across extensive experiments, our approach outperforms a variety of baselines at (i) estimating grasp adjustment outcomes, (ii) selecting efficient grasp adjustments for quick grasping, and (iii) reducing the amount of force applied at the fingers, while maintaining competitive performance. Finally, we study the choices made by our model and show that it has successfully acquired useful and interpretable grasping behaviors.

A low-cost, human-like, high-resolution, tactile sensor based on optical fibers and an image sensor

Tactile sensors are commonly a coordinated group of receptors forming a matrix array meant to measure force or pressure similar to the human skin. Optic-based tactile sensors are flexible, sensitive, and fast; however, the human fingertip’s spatial resolution, which can be regarded as the desired spatial resolution, still could not be reached because of their bulky nature. This article proposes a novel and patented optic-based tactile sensor design, in which fiber optic cables are used to increase the number of sensory receptors per square centimeter. The proposed human-like high-resolution tactile sensor design is based on simple optics and image processing techniques, and it enables high spatial resolution and easy data acquisition at low cost. This design proposes using the change in the intesity of the light occured due to the deformation on contact/measurement surface. The main idea is using fiber optic cables as the afferents of the human physiology which can have 9 µm diameters for both delivering and receiving light beams. The variation of the light intensity enters sequent mathematical models as the input, then, the displacement, the force, and the pressure data are evaluated as the outputs. A prototype tactile sensor is manufactured with 1-mm spatial and 0.61-kPa pressure measurement resolution with 0–15.6 N/cm2 at 30 Hz sampling frequency. Experimental studies with different scenarios are conducted to demonstrate how this state-of-the-art design worked and to evaluate its performance. The overall accuracy of the first prototype, based on different scenarios, is calculated as 93%. This performance is regarded as promising for further developments and applications such as grasp control or haptics.

The undamaged tissue grasping in a laparoscopic surgical grasper via distributed pressure measurement

The grasping action in a minimally invasive surgery operation is a blind operation in terms of the touch feedback and may result an undesirable tissue damage during the surgery. In this study, the interaction of the minimally invasive surgery tool with the tissue investigated by integrating a high resolution tactile sensor to the end-effector of the tool. The pressure distribution during the closing action of the grasper is characterized using the non-organic viscoelastic materials as well as organic tissues such as dead chicken liver and meat. Pressure mapping is used to calculate the net grasping force acting on the contact interface. The results show the total grasping force changes in the range of 0.2–0.7N for different types of tissues.

Design and Simulation of Piezoelectric-Charge-Gated Thin-Film Transistor for Tactile Sensing

This letter reports on an electronic device that can work as a tactile force sensor with a high spatial resolution and sensitivity. It integrates a thin-film piezoelectric sensor with a thin-film transistor (TFT) to form a dual-gate force-sensitive TFT. The piezoelectric sensor acts as a top-sensing terminal, which is charge-gated. An analytical model of the device is given to elaborate how the output current varies with the applied force, and a numerical study is conducted to evaluate its sensitivity, noise, and dynamic range. The integrated sensor can be potentially pixelated to form a high-resolution tactile sensor array, promising for electronic skin applications.

Stress distribution and contact area measurements of a gecko toe using a high-resolution tactile sensor

The adhesive systems of geckos have been widely studied and have been a great source of bioinspiration. Load-sharing (i.e. preventing stress concentrations through equal distribution of loads) is necessary to maximize the performance of an adhesive system, but it is not known to what extent load-sharing occurs in gecko toes. In this paper, we present in vivo measurements of the stress distribution and contact area on the toes of a tokay gecko (Gekko gecko) using a custom tactile sensor with 100 μm spatial resolution. We found that the stress distributions were nonuniform, with large variations in stress between and within lamellae, suggesting that load-sharing in the tokay gecko is uneven. These results may be relevant to the understanding of gecko morphology and the design of improved synthetic adhesive systems.

Active Touch Sensing by Multi-axial Force Measurement Using High-Resolution Tactile Sensor with Microcantilevers

Active Touch Sensing by Multi-axial Force Measurement Using High-Resolution Tactile Sensor with Microcantilevers

Application of modern high resolution tactile sensors for micro-objects

Following the miniaturisation of electrical, mechanical and optical components, the demands on metrological quality control are increasing. Several measurement principles are available for micro- and nanometrology, yet, are usually limited to 2D or 2.5D measurements. But recent micro-objects used for automotive and medical applications hold 3D features, raising the need for full 3D capability. Today, only micro tactile probing systems can meet this requirement with the demanded precision. Due to physical limitations, a simple miniaturisation of available tactile probes is not possible. Several side effects of macro scale tactile probing disproportionately rise and new effects gain influence. To adapt tactile probing to nanometrology, new designs and working principles are necessary to achieve nanometer resolution. Even principles with negligible or even no probing force were achieved. Promising designs of different institutes are shown as also own work in this field.

Edge identification of a small object through a low-resolution tactile sensor array

Edge detection is always an interesting field for researchers in the area of artificial intelligence, including machine visions and robotics. In the context of tactile sensors, many studies have been conducted on edge-identification algorithms. These investigations often use high-resolution tactile sensor arrays to track an edge of a large object. In this work, edge identification is examined in which a coarse-resolution tactile sensor, specif., a 2×2 array, is used to detect and track edges of a small object. To simplify the control, we control the sensor just in 2D and in just one direction at a time. We propose two methods to solve this problem: a scanning method, of which the principle of operation is similar to that of a scanner, and an edge-following method that can make the sensor follow the boundary of the object. The flowchart of each method will be presented in detail in this paper. Some simulations in MATLAB, including four shapes, viz., square, triangle, ellipse, and hexagon, have been executed to verify our proposal. A comparison and a critique of the two methods are also presented for improving the methods.

Grip Norms And Reliability Of The Hand Grip Forcemap System

Strength testing of the injured hand and forearm is of considerable interest to patients, treating physicians, therapists, and insurance companies. While, over the past many years, traditional parallel-bar dynamometers have made a significant and positive contribution to medical grip strength evaluation, these devices are now less than optimal for the comprehensive analysis of overall grip and finger function. The ForceMap grip force mapping system utilizes tactile array technology to assess the complex grip capability/capacity of a human hand. PURPOSE: This study‘s aim was to evaluate the reliability of the hand grip ForceMap system and to establish max grip force (MGF) and force distribution norms (D1-thumb; D2-index; D3-middle; D4-ring; D5-little finger; N/cm2) for healthy adults. METHODS: Adults (Female n=28; 38.9 ±11.0 yrs; 165.3± 6.7 cm; 69.8± 13.0 kg; Male n=28; 34.2 ±13.0 yrs; 181.3± 8.0 cm; 86.4± 15.1 kg) completed trials of maximal grip on the ForceMap system with the elbow at 90 degrees flexion. The ForceMap system is comprised of a high resolution tactile array sensor mounted to a 5 × 15 cm solid Delrin cylinder (grip map transducer) that is interfaced to an amplifier/processor and laptop computer. The ForceMap system provides visual and numerical assessment of grip force including the palm and individual digits both statically and dynamically (100 hz). The reliability of the ForceMap was determined using a Pearson Correlation Coefficient (r), Intra-class Correlation Coefficient (ICC) and a Bland-Altman Plot (BAP) of repeated trials of the male dominant hand MGF. RESULTS: Female norms dominant hand: MGF= 471.2±119.6 (N); D1=29.0 ±9.4; D2=23.6 ±8.3; D3= 25.0±5.6; D4= 20.8±6.3; D5=14.9 ±6.6 (N/cm2). Male norms dominant hand: MGF= 820.9± 210.5 (N); D1= 34.5± 9.1; D2= 29.5 ± 8.3; D3= 34.3 ± 8.8; D4= 26.9 ± 9.0; D5= 23.2 ± 9.2 (N/cm2). Male dominant hand MGF: r=0.92 and ICC=0.92. The BAP plot revealed that 100% of the differences between trails were within ±2 standard deviations of the mean difference. CONCLUSION: Given the ability of the ForceMap system to reliably assess grip force along with the establishment of grip force and force distribution norms, it appears the ForceMap system is a viable means for detailed grip assessment.

High sampling resolution tactile sensor for object recognition

High sampling resolution tactile sensor for object recognition

Profile detection of objects by a high-resolution tactile sensor using a light conductive plate.

A high resolution tactile sensor of pressure to-light conversion type is developed. An object is pressed on a white elastic sheet to make close contact with a transparent plate underneath. Then the side of the plate is irradiated. The sheet contact areas on the plate scatter the light which provides a tactile image of the object profile. This is based on the mechanism that the contact material at the surface of the transparent plate is changed from the air to an elastic sheet, and then the condition of total reflection does not stand any more. A CCD-camera is used to detect the images and microcomputer is used to convert to binary images with a fixed threshold. In order to get an optimal sensor system, how to design the parameters of each sensor assembly is discussed. Through the evaluation experiment for the sensor system, it is found that spatial resolution of measuring object profiles is about 2 mm and pressure sensitivity is about 2×104 Pa.