Robot Skin Research Articles

Pressure distribution classification and segmentation of human hands in contact with the robot body

This article deals with the problem of the recognition of human hand touch by a robot equipped with large area tactile sensors covering its body. This problem is relevant in the domain of physical human–robot interaction for discriminating between human and non-human contacts and to trigger and to drive cooperative tasks or robot motions, or to ensure a safe interaction. The underlying assumption used in this article is that voluntary physical interaction tasks involve hand touch over the robot body, and therefore the capability to recognize hand contacts is a key element to discriminate a purposive human touch from other types of interaction. The proposed approach is based on a geometric transformation of the tactile data, formed by pressure measurements associated to a non-uniform cloud of 3D points (taxels) spread over a non-linear manifold corresponding to the robot body, into tactile images representing the contact pressure distribution in two dimensions. Tactile images can be processed using deep learning algorithms to recognize human hands and to compute the pressure distribution applied by the various hand segments: palm and single fingers. Experimental results, performed on a real robot covered with robot skin, show the effectiveness of the proposed methodology. Moreover, to evaluate its robustness, various types of failures have been simulated. A further analysis concerning the transferability of the system has been performed, considering contacts occurring on a different sensorized robot part.

Artificial Sensitive Skin for Robotics Based on Electrical Impedance Tomography

Electrical impedance tomography (EIT) is a noninvasive measurement technique that estimates the internal resistivity distribution based on the boundary voltage–current data measured from the surface of the conductor. If a thin stretchable soft material with a certain piezoresistive property is used as the electrical conductor, EIT has the ability to reconstruct the position where the resistivity changes due to the inside pressure contact, so that a large‐scale artificial sensitive skin is provided for robotics. First, the different conduction principles and material types of artificial sensitive skins are discussed, which is next followed by the different driving modes and image reconstruction techniques. Then, details on how EIT is used for robotic skin applications are described. Finally, the development trends and future potentials of EIT‐based robotic skins are expounded.

A Flexible Microwave Sensor Based on Complementary Spiral Resonator for Material Dielectric Characterization

In this paper, a flexible microwave sensor for material dielectric characterization is presented. The sensor is composed of a two-turn complementary spiral resonator (CSR) in coplanar waveguide (CPW) technology developed on a flexible polyethylene terephthalate (PET) substrate. The sensing principle is based on the measured transmission coefficient of the designed structure, where the resonant frequency and its magnitude are in function of the dielectric properties of loaded materials under test (MUTs). A prototype of the proposed sensor is fabricated and experimentally validated by extracting the dielectric constant and loss tangent of the reference samples loaded above the region of the CSR. Specially, due to the flexibility, the fabricated sensor can be conformally attached to the palm of a dexterous robotic hand to function as robotic electronic skin for material dielectric characterization, more specifically, to obtain dielectric properties of the objects grasped by the dexterous robotic hand. Moreover, the sensing functionality of the sensor structure under different bending levels is validated by calculating the dielectric property of loaded objects with non-planar surfaces.

Carbon nanotube-integrated conductive hydrogels as multifunctional robotic skin

Silicone elastomers with high mechanical stability have been conventionally adopted for the fabrication of emerging soft robots. However, these elastomers exhibit limited electrical and thermal properties that restrict the development of functional soft robots. A promising approach is to integrate carbon nanomaterials into hydrogels to develop multifunctional robotic skin with high deformability and diverse built-in functions for fabricated soft machines. Herein, a scalable approach was developed to fabricate a conductive hydrogel by integrating pristine multi-walled carbon nanotubes (MWNTs) into gelatin solution followed by the introduction of a crosslinking agent (i.e., glutaraldehyde). After the addition of glutaraldehyde, the viscosity of MWNT-gelatin dispersion increased with time, and a viscous precursor paste for conductive hydrogels was achieved for various scalable coating techniques including doctor blading. After large-area printing, the MWNT-gelatin paste continued to crosslink, and an MWNT-integrated gelatin hydrogel (MW-hydrogel) was obtained. The MW-hydrogels were highly deformable, and the electrical resistance of conductive MW-hydrogels was responsive to various mechanical deformations, enabling their applications in electronic robotic skin to monitor the actuations of soft robots in real time. Also, the MW-hydrogels were further utilized as flame-retardant skin for a soft robotic gripper, which could manipulate and rescue irregularly shaped objects from a fire scene.

Stretchable Energy‐Harvesting Tactile Interactive Interface with Liquid‐Metal‐Nanoparticle‐Based Electrodes

AbstractEnergy‐harvesting electronic skin (E‐skin) is highly promising for sustainable and self‐powered interactive systems, wearable human health monitors, and intelligent robotics. Flexible/stretchable electrodes and robust energy‐harvesting components are critical in constructing soft, wearable, and energy‐autonomous E‐skin systems. A stretchable energy‐harvesting tactile interactive interface is demonstrated using liquid metal nanoparticles (LM‐NPs)‐based electrodes. This stretchable energy‐harvesting tactile interface relies on triboelectric nanogenerator composed of a galinstan LM‐NP‐based stretchable electrode and patterned elastic polymer friction and encapsulation layer. It provides stable and high open‐circuit voltage (268 V), short‐circuit current (12.06 µA), and transferred charges (103.59 nC), which are sufficient to drive commercial portable electronics. As a self‐powered tactile sensor, it presents satisfactory and repeatable sensitivity of 2.52 V·kPa−1 and is capable of working as a touch interactive keyboard. The demonstrated stretchable and robust energy‐harvesting E‐skin using LM‐NP‐based electrodes is of great significance in sustainable human–machine interactive system, intelligent robotic skin, security tactile switches, etc.

Plantar Tactile Feedback for Biped Balance and Locomotion on Unknown Terrain

This work introduces a new sensing system for biped robots based on plantar robot skin, which provides not only the resultant forces applied on the ankles but a precise shape of the pressure distribution in the sole together with other extra sensing modalities (temperature, pre-touch and acceleration). The information provided by the plantar robot skin can be used to compute the center of pressure and the ground reaction forces. This information also enables the online construction of the supporting polygon and its preemptive shape before foot landing using the proximity sensors in the robot skin. Two experiments were designed to show the advantages of this new sensing technology for improving balance and walking controllers for biped robots over unknown terrain.

Anodized Aluminum Oxide-Assisted Low-Cost Flexible Capacitive Pressure Sensors Based on Double-Sided Nanopillars by a Facile Fabrication Method.

Flexible pressure sensors have garnered enormous attention in recent years as they hold great promise in wearable electronic devices. However, the realization of a high-performance flexible pressure sensor via a facile and cost-effective approach still remains a challenge. In this work, a capacitive pressure sensor based on a poly(vinylidenefluoride-co-trifluoroethylene) [P(VDF-TrFE)] dielectric film that incorporates nanopillars into both sides is demonstrated. Unlike the previous complicated and expensive methods, large-scale regular and uniform nanopillars are easily and economically achieved by the pattern transfer of anodized aluminum oxide templates. The double-sided nanopillars constituting the P(VDF-TrFE) dielectric layer enable the pressure sensor with high sensitivity (∼0.35 kPa-1), wide working range (4 Pa to 25 kPa), short response time (∼48 ms), and excellent durability. In addition to these salient features, our sensor also exhibits superior performances under bending states, ensuring that it can be used for detecting diverse practical stimuli as experimentally validated by perceiving real-time and in-site human physiological signals and body motions that, respectively, correspond to the low- and high-pressure range. A sensor array is finally constructed and is shown to be capable of perceiving the spatial pressure distribution of either a contact or noncontact object. These demonstrations show a promising future of our sensor in healthcare monitoring, smart robot skin, and human-machine interfaces.

Flexible Piezoresistive Sensor with the Microarray Structure Based on Self-Assembly of Multi-Walled Carbon Nanotubes.

High-performance flexible pressure sensors have great application prospects in numerous fields, including the robot skin, intelligent prosthetic hands and wearable devices. In the present study, a novel type of flexible piezoresistive sensor is presented. The proposed sensor has remarkable superiorities, including high sensitivity, high repeatability, a simple manufacturing procedure and low initial cost. In this sensor, multi-walled carbon nanotubes were assembled onto a polydimethylsiloxane film with a pyramidal microarray structure through a layer-by-layer self-assembly system. It was found that when the applied external pressure deformed the pyramid microarray structure on the surface of the polydimethylsiloxane film, the resistance of the sensor varied linearly as the pressure changed. Tests that were performed on sensor samples with different self-assembled layers showed that the pressure sensitivity of the sensor could reach , which ensured the high dynamic response ability and the high stability of the sensor. Moreover, it was proven that the sensor could be applied as a strain sensor under the tensile force to reflect the stretching extent or the bending object. Finally, a flexible pressure sensor was installed on five fingers and the back of the middle finger of a glove. The obtained results from grabbing different weights and different shapes of objects showed that the flexible pressure sensor not only reflected the change in the finger tactility during the grasping process, but also reflected the bending degree of fingers, which had a significant practical prospect.

Tactile Robotic Skin with Pressure Direction Detection.

Tactile sensing is the current challenge in robotics and object manipulation by machines. The robot’s agile interaction with the environment requires pressure sensors to detect not only location and value, but also touch direction. The paper presents a new, two-layer construction of artificial robotic skin, which allows measuring the location, value, and direction of pressure from external force. The main advantages of the proposed solution are its low cost of implementation based on two FSR (Force Sensitive Resistor) matrices and real-time operation thanks to direction detection using fast matching algorithms. The main contribution is the idea of detecting the pressure direction by determining the shift between the pressure maps of the skin’s upper and lower layers. The pressure map of each layer is treated as an image and registered using a phase correlation (POC–Phase Only Correlation) method. The use of the developed device can be very wide. For example, in the field of cooperative robots, it can lead to the improvement of human machine interfaces and increased security of human–machine cooperation. The proposed construction can help meet the increasing requirements for robots in cooperation with humans, but also enable agile manipulation of objects from their surroundings.

Evaluation of a Large Scale Event Driven Robot Skin

This letter evaluates and describes the large-scale integration of our multi-modal event-driven robot skin system on our humanoid robot H1 (REEM-C, PAL robotics). The robot skin is powered by the robot and all processing of tactile perception and control are executed onboard the robot. The robot skin system largely covers the humanoid and employs 1260 skin cells in 47 skin patches, in total 7560 multi-modal tactile sensors. The robot skin system is driven by events, i.e., the occurrence of novel information drives the acquisition, transmission, and processing of information rather than the synchronous sampling clock as in clock-driven systems. The event-driven robot skin system enables efficient multi-modal large area tactile perception, which is a prerequisite for realizing reactive whole-body control. We analyze the new robot skin system and evaluate its performance in clock-driven mode and event-driven mode. This analysis includes modeling the required CPU load and investigating the scaling of the system in multi-core systems. We evaluate the robot skin system with an experiment, where a large number of skin cells is activated (>680 cells), and a large number of events is generated. The obtained results demonstrate the superior performance of the event-driven system. In the clock-driven mode, the robot skin system constantly produces 315 000 packets/s, while in event-driven mode the system at most produces 40 000 packets/s (13%). The CPU load reduces from constantly 270% to at most 100% (37%). In clock-driven mode, the PC drops on average 80 000 packets/s (25% of all packets), while in event-driven mode, the package loss is practically neglectable. This efficient large-scale robot skin enables the complete onboard integration into a humanoid robot without the need for additional external power or processing capabilities.

Pressure-Driven Body Compliance Using Robot Skin

Skin can provide rich multi-modal contact information about the interaction forces of a robot with its environment. With this new way of sensing, a new generation of compliant controllers can be developed to enable different kinds of interactions. In this letter, a pressure-driven compliance controller is presented to generate virtual compliant forces depending on both external contact forces and their area of contact. The proposed controller can be adapted for any skin technology that provides a spatial distribution of force sensors over the surface of a robot and the geometry of the contact areas. This modality of body compliance is formulated as a task function, which can be inserted in strict or non-strict hierarchical task execution policies. The controller was tested on the upper body of a humanoid robot (16 DoF) covered with robot skin.

A Comprehensive Realization of Robot Skin: Sensors, Sensing, Control, and Applications

This article presents a holistic approach to the engineering of an artificial robot skin for robots. An example of a multimodal skin cell is given, one that supports multiple human-like sensing modalities, and support for skin cell network is also provided; this is essential to form large-area skin patches in order to cover the surfaces of robots. The essential elements of efficiently handling a large amount of tactile data are explained. A general control framework, which supports robots commanded in position, velocity, and torque, is provided and validated. Several applications of this robot skin will be presented, demonstrating the effectiveness and efficiency of our artificial robot skin to support a wide number of robotic platforms as well as its ease of use across different domains.

FLEXIBLE ROBOTIC ELECTRONIC SKIN WITH HIGH SENSITIVITY SENSOR ARRAYS

In this paper, we propose a flexible robotic electronic skin (e-skin) with high sensitivity sensor arrays. The sensor arrays are designed and fabricated on thin flexible silicone film. It uses piezo-resistive material Samarian Monosulphide and graphite rods for interconnection between the layers. Along with the pressure sensing capability of SmS, temperature and humidity sensors are also integrated in the silicone layers along with photovoltaic thin layer cells that provide energy independence to the module. The data procured from the e-skin is transferred to the analog to digital converter unit and further, to the PC through USB interface for analysis.

Robot Skin: Fully-Compliant Control Framework Using Multi-modal Tactile Events

In this paper, we present a multi-modal control framework to provide compliant behaviors on industrial robots, even when the robots are position commanded. This is obtained by fusing multi-modal sensor signals from robot skin with different control approaches. These compliant behaviors allow to teach robots safely. The presented framework is able to bridge kinesthetically demonstrated activities with low-level robot commands using a state-of-the-art teaching by demonstration method based on a semantic engine. We validate our framework in a real wheeled robot for an industrial scenario, where our presented framework enables a stiff robotic system to be compliant, flexible, and adaptable to different working conditions, e.g. different end-effectors with multiple command interfaces (position/velocity and torque interfaces).

A wearable, anti-bacterial strain sensor prepared by silver plated cotton/spandex blended fabric for human motion monitoring

The development of flexible wearable signal monitoring system is an important subject in medical, robotic skin, and smart electronic products. However, the desirable performance and easy-to-produce requirement of strain sensors are difficult to strike a balance. In this paper, we propose to fabricate a sensor based on cotton/spandex blended fabric by electroless silver plating. The resultant fabric has high sensitivity with the gauge factor (GF) 26.11 and robust electrical conductivity 15.7 S/·m, while retaining flexible properties, indicating good potential of the silver-plated fabrics to be used as strain sensing materials. Additionally, because of the silver layer, the fabric also has good anti-bacterial property. Based on the results of a series of performance tests, we find it exhibits regular resistance changes used for human monitoring and can monitor small external deformations with a response time of 0.04 s. Therefore, it can be confirmed that the wearable anti-bacterial strain sensor prepared by means of silver-plated cotton/spandex blended fabric has a good potential prospect in wearable textiles fields.

Modeling and Fabrication of Scalable Tactile Sensor Arrays for Flexible Robot Skins

Artificial robot skins capable of multi-modal tactile feedback are anticipated to revolutionize how robots perceive their immediate surroundings and how they collaborate with humans on tasks that are more complicated and dynamic than currently possible. Efforts to develop the tactile skins must tackle significant hurdles related to flexibility, robustness, and scalability of sensors. In this paper, a flexible tactile sensor array is developed to address these hurdles via a novel sensor geometry and thermal compensation technique. This tactile skin relies on a polymeric piezoresistive array of strain gauges with a novel star-shaped geometry. Due to their radial symmetry, the proposed strain gauges exhibit a biaxial response to local strain-stress loading, avoiding “blind spots” in the vicinity of individual tactels. The sensors are fabricated using gold electrodes and piezoresistive polymers (PEDOT:PSS) micro-patterned on a flexible polyimide substrate enclosed in a compliant silicone matrix. The sensor is first modeled using multiphysics finite elements software to guide the design choices for a variety of geometric parameters under various load profiles and locations with respect to the gauge’s center. A novel homogeneous thermal compensation technique is proposed, which packages the sensors on the opposite sides of a flexible carrier. This thermal compensation technique achieves a 10-fold attenuation in temperature drifts as well as a double sensitivity in terms of strain measurement. After numerical modeling, a $4\times 4$ tactile array with the optimal geometry obtained through simulation was fabricated and interfaced to a custom-made electronic data acquisition system. Samples were tested using an automated dynamic force experimental test bed. The tactile skin array was experimentally characterized in terms of force sensitivity, spatial response, dynamic bandwidth, and temperature drift. The array exhibits a sensitivity of 1125 nV/N in the range of 0–0.5 N, a sensitivity of 412 nV/N in the range of 0.5–2.3 N, a hysteresis of 11.13%, a dynamic bandwidth of 1.49 Hz, and a temperature sensitivity of 6 nV/°C, averaged among individual sensors.

A flexible resistive temperature detector (RTD) based on in-situ growth of patterned Ag film on polyimide without lithography

Flexible resistive temperature detectors (RTDs) based on metal thin film have broad application prospects in the field of portable and wearable electronics. However, most of flexible RTDs are constructed with patterned metal films using complicated manufacturing processes due to the lack of simple, scalable, environmentally friendly processing technologies. In this work, we report a flexible RTD based on in-situ growth of patterned Ag film on polyimide without lithography. The manufacturing process of flexible RTD is realized by combining ion-exchange surface modification with laser inkjet printed carbon masking processes. The developed RTDs exhibit high sensitivity and good linearity. The sensitivity in the range of −20 to +100 °C is 0.36% per °C and the linearity is ~0.16%. In addition, the flexibility and the mechanical stability of the RTDs are outstanding, with negligible degradation even in severe bending states and after 5000 times bending cycling. For demonstration, the devices are utilized to monitor the human body temperature. Therefore, this RTD demonstrates potential applications in detecting environmental temperatures, thereby marking an important progress for constructing advanced electronic skin for flexible robots. This flexible RTD opens up a new opportunity for the development of wearable and portable temperature detection devices.

Robotic Skins That Learn to Control Passive Structures

Robotic skins incorporate sensors and actuators into stretchable and flexible planar substrates. Wrapping a robotic skin around a passive, deformable structure imparts controllable motion onto that structure, rendering it an active robotic system. Robotic skins can be applied to the surface of a structure, then removed and re-applied to the surface of another structure. This reconfigurability enables use of the same robotic skin to achieve multiple motions, which depend on the interaction between the skin and its host structure. Considering the broad range of use cases for robotic skins in resource-limited environments, it may not be possible to pre-characterize this skin-structure interaction for all potential systems. Therefore, it is advantageous to have systems that can learn their models in situ, which saves considerable time in realizing a functional system. Previously, we have shown that robotic skins can be used to estimate state and stiffness of the underlying passive structure they are attached to. In this letter, we demonstrate how this ability to measure state and stiffness can be used to learn model parameters in situ for feedforward control, and show how feedback control can be implemented simultaneously with the same system. We further show how this learning is expandable to multi-segment systems and will compensate for gravitational effects by adjusting model parameters.

Sensing the Frictional State of a Robotic Skin via Subtractive Color Mixing

The perception of surface properties such as shape and slipperiness is crucial to ensure that the hand-held object is stable. Without touch, precise manipulation becomes difficult. Some robotic tactile sensors use cameras that observe the elastic deformation of a membrane to detect edges or slippage of the contact. The perception of the contact state drove innovative control strategies. However, previous methods on these lines do not include quantitative means of measuring the three-dimensional (3-D) deformation of the skin or suffer from a lack of spatial resolution. Here, we present a tactile sensor based on a subtractive color mixing process designed to track the 3-D displacement of an array of markers, using the information delivered by the color channel of off-the-shelf cameras. The distributed shear and normal deformation can be assessed from the spectrum of the light reflected and refracted by an array of diffusive and transmissive markers placed on two superimposed layers. The markers show various blends of colors, depending on the displacement at the surface. The color pattern of each marker can be tracked with little computation and remains robust to external lighting. The ability to sense the 3-D deformation field can improve robotic perception of frictional properties that have applications in the fields of robotic control and human-robot interactions.

Low-hysteresis and low-interference soft tactile sensor using a conductive coated porous elastomer and a structure for interference reduction

The need for soft whole-body tactile sensors is emerging. Piezoresistive materials are advantageous in terms of making large tactile sensors, but the hysteresis of piezoresistive materials is a major drawback. The hysteresis of a piezoresistive material should be attenuated to make a practical piezoresistive soft tactile sensor. In this paper, we introduce a low-hysteresis and low-interference soft tactile sensor using a conductive coated porous elastomer and a structure to reduce interference (grooves). The developed sensor exhibits low hysteresis because the transduction mechanism of the sensor is dominated by the contact between the conductive coated surface. In a cyclic loading experiment with different loading frequencies, the mechanical and piezoresistive hysteresis values of the sensor are less than 21.7% and 6.8%, respectively. The initial resistance change is found to be within 4% after the first loading cycle. To reduce the interference among the sensing points, we also propose a structure where the grooves are inserted between the adjacent electrodes. This structure is implemented during the molding process, which is adopted to extend the porous tactile sensor to large-scale and facile fabrication. The effects of the structure are investigated with respect to the normalized design parameters θD, θW, and θT in a simulation, and the result is validated for samples with the same design parameters. An indentation experiment also shows that the structure designed for interference reduction effectively attenuates the interference of the sensor array, indicating that the spatial resolution of the sensor array is improved. As a result, the sensor can exhibit low hysteresis and low interference simultaneously. This research can be used for many applications, such as robotic skin, grippers, and wearable devices.