Tactile Intelligence Research Articles

Deep learning-assisted object recognition with hybrid triboelectric-capacitive tactile sensor

Tactile sensors play a critical role in robotic intelligence and human-machine interaction. In this manuscript, we propose a hybrid tactile sensor by integrating a triboelectric sensing unit and a capacitive sensing unit based on porous PDMS. The triboelectric sensing unit is sensitive to the surface material and texture of the grasped objects, while the capacitive sensing unit responds to the object’s hardness. By combining signals from the two sensing units, tactile object recognition can be achieved among not only different objects but also the same object in different states. In addition, both the triboelectric layer and the capacitor dielectric layer were fabricated through the same manufacturing process. Furthermore, deep learning was employed to assist the tactile sensor in accurate object recognition. As a demonstration, the identification of 12 samples was implemented using this hybrid tactile sensor, and an recognition accuracy of 98.46% was achieved. Overall, the proposed hybrid tactile sensor has shown great potential in robotic perception and tactile intelligence.

Vision-based tactile intelligence with soft robotic metamaterial

Robotic metamaterials represent an innovative approach to creating synthetic structures that combine desired material characteristics with embodied intelligence, blurring the boundaries between materials and machinery. Inspired by the functional qualities of biological skin, integrating tactile intelligence into these materials has gained significant interest for research and practical applications. This study introduces a Soft Robotic Metamaterial (SRM) design featuring omnidirectional adaptability and superior tactile sensing, combining vision-based motion tracking and machine learning. The study compares two sensory integration methods to a state-of-the-art motion tracking system and force/torque sensor baseline: an internal-vision design with high frame rates and an external-vision design offering cost-effectiveness. The results demonstrate the internal-vision SRM design achieving an impressive tactile accuracy of 98.96%, enabling soft and adaptive tactile interactions, especially beneficial for dexterous robotic grasping. The external-vision design offers similar performance at a reduced cost and can be adapted for portability, enhancing material science education and robotic learning. This research significantly advances tactile sensing using vision-based motion tracking in soft robotic metamaterials, and the open-source availability on GitHub fosters collaboration and further exploration of this innovative technology (https://github.com/bionicdl-sustech/SoftRoboticTongs).

Magnetostrictive-based multimodal tactile sensors for object recognition

Tactile perception of the human hand plays a critical role in everyday object recognition. The development of multimodal tactile sensors that can sense stimuli with high sensitivity and low cost is important for intelligent perception. In this article, a multimodal tactile sensor attached on a mechanical hand is studied, which consists of a magnetostrictive tactile sensor, a temperature sensor, and a flex sensor. By applying multimodal tactile sensors to a robotic hand to grasp objects, the output voltage of the magnetotactictive tractile tactile sensor can be used for object shape and softness recognition. The bending angle of the knuckles can be obtained by the flex sensor for object contour size recognition, while the temperature distribution of objects can be obtained through temperature sensors. In order to improve the accuracy, a 1-D convolutional neural network-extreme learning machine (CNN-ELM) pattern recognition model based on the combination of 1-D CNN and ELM is presented, with the accuracy of 97.14% for 21 objects. This multimodal tactile sensor has promising applications in the field of tactile intelligence and humanoid robotics.

Autoencoding a Soft Touch to Learn Grasping from On‐Land to Underwater

Robots play a critical role as the physical agent of human operators in exploring the ocean. However, it remains challenging to grasp objects reliably while fully submerging under a highly pressurized aquatic environment with little visible light, mainly due to the fluidic interference on the tactile mechanics between the finger and object surfaces. This study investigates the transferability of grasping knowledge from on‐land to underwater via a vision‐based soft robotic finger that learns 6D forces and torques (FT) using a supervised variational autoencoder (SVAE). A high‐framerate camera captures the whole‐body deformations while a soft robotic finger interacts with physical objects on‐land and underwater. Results show that the trained SVAE model learns a series of latent representations of the soft mechanics transferable from land to water, presenting a superior adaptation to the changing environments against commercial FT sensors. Soft, delicate, and reactive grasping enabled by tactile intelligence enhances the gripper's underwater interaction with improved reliability and robustness at a much‐reduced cost, paving the path for learning‐based intelligent grasping to support fundamental scientific discoveries in environmental and ocean research.

Active Multiobject Exploration and Recognition via Tactile Whiskers

Robotic exploration under uncertain environments is challenging when optical information is not available. In this paper, we propose an autonomous solution of exploring an unknown task space based on tactile sensing alone. We first designed a whisker sensor based on MEMS barometer devices. This sensor can acquire contact information by interacting with the environment non-intrusively. This sensor is accompanied by a planning technique to generate exploration trajectories by using mere tactile perception. This technique relies on a hybrid policy for tactile exploration, which includes a proactive informative path planner for object searching, and a reactive Hopf oscillator for contour tracing. Results indicate that the hybrid exploration policy can increase the efficiency of object discovery. Last, scene understanding was facilitated by segmenting objects and classification. A classifier was developed to recognize the object categories based on the geometric features collected by the whisker sensor. Such an approach demonstrates the whisker sensor, together with the tactile intelligence, can provide sufficiently discriminative features to distinguish objects.

Ultrahigh-transparency and pressure-sensitive iontronic device for tactile intelligence

Emerging tactile sensing devices mimic biological functions of human mechanoreception. By introducing the feature of optical transparency, it can lead to a combined capacities of tactile and visual intelligence into single system. Yet, it is difficult to realize ultrahigh level of optical transparency and device sensitivity in single structure, for the widely used methods for sensitivity improvement, such as elevating the interfacial roughness, may further reduce the transparency. By utilizing a transparent ionic material with tunable surface topologies, as well as introducing a strategy of refractive index matching, we have proposed a transparent iontronic sensing (TIS) device based on the iontronic sensing mechanism, simultaneously offering combined high device sensitivity (83.9 kPa−1), with ultrahigh optical transparency (96.9%), the highest reported value in literature. Benefiting from its comprehensive performance in sensing and optical characteristics, the TIS devices hold enormous potential for the human-machine interfaces for industrial and medical applications.

From sense of touch to tactile intelligence: Thoughts from the 2021 Nobel Prize in Physiology or Medicine

<p indent=0mm>Human beings have highly developed sense of touch. Through touching we can perceive heat, pain, mechanical stimuli, recognize features of an object such as texture and hardness, and even understand emotion of a person. Hence, human touch is one of the most important ways to sense and adapt to the world around us. However, we had little understanding of how cutaneous perception was formed in our nervous system, until the groundbreaking discoveries were made by this year’s Nobel Prize laureates in Physiology or Medicine, David Julius from University of California, San Francisco, and Ardem Patapoutian from Scripps Research. They found that heat and mechanical stimuli can initiate specific nerve impulses for temperature, pain, and touch sensations. The quest to understand temperature sensation and pain spans millennia. In the early 20th century, the Greek physician Galen proposed that sensations were transmitted along pathways that connect the organs to the brain. It was not until 1997 that the true underlying molecular mechanism was revealed by Julius in a breakthrough work. Julius and his team found that a receptor on the cell’s membrane arranges itself from one side to the other, so that it creates a passage called a transient receptor potential (TRP) channel. This ion channel opens or closes in response to heat. Inspired by Julius’s work, Patapoutian in the early 21st century explored a mechanosensitive ion channel, called the Piezo channel, and identified that it is essential for the sense of touch. Patapoutian and his team also found that the Piezo channel is responsible for the sensing of position and motion of our body, which explains why we can balance on a steep stair. Not only did these discoveries elucidate the physiological understanding of human somatosensory, they also enlightened the researchers who work on reproducing such human functions in robotics and prosthetics. The molecular mechanisms of thermo-sensation and mechano-sensation provided a new paradigm for designing artificial cutaneous perception and constructing electronic skin for robots. Once fully armed by tactile perception and haptic feedback, smart robots and artificial intelligence may start to have consciousness and feel emotions like we humans do. With the awarding of the 2021 Nobel Prize in Physiology or Medicine to research in how humans perceive touch and pain, the research in tactile intelligence now has a clearer roadmap and gains a greater momentum. We foresee many exciting breakthroughs in this field in the near future.

A Force and Temperature Sensor Array Based on 3-D Field-Coupled Thin-Film Transistors for Tactile Intelligence

Tactile sensors that acquire force and temperature information are key to machine tactile intelligence. In this article, an active tactile sensor incorporating an array of 3-D field-coupled thin-film transistors (FCTFTs) with a $4 \times 4$ pixelated polyvinylidene fluoride (PVDF)-based force and temperature sensors is studied. In order to enhance sensitivity and reduce power consumption, a 3-D FIN-shaped FCTFT is designed and a subthreshold operation is implemented. We have demonstrated its functions in human–machine interfacing, object characterization and recognition, force and temperature mapping, and dynamic force tracing. Such a tactile sensor array is promising for applications in machine tactile intelligence and humanoid robotics.

Embodied tactile perception and learning

Various living creatures exhibit embodiment intelligence, which is reflected by a collaborative interaction of the brain, body, and environment. The actual behavior of embodiment intelligence is generated by a continuous and dynamic interaction between a subject and the environment through information perception and physical manipulation. The physical interaction between a robot and the environment is the basis for realizing embodied perception and learning. Tactile information plays a critical role in this physical interaction process. It can be used to ensure safety, stability, and compliance, and can provide unique information that is difficult to capture using other perception modalities. However, due to the limitations of existing sensors and perception and learning methods, the development of robotic tactile research lags significantly behind other sensing modalities, such as vision and hearing, thereby seriously restricting the development of robotic embodiment intelligence. This paper presents the current challenges related to robotic tactile embodiment intelligence and reviews the theory and methods of robotic embodied tactile intelligence. Tactile perception and learning methods for embodiment intelligence can be designed based on the development of new large‐scale tactile array sensing devices, with the aim to make breakthroughs in the neuromorphic computing technology of tactile intelligence.

Recent developments in robotic tactile perception

PurposeThis paper aims to provide details of recent developments in robotic tactile sensing.Design/methodology/approachFollowing a short introduction, this paper first provides an overview of tactile sensing effects and technologies. It then discusses recent developments in tactile sensing skins. Tactile sensing for robotic prosthetics and hands is then considered and is followed by a discussion of “tactile intelligence”. Various experimental results are included. Finally, brief concluding comments are drawn.FindingsThis shows that many advanced, sensitive and technologically varied tactile sensing devices are being developed. These devices are expected to impart robots with a range of enhanced capabilities such as improved gripping and manipulation, object recognition, the control and robotic hands and prosthetics and collision detection.Originality/valueTactile sensing has an increasingly important role to play in robotics, and this paper provides a technical insight into a number of recent developments and their applications.

渡邊淳司著 情報を生み出す触覚の知性 情報社会をいきるための感覚のリテラシー

渡邊淳司著 情報を生み出す触覚の知性 情報社会をいきるための感覚のリテラシー

Educación, símbolo, tacto. Más allá del modelo instrumental en la pedagogía

I argue that the pedagogical theories based on modern epistemologies of instrumental reason are not entirely suited to studying education. This is due to the huge complexity and depth of man and education, which needs to be captured interpretively, with a hermeneutic or tactile intelligence, which is the Greek phronesis, or practical wisdom. This requires an effort to understand what can never be fully understood or illuminated, considered something symbolic, ambiguous, dark, singular, and contingent. Scientific reasoning, which aims to fully illuminate and master it is taking the wrong approach and one that is hostile to the nuances of this human reality called “education”.