Tactile Information Research Articles

A tactile perception method with flexible grating structural color

Abstract Affordable high-resolution cameras and state-of-the-art computer vision techniques have led to the emergence of various vision-based tactile sensors. However, current vision-based tactile sensors mainly depend on geometric optics or marker tracking for tactile assessments, resulting in limited performance. To solve this dilemma, we introduce optical interference patterns as the visual representation of tactile information for flexible tactile sensors. We propose a novel tactile perception method and its corresponding sensor combining structural colors from flexible blazed gratings with deep learning. The richer structural colors and finer data processing foster the tactile estimation performance. The proposed sensor has an overall normal force magnitude accuracy of 6 mN, a planar resolution of 79 μm, and a contact-depth resolution of 25 μm. This work presents a promising tactile method that combines wave optics, soft materials, and machine learning. It demonstrates well in tactile measurement, and can be expanded into multiple sensing fields.

Abraded optical fibre-based dynamic range force sensor for tissue palpation

Tactile information acquired through palpation plays a crucial role in relation to surface characterisation and tissue differentiation - an essential clinical requirement during surgery. In the case of Minimally Invasive Surgery, access is restricted, and tactile feedback available to surgeons is therefore reduced. This paper presents a novel stiffness controllable, dynamic force range sensor that can provide remote haptic feedback. The sensor has an abraded optical fibre integrated into a silicone dome. Forces applied to the dome change the curvature of the optical fibres, resulting in light attenuation. By changing the pressure within the dome and thereby adjusting the sensor’s stiffness, we are able to modify the force measurement range. Results from our experimental study demonstrate that increasing the pressure inside the dome increases the force range whilst decreasing force sensitivity. We show that the maximum force measured by our sensor prototype at 20 mm/min was 5.02 N, 6.70 N and 8.83 N for the applied pressures of 0 psi (0 kPa), 0.5 psi (3.45 kPa) and 1 psi (6.9 kPa), respectively. The sensor has also been tested to estimate the stiffness of 13 phantoms of different elastic moduli. Results show the elastic modulus sensing range of the proposed sensor to be from 8.58 to 165.32 kPa.

Low power tactile sensory neuron using nanoparticle-based strain sensor and memristor

Endowing strain sensors with neuromorphic computing capabilities could permit the efficient processing of tactile information on the edge. The realization of such functionalities from a simple circuit without software processing holds promise for attaining skin-based perception. Here, leveraging the intrinsic neuronal plasticity of memristive neurons, various firing patterns induced by the applied strain were demonstrated. More specifically, tonic, bursting, transition from tonic to bursting, adaptive, and nociceptive activities were captured. The implementation of these patterns permits the facile translation of the analog pressure signals into digital spikes, attaining accurate perception of various tactile characteristics. The tactile sensory neuron consisting of an RC circuit was composed of a SiO2-based conductive bridge memristor exhibiting leaky integrate-and-fire properties and a Pt nanoparticles (NPs)-based strain sensor with a gauge factor of ∼270. A dense layer of Pt NPs was also used as the bottom electrode for the memristive element, yielding the manifestation of a threshold switching mode with a switching voltage of only ∼350 mV and an exceptional switching ratio of 107. Our work provides valuable insights for developing low power neurons with tactile feedback for prosthetics and robotics applications.

Stress-induced artificial neuron spiking in diffusive memristors

Diffusive memristors owing to their ability to produce current spiking when a constant or slowly changing voltage is applied are competitive candidates for development of artificial electronic neurons. These artificial neurons can be integrated into various prospective autonomous and robotic systems as sensors, e.g. ones implementing object grasping and classification. We report here Ag nanoparticle-based diffusive memristor prepared on a flexible polyethylene terephthalate substrate in which the electric spiking behaviour was induced by the electric voltage under an additional stimulus of external mechanical impact. By changing the magnitude and frequency of the mechanical impact, we are able to manipulate the spiking response of our artificial neuron. This functionality to control the spiking characteristics paves a pathway for the development of touch-perception sensors that can convert local pressure into electrical spikes for further processing in neural networks. We have proposed a mathematical model which captures the operation principle of the fabricated memristive sensors and qualitatively describes the measured spiking behaviour. Employing such flexible diffusive memristors that can directly translate tactile information into spikes, similar to force and pressure sensors, could offer substantial benefits for various applications in robotics.

During haptic communication, the central nervous system compensates distinctly for delay and noise.

Physically connected humans have been shown to exploit the exchange of haptic forces and tactile information to improve their performance in joint action tasks. As human interactions are increasingly mediated through robots and networks it is important to understand the impact that network features such as lag and noise may have on human behaviour. In this paper, we investigated interaction with a human-like robot controller that provides similar haptic communication behaviour as human-human interaction and examined the influence and compensation mechanisms for delay and noise on haptic communication. The results of our experiments show that participants can perceive a difference between noise and delay, and make use of compensation mechanisms to preserve performance in both cases. However, while noise is compensated for by increasing co-contraction, delay compensation could not be explained by this strategy. Instead, computational modelling suggested that a distinct mechanism is used to compensate for the delay and yield an efficient haptic communication.

HaptoFloater: Visuo-Haptic Augmented Reality by Embedding Imperceptible Color Vibration Signals for Tactile Display Control in a Mid-Air Image.

We propose HaptoFloater, a low-latency mid-air visuo-haptic augmented reality (VHAR) system that utilizes imperceptible color vibrations. When adding tactile stimuli to the visual information of a mid-air image, the user should not perceive the latency between the tactile and visual information. However, conventional tactile presentation methods for mid-air images, based on camera-detected fingertip positioning, introduce latency due to image processing and communication. To mitigate this latency, we use a color vibration technique; humans cannot perceive the vibration when the display alternates between two different color stimuli at a frequency of 25 Hz or higher. In our system, we embed this imperceptible color vibration into the mid-air image formed by a micromirror array plate, and a photodiode on the fingertip device directly detects this color vibration to provide tactile stimulation. Thus, our system allows for the tactile perception of multiple patterns on a mid-air image in 59.5 ms. In addition, we evaluate the visual-haptic delay tolerance on a mid-air display using our VHAR system and a tactile actuator with a single pattern and faster response time. The results of our user study indicate a visual-haptic delay tolerance of 110.6 ms, which is considerably larger than the latency associated with systems using multiple tactile patterns.

Wearable Haptics for Orthotropic Actuation Based on Perpendicularly Nested Auxetic SMA Knotting.

Smart wearable tactile systems, designed to deliver different types of touch feedback on human skin, can significantly improve engagement through diverse actuation patterns in virtual or augmented reality environments. Here, a perpendicularly nested auxetic wearable haptic interface is reported for orthotropically decoupled multimodal actuation (WHOA), capable of producing diverse tactile feedback modes with 3D sensory perception. WHOA incorporates shape memory alloy wires that are intricately knotted into an auxetic structure oriented along orthotropic dual axes. Its perpendicularly nested auxetic structure enables orthotropic actuation, allowing independent expansion and contraction along both x and y-axes, as confirmed by force-strain and displacement-time performance tests. Additionally, the perylene coating provides orthogonal electrical isolation to WHOA, allowing for stripe-specific localized actuation and enabling multiple tactile feedback modes. As an orthotropic wearable haptic interface, WHOA distinguishes between x-axis and y-axis directions and ultimately delivers multi-dimensional information regarding movements in 3D space through tactile feedback. As a result, when worn on the foot or arm, WHOA naturally delivers spatiotemporal tactile information to the user, facilitating navigation and teleoperation with 3D sensory perception.

Valenced tactile information is evoked by neutral visual cues following emotional learning

Abstract Learning which stimuli in our environment co-occur with painful or pleasurable events is critical for survival. Previous research has established the basic neural and behavioral mechanisms of aversive and appetitive conditioning; however, it is unclear precisely what information content is learned. Here we examined the degree to which aspects of the unconditioned stimulus (US)—sensory information versus affective salience—are transferred to the conditioned stimulus (CS). To decode what stimuli features (e.g., valence vs. discriminative somatosensation) are represented in patterns of brain activation elicited during appetitive (soft touch) and aversive (painful touch) conditioning to faces, a novel approach to using modeling with representational similarity analysis (RSA) based on theoretically driven representational patterns of interest (POIs) was applied to fMRI data. Once associations were learned through conditioning, globally, the CS reactivated US representational patterns showing conditioning-dependent reactivation in specific high-order brain regions: In the dorsal anterior cingulate cortex, the CS reactivated patterns associated with the affective salience of the US—suggesting that, with affective conditioning, these regions carry forward the affective associations of the experience.

Characteristics of Perceived Location of Tactile Information Presented to the Palm

Characteristics of Perceived Location of Tactile Information Presented to the Palm

Effect of touch on proprioception: non-invasive trigeminal nerve stimulation suggests general arousal rather than tactile-proprioceptive integration.

Proprioceptive error of estimated fingertip position in two-dimensional space is reduced with the addition of tactile stimulation applied at the fingertip. Tactile input does not disrupt the participants' estimation strategy, as the individual error vector maps maintain their overall structure. This relationship suggests integration of proprioception and tactile information improves proprioceptive estimation, which can also be improved with trained mental focus and attention. Task attention and arousal are physiologically regulated by the reticular activating system (RAS), a brainstem circuit including the locus coeruleus (LC). There is direct and indirect evidence that these structures can be modulated by non-invasive trigeminal nerve stimulation (nTNS), providing an opportunity to examine nTNS effect on the integrative relationship of proprioceptive and tactile information. Fifteen right-handed participants performed a simple right-handed proprioceptive estimation task with tactile feedback (touch) and no tactile (hover) feedback. Participants repeated the task after nTNS administration. Stimulation was delivered for 10 min, and stimulation parameters were 3,000 Hz, 50 μs pulse width, with a mean of 7 mA. Error maps across the workspace are generated using polynomial models of the participants' target responses. Error maps did not demonstrate significant vector direction changes between conditions for any participant, indicating that nTNS does not disrupt spatial proprioception estimation strategies. A linear mixed model regression with nTNS epoch, tactile condition, and the interaction as factors demonstrated that nTNS reduced proprioceptive error under the hover condition only. We argue that nTNS does not disrupt spatial proprioceptive error maps but can improve proprioceptive estimation in the absence of tactile feedback. However, we observe no evidence that nTNS enhances tactile-proprioceptive integration as the touch condition does not exhibit significantly reduced error after nTNS.

Contributions of tactile information to the sense of agency and its metacognitive representations.

Despite the ubiquitous presence of tactile information in our daily activities, studies of how we experience agency of our actions have rarely relied on manipulated visuo-tactile feedback. Instead, what is often manipulated are the distal (and arbitrarily associated) consequences of our actions. The few studies that did investigate whether tactile information contributes to the experience of agency have been limited to the binary assessment of tactile feedback about the outcome of an action being present or absent. Here, we went beyond the coarse comparison of agency with versus without tactile feedback and introduced instead an experimental manipulation where we could control the amount of mismatch between predictions and observations. Participants (N = 40) reached with their right hand toward a ridged plate with a specific orientation and saw online feedback that could match or differ from their action in one of three ways: the physical plate's orientation, the action's timing, or the hand's position in space. Absolute subjective ratings revealed that an increased mismatch in tactile information led to a diminished sense of agency, similar to what has been reported for spatial and temporal mismatches. Further, estimations of metacognitive efficiency revealed similar Mratios in identifying visuo-tactile violation predictions as compared to visuo-temporal violations (but lower than visuospatial). These findings emphasize the importance of tactile information in shaping our experience of acting voluntarily and show how this important component can be experimentally probed. (PsycInfo Database Record (c) 2024 APA, all rights reserved).

Lack of visual experience leads to severe distortions in the hand representation of the body model

This study investigates the impact of vision on the maintenance of hand representation in the implicit body model, particularly focusing on congenitally blind individuals. To address this, we performed a hand landmark localization task on blind individuals who lacked visual experience of their bodies and compared their performance to normally sighted and normally sighted but blindfolded participants. Through measurements of finger lengths, hand width, and shape index, we demonstrate that blind participants exhibit significantly greater distortions in their hand representation compared to sighted and blindfolded controls. Notably, blind individuals displayed a marked overestimation of hand width and an underestimation of finger lengths, particularly in digits D2, D3, and D4. Surprisingly, blind subjects with partial vision displayed more severe distortions than those with no residual vision. Furthermore, our findings reveal that late-blind participants exhibit similar levels of distortion as congenitally blind individuals, suggesting an extended period of susceptibility to the lack of visual input in shaping body representations. The Reverse Distortion (RD) hypothesis provides a plausible explanation for these distortions, suggesting that compensatory mechanisms occur within the body model to counteract the anisotropic cortical representations. Our results support this hypothesis: blind individuals have expanded cortical representations processing tactile information, so this could lead to more pronounced distortions in their hand representation of the body model. This underscores the importance of visual input in modulating body representations. Overall, our study highlights the malleability of body representations and the intricate interplay between sensory inputs and cortical processing in shaping the implicit body model.

Soft Micropneumatic Touchpad

Soft tactile sensors outputting fluidic signals have many potential applications in soft microfluidic devices and soft robots. However, existing systems have been limited to a single or a few sensors in parallel, so they are not comparable to the state‐of‐the‐art electrical resistive and capacitive touchpads, which can detect rich tactile information, including touch location, pressure, area, and even multiple touches simultaneously. This work reports a soft micropneumatic touchpad. The touchpad consists of 32 pneumatic channels inside soft elastomer, with 16 channels aligned row‐wise and 16 column‐wise. The flow resistance of each channel is measured using a pressure divider. When the pad is touched, the cross‐sectional area of the channels close to the contact location deforms, which changes the flow resistance of those channels. With 32 sensing channels, the location, depth, area of the contact, and even two simultaneous touches can be detected. Letters hand‐written on the touchpad can be reconstructed from the measured data. With the assumption of sparsity, a tactile pressure map, with a value at each 16 × 16 grid point, can also be reconstructed. This work opens a path to replace electronic tactile sensors in soft devices with all‐fluidic alternatives.

Tactile Sensing and Rendering Patch with Dynamic and Static Sensing and Haptic Feedback for Immersive Communication.

Wearable human-machine interface (HMI) with bidirectional and multimodal tactile information exchange is of paramount importance in teleoperation by providing more intuitive data interpretation and delivery of tactilely related signals. However, the current sensing and feedback devices still lack enough integration and modalities. Here, we present a Tactile Sensing and Rendering Patch (TSRP) that is made of a customized expandable array which consists of a piezoelectric sensing and feedback unit fused with an elastomeric triboelectric multidimensional sensor and its inner pneumatic feedback structure. The primary functional unit of TSRP is mainly featured with a soft silicone substrate with compact multilayer structure integrating static and dynamic multidimensional tactile sensing capabilities, which synergistically leverage both triboelectric and piezoelectric effects. Additionally, based on the air chamber created by the triboelectric sensor and the converse piezoelectric effect, it provides pneumatic and vibrational haptic feedback simultaneously for both static and dynamic perception regeneration. With the aid of the other variants of this unit, the array shaped TSRP is capable of simulating different terrains, geometries, sliding, collisions, and other critical interactive events during teleoperation via skin perception. Moreover, immediate manipulation can be done on TSRP through the tactile sensors. The preliminary demonstration of TSRP interface with a completed control module in robotic teleoperation is provided, which shows the feasibility of assisting certain tasks in a complex environment by direct tactile communication. The proposed device offers a potential method of enabling bidirectional tactile communication with enriched key information for improving interaction efficiency in the fields of robot teleoperation and training.

AI-assisted 3D analysis of grasping and reaching behavior of squirrel monkeys during recovery from cervical spinal cord injury

We have previously demonstrated that machine learning-based video analysis, conducted via DeepLabCut, is more sensitive for detecting subtle deficits in hand grasping behavior than traditional end-point performance assessments. This superiority was observed in a nonhuman primate (NHP) model of cervical spinal cord injury, specifically a dorsal column lesion (DCL). The current study aims to further characterize the kinematic aspects of the deficits in hand reaching, grasping, and retrieving behavior from a 3D perspective following a DCL. Squirrel monkeys were trained to retrieve sugar pellets from eight wells, which were located either on a flat plate or a raised tube with varying well depths. This setup was designed to require coordinated finger movements during the task. Immediately after the DCL, the animals exhibited measurable behavioral deficits. These were characterized by significant increases in grasping speed squared and trial completion time, markedly widened movement trajectories of individual fingers, and abnormalities in inter-finger distance and orientation. Increased task difficulty was associated with more pronounced behavioral deficits. By three months post-DCL, video-based measurements indicated no significant recovery, even though global end-point performance had returned to baseline levels. Our findings demonstrate that deprivation of tactile information results in impaired dexterous hand behavior involving coordinated finger movements, and the impairment is sustained for 20 weeks. This spinal cord injury (SCI) model, along with DeepLapCut analysis, provides a valuable platform for separately evaluating sensory and motor functions and their contributions to dexterous hand behavior and may be used for evaluating therapeutic interventions using more sensitive behavioral outcome readouts.

Brain encoding of naturalistic, continuous, and unpredictable tactile events.

Studies employing EEG to measure somatosensory responses have been typically optimized to compute event-related potentials in response to discrete events (ERPs). However, tactile interactions involve continuous processing of non-stationary inputs that change in location, duration, and intensity. To fill this gap, this study aims to demonstrate the possibility of measuring the neural tracking of continuous and unpredictable tactile information. Twenty-seven young adults (females = 15) were continuously and passively stimulated with a random series of gentle brushes on single fingers of each hand, which were covered from view. Thus, tactile stimulations were unique for each participant, and stimulated fingers. An encoding model measured the degree of synchronization between brain activity and continuous tactile input, generating a temporal response function (TRF). Brain topographies associated with the encoding of each finger stimulation showed a contralateral response at central sensors starting at 50 ms and peaking at about 140 ms of lag, followed by a bilateral response at about 240 ms. A series of analyses highlighted that reliable tactile TRF emerged after just 3 minutes of stimulation. Strikingly, topographical patterns of the TRF allowed discriminating digit lateralization across hands and digit representation within each hand. Our results demonstrated for the first time the possibility of using EEG to measure the neural tracking of a naturalistic, continuous, and unpredictable stimulation in the somatosensory domain. Crucially, this approach allows the study of brain activity following individualized, idiosyncratic tactile events to the fingers.Significant Statement This study expands the current research conducted on neural tracking, opening the exploration of idiosyncratic tactile events and overcoming constraints of laboratory tasks that typically rely on discrete events. We validated a protocol for the ecological investigations of continuous, slow, tactile processing of the hands. The employed approach enriches the possible use of the EEG to characterize somatosensory neural representations of tactile events. Findings unravel coherent neural responses to continuous and naturalistic touch, with sensitivity for digit lateralization and representation.

A Multifunctional Tactile Sensory System for Robotic Intelligent Identification and Manipulation Perception.

Humans recognize and manipulate objects relying on the multidimensional force features captured by the tactile sense of skin during the manipulation. Since the current sensors integrated in robots cannot support the robots to sense the multiple interaction states between manipulator and objects, achieving human-like perception and analytical capabilities remains a major challenge for service robots. Prompted by the tactile perception involved in robots performing complex tasks, a multimodal tactile sensory system is presented to provide in situ simultaneous sensing for robots when approaching, touching, and manipulating objects. The system comprises a capacitive sensor owning the high sensitivity of 1.11E-2pFmm-1, a triboelectricity nanogenerator with the fast response speed of 30ms, and a pressure sensor array capable of 3D force detection. By Combining transfer learning models, which fuses multimodal tactile information to achieve high-precision (up to 95%) recognition of the multi-featured targets such as random hardness and texture information under random sampling conditions, including random grasp force and velocity. This sensory system is expected to enhance the intelligent recognition and behavior-planning capabilities of autonomous robots when performing complex tasks in undefined surrounding environments.

Effect of training in a sandy environment on foot morphology and function

BackgroundAthletes need to enhance their foot function to improve their performance and prevent injuries and disability. Although foot function is believed to be improved by stabilizing the body on an unstable surface (e.g., a sandy beach), there have been no relevant studies to our knowledge. Here, we identified the differences in foot and balance functions between beach volleyball and volleyball players and investigated the effects of sandy surface training. MethodsWe included six male beach volleyball players (BVB group) and six male volleyball players (VB group). The following six parameters were measured and compared between the groups: foot morphology, plantar surface perception, isometric ankle muscle strength, toe grip strength, static balance, and dynamic balance. ResultsThe BVB group had significantly higher values in sensation perception, isometric ankle dorsiflexion muscle strength, and toe grip strength in the standing posture, with significant intrinsic foot muscle dominance, compared to the VB group. ConclusionPlaying barefoot increases the amount of tactile information received from the surface of the sole, which may lead to enhanced sensory perception by the foot. In addition, owing to the unstable sandy surface, the intrinsic muscles of the foot and lower leg may strengthen to maintain balance; therefore, training on a sandy surface may lead to improved foot and balance functions.

An improved DFD method for three-dimensional displacement measurement of vision-based tactile sensor

Currently, traditional tactile sensors based on the principles of capacitance or piezoelectricity have complex structures and difficulty in obtaining tactile information. A vision-based tactile sensor is introduced which can realize visual measurement of three-dimensional displacement in this paper. The vision-based tactile sensor is mainly composed of an elastomer embedded with marker point array, a transparent acrylic plate, 8 LED lights and a micro monocular camera. The elastomer deforms when the tactile sensor contacts an object, and the micro monocular camera is used to capture the elastomer deformation and transmit it to the computer in the form of image, and then the three-dimensional displacement information is obtained by processing the image. In order to more accurately recover the missing dimensional information in the three-dimensional displacement detection of monocular camera, an improved DFD (Depth from Defocus) method based on finite element theory is proposed in this paper. It is verified by experiments that the improved DFD method proposed in this paper can measure the three-dimensional displacement information more accurately compared with the DFD method. In addition, an experiment is conducted to prove the robustness of the improved DFD method on the robotic gripper. The experimental results demonstrate that the three-dimensional displacement measurement method proposed in this paper can provide technical support for the design and development of vision-based tactile sensors.

A bioinspired tactile scanner for computer haptics

Computer haptics (CH) is about integration of tactile sensation and rendering in Metaverse. However, unlike computer vision (CV) where both hardware infrastructure and software programs are well developed, a generic tactile data capturing device that serves the same role as what a camera does for CV, is missing. Bioinspired by electrophysiological processes in human tactile somatosensory nervous system, here we propose a tactile scanner along with a neuromorphically-engineered system, in which a closed-loop tactile acquisition and rendering (re-creation) are preliminarily achieved. Based on the architecture of afferent nerves and intelligent functions of mechano-gating and leaky integrate-and-fire models, such a tactile scanner is designed and developed by using piezoelectric transducers as axon neurons and thin film transistor (TFT)-based neuromorphic circuits to mimic synaptic behaviours and neural functions. As an example, the neuron-like tactile information of surface textures is captured and further used to render the texture friction of a virtual surface for “recreating” a “true” feeling of touch.