Tactile Modality Research Articles

Reading Braille by Touch Recruits Posterior Parietal Cortex.

Blind readers use a tactile reading system consisting of raised dot arrays: braille/⠃⠗⠇. How do human brains implement reading by touch? The current study looked for signatures of reading-specific orthographic processes in braille, separate from low-level somatosensory responses and semantic processes. Of specific interest were responses in posterior parietal cortices (PPCs), because of their role in high-level tactile perception. Congenitally blind, proficient braille readers read real words and pseudowords by touch while undergoing fMRI. We leveraged the system of contractions in English braille, where one braille cell can represent multiple English print letters (e.g., "ing" ⠬, "one" ⠐⠕), making it possible to separate physical and orthographic word length. All words in the study consisted of four braille cells, but their corresponding Roman letter spellings varied from four to seven letters (e.g., "con-c-er-t" ⠒⠉⠻⠞. contracted: four cells; uncontracted: seven letters). We found that the bilateral supramarginal gyrus in the PPC increased its activity as the uncontracted word length increased. By contrast, in the hand region of primary somatosensory cortex (S1), activity increased as a function of a low-level somatosensory feature: dot-number per word. The PPC also showed greater response to pseudowords than real words and distinguished between real and pseudowords in multivariate-pattern analysis. Parieto-occipital, early visual and ventral occipito-temporal, as well as prefrontal cortices also showed sensitivity to the real-versus-pseudoword distinction. We conclude that PPC is involved in orthographic processing for braille, that is, braille character and word recognition, possibly because of braille's tactile modality.

Crossmodal transfer of perceptual learning: Evidence from the recognition of first and second language characters in young children.

Crossmodal transfer of learning is a neurocognitive process whereby a learner's experience and knowledge acquired through one sensory mode enable him/her to perform a similar task using a different sensory mode. This study examined the transfer of (mostly) visually acquired knowledge of first- and second-language characters to the tactile modality typically not used in that acquisition process. Two experiments were conducted, one to assess letter recognition skills and one to assess digit recognition skills in both Bangla and English, in 30 sighted young children who had mastered those characters through sensory learning in natural settings. Results unequivocally demonstrated that children were able to recognize/classify the first and second language letters or digits presented not only to the (trained) visual modality but to the (untrained) tactile modality as well, and as expected, with greater recognition accuracy and shorter recognition time in the former than the latter modality. Their character recognition performance was found to be significantly influenced not by language but by character type, with digits being more accurately and more speedily recognized than letters. Moreover, language-task modality interaction was found to mediate letter recognition accuracy, digit recognition accuracy, and digit recognition time, whereas character type-task modality interaction was found to significantly mediate character recognition time only. The ecological and theoretical significance of these findings is discussed. (PsycInfo Database Record (c) 2023 APA, all rights reserved).

Self-powered smart skins for multimodal tactile perception based on triboelectric and hygroelectric working principles

Human being perceives multiple tactile modalities in the process of sensation on the skin and interpretation in the brain. To date, several sensing techniques facilitate the accurate measurement of individual tactile modality, but multimodal static and dynamic sensing remain challenging. Moreover, low-cost and highly efficient interpretation techniques are still required for tactile perception. Herein, we present cost-effective and high-performing self-powered smart skins that mimic multimodal tactile perception, enabling accurate perception of pressure, vibration, and humidity in the process of sensation on the smart skin and interpretation by machine learning. The dynamic and static stimuli are encoded by triboelectric and hygroelectric principles in the smart skins, respectively, while the hygroscopic nature empowers humidity sensation capability in the smart skin with an accuracy rate as high as 84.0%−100.0%. We believe our smart skin will enable the smooth transition of e-skin into practical applications, such as robotics, prosthetics, healthcare, and intelligent industry.

No evidence for tactile entrainment of attention.

Temporal patterns in our environment provide a rich source of information, to which endogenous neural processes linked to perception and attention can synchronize. This phenomenon, known as entrainment, has so far been studied predominately in the visual and auditory domains. It is currently unknown whether sensory phase-entrainment generalizes to the tactile modality, e.g., for the perception of surface patterns or when reading braille. Here, we address this open question via a behavioral experiment with preregistered experimental and analysis protocols. Twenty healthy participants were presented, on each trial, with 2 s of either rhythmic or arrhythmic 10 Hz tactile stimuli. Their task was to detect a subsequent tactile target either in-phase or out-of-phase with the rhythmic entrainment. Contrary to our hypothesis, we observed no evidence for sensory entrainment in response times, sensitivity or response bias. In line with several other recently reported null findings, our data suggest that behaviorally relevant sensory phase-entrainment might require very specific stimulus parameters, and may not generalize to the tactile domain.

Haptic Communication of Language.

In this review, the development of communication systems and devices that convey language tactually is examined, first from an historical perspective focusing on the communities who use the tactile modality to substitute for impairments in vision and/or hearing. Then, the more recent developments in wearable tactile communication systems for conveying text and speech to those without sensory impairments are reviewed. The performance of tactile display technology developed for these user communities is discussed in the context of the proficiency achieved by skilled users of natural methods of tactile communication. In tracing the history of tactile devices used to convey language, it is evident that technological advances in other domains, such as screen readers and speech synthesizers for the visually impaired and cochlear implants for those with hearing loss, have had a profound impact on the requirements for effective tactile language systems. For some communities, such as the Deafblind, it is essential that the tactile communication platform is bi-directional so that the user can both send and receive language. Devices developed to address such needs have yet to achieve commercial success. Recent research on wearable tactile displays has highlighted the importance of extensive training for learning and retaining languages presented tactually.

Auditory, tactile, and multimodal noise reduce balance variability

Auditory and somatosensory white noise can stabilize standing balance. However, the differential effects of auditory and tactile noise stimulation on balance are unknown. Prior work on unimodal noise stimulation showed gains in balance with white noise through the auditory and tactile modalities separately. The current study aims to examine whether multimodal noise elicits similar responses to unimodal noise. We recorded the postural sway of healthy young adults who were presented with continuous white noise through the auditory or tactile modalities and through a combination of both (multimodal condition) using a wearable device. Our results replicate previous work that showed that auditory or tactile noise reduces sway variability with and without vision. Additionally, we show that multimodal noise also reduces the variability of sway. Analysis of different frequency bands of sway is typically used to separate open-loop exploratory (< 0.3 Hz) and feedback-driven (> 0.3 Hz) sway. We performed this analysis and showed that unimodal and multimodal white noise affected postural sway variability similarly in both timescales. These results support that the sensory noise effects on balance are robust across unimodal and multimodal conditions and can affect both mechanisms of sway represented in the frequency spectrum. In future work, the parameters of acoustic/tactile manipulation should be optimized for the most effective balance stabilization, and multimodal therapies should be explored for older adults with typical age-related balance instabilities.

The multimodal emotion perception in codependent individuals

The emotional disturbances of individuals with codependency are often ignored. This study aimed to investigate the emotional perception of codependent individuals in four modalities – visual, auditory, tactile and olfactory. An EEG study was performed and presented pleasant and unpleasant stimuli selected by a panel of experts for each modality. Participants (fifteen codependent individuals and fifteen healthy volunteers) were instructed to assess the emotional impact and pleasantness of stimuli. The method of EEG spaces was used to visualize how close perceived stimuli were according to EEG data. The results showed ambivalence of emotional response to emotional stimuli with social component and lack of recognition of emotional tone detected in EEG and behavioral levels. The empathy feeling in codependent individuals was detected for fewer emotional stimuli. The group differences were associated with evolutionary newer modalities (auditory and visual). The lack of emotional perception in codependent subjects was determined by social factors and was detected in visual and auditory modalities, which were more involved in social interactions than olfactory and tactile modalities.

β-Bursts over Frontal Cortex Track the Surprise of Unexpected Events in Auditory, Visual, and Tactile Modalities.

One of the fundamental ways in which the brain regulates and monitors behavior is by making predictions about the sensory environment and adjusting behavior when those expectations are violated. As such, surprise is one of the fundamental computations performed by the human brain. In recent years, it has been well established that one key aspect by which behavior is adjusted during surprise is inhibitory control of the motor system. Moreover, because surprise automatically triggers inhibitory control without much proactive influence, it can provide unique insights into largely reactive control processes. Recent years have seen tremendous interest in burst-like β frequency events in the human (and nonhuman) local field potential-especially over (p)FC-as a potential signature of inhibitory control. To date, β-bursts have only been studied in paradigms involving a substantial amount of proactive control (such as the stop-signal task). Here, we used two cross-modal oddball tasks to investigate whether surprise processing is accompanied by increases in scalp-recorded β-bursts. Indeed, we found that unexpected events in all tested sensory domains (haptic, auditory, visual) were followed by low-latency increases in β-bursting over frontal cortex. Across experiments, β-burst rates were positively correlated with estimates of surprise derived from Shannon's information theory, a type of surprise that represents the degree to which a given stimulus violates prior expectations. As such, the current work clearly implicates frontal β-bursts as a signature of surprise processing. We discuss these findings in the context of common frameworks of inhibitory and cognitive control after unexpected events.

Sensorimotor synchronization with visual, auditory, and tactile modalities.

While it is well known that humans are highly responsive to rhythm, the factors that influence our ability to synchronize remain unclear. In the current study, we examined how stimulus modality and rhythmic deviation, along with the synchronizer's level of musicality, impacted sensorimotor synchronization (SMS). Utilizing a finger-tapping task and three sensory modalities (visual, auditory, and tactile), we manipulated rhythmic deviation by varying the temporal position, intensity, and availability of cues across four deviation levels. Additionally, to determine our participants' musical familiarity and aptitude, we administered the Goldsmiths Musical Sophistication Index (Gold-MSI) questionnaire. We found that SMS to external rhythmic stimuli was significantly more precise for auditory and tactile than for visual sequences. Further, we found SMS consistency significantly decreased in all modalities with increased rhythmic deviation, suggesting rhythmic deviation directly relates to SMS difficulty. Moreover, a significant correlation was found between Gold-MSI scores and SMS consistency in the most rhythmically deviant level, such that the higher one's musical general sophistication score, the greater one's SMS ability. This held for all three modalities. Combined, these findings suggest that rhythmic synchronization performance is affected not only by the modality and rhythmic deviation of the stimuli but also by the musical general sophistication of the synchronizer.

A Highly Sensitive Multimodal Tactile Sensing Module with Planar Structure for Dexterous Manipulation of Robots

Herein, a multimodal tactile sensing module that can improve the dexterous manipulation capabilities of robots with parallel grippers is reported. The tactile sensor consists of a 4 × 4 matrix 3‐axis force sensor array (with spacings of 2 mm) and a single‐temperature sensor. The tactile sensor uses inorganic silicon and gold as materials for the detection of strain and temperature and a polymer‐based elastomer to encapsulate the sensing layer. The sensing module is equipped with a readout circuitry for signal processing. Within the measurable force range (≈1.5 N) of the sensor cell, a prototype module exhibits a repeatability error within 2%, a hysteresis error within 3%, and a resolution as small as 10 mN. Furthermore, each sensor cell independently measures 3‐axis forces with a cross‐talk error of approximately 3%. The temperature sensor exhibits linear output properties in the approximate range of 5–75 °C. Experiments are performed by mounting the module on a parallel gripper to grasp a paper cup and a reflex hammer toy. According to the experimental results, the sensor can accurately detect the state of contact with an object by analyzing three tactile modalities (i.e., 3‐axis force distribution, vibration, and temperature).

Lifelong Visual-Tactile Spectral Clustering for Robotic Object Perception

This work presents a novel visual-tactile fused clustering framework, called <underline xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">L</u> ifelong <underline xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">V</u> isual- <underline xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">T</u> actile <underline xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">S</u> pectral <underline xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">C</u> lustering (i.e., LVTSC), to effectively learn consecutive object clustering tasks for robotic perception. Lifelong learning has become an important and hot topic in recent studies on machine learning, aiming to imitate “human learning” and reduce the computational cost when consecutively learning new tasks. Our proposed LVTSC model explores the knowledge transfer and representation correlation from a local modality-invariant perspective under modality-consistent constraint guidance. For the modality-invariant part, we design a set of modality-invariant basis libraries to capture the latent clustering centers of each modality and a set of modality-invariant feature libraries to forcibly embed the manifold information of each modality. A modal-consistent constraint reinforces the correlation between visual and tactile modalities by maximizing the feature manifold correspondences. When the object clustering task comes continuously, the overall objective is optimized by an effective alternating direction method with guaranteed convergence. Our proposed LVTSC framework has been extensively validated for its effectiveness and efficiency on the three challenging real-world robotic object perception datasets.

How an Intervention on the Bodily Tactile Modality Can Improve Communication

Introduction: In this case study, performed in a home situation in Zambia, it was assumed that the child with congenital deafblindness had too poor vision to access sign language without support by the bodily tactile modality. The aim was to explore if a theoretically based intervention on Communicative Relations with a role model who demonstrates communication in the bodily tactile modality and builds on knowledge and competence in sign language can improve bodily tactile aspects of communication between the child and her mother and brother. Method: This study has a qualitative single case study design, which enabled a close analysis of the communication between the child and her mother and brother, by a reflective team. Four members of the reflective team provided insight into the videorecorded episodes in home situations such as bathing and play. Data analysis took place before and after the intervention. The intervention was performed by the researcher as a role model, who is a mother herself, and well trained in the Diamond Model and Dialogical Model underlying the theory of Communicative Relations ( Nafstad &amp;amp; Rodbroe, 2015). Results: The results demonstrated that the intervention with the application of aspects of the Diamond Model was effective in face-to-face interaction, gestures and social interaction, and the application of aspects of the Dialogical Model enhanced the ability to reciprocate signs during the interaction. The findings in this study clarify how use of sign language and hand positions can benefit persons with CDB to improve their communication. The study has shown how competent mother and brother were communicating with the child after intervention using ZSL, reciprocation and imitation and hand positions. The utterances that were addressed by the child to the partners in gestural expressions were answered by the partners. Lastly, it has shown the need to focus on whole-body communication, understanding, and relating to communicative expressions. Conclusion and discussion: It can be concluded that the intervention had indeed a positive effect on a variety of bodily tactile communication aspects between the child and her family communication partners. The role model approach that was used in this study enabled mother and brother and child to learn through reciprocation of roles and imitation. It is a natural form of learning which does not necessarily require shared reference to theoretical concepts. In addition, the study showed that only sign language is not enough. There must be a connectedness between signing and adapted interaction patterns.

Coining Tactile Signs: A Guided Experience of Tactile Communication for People with Acquired Deafblindness and their Communication Partners

Introduction: When hearing and vision are affected simultaneously, communication can be challenging (Nordic Welfare Centre, 2016). However, undertaking a bodily – tactile modality of communication can benefit the interactions for people with acquired deafblindness and their partners (Lahtinen, 2008). This research aimed to understand the process of creating tactile communication. It focused on analysing the process of coining tactile signs for specific objects. Method: A qualitative and descriptive case study was conducted, utilizing video-analysis and questionnaires. An intervention was based on a learning guide and inspired by the project Landscape of Touch (Tactile Communication Group DbI, 2012). Different activities of tactile communication, progressive in complexity, led the participants to a final experience of coining (the creation of) their own tactile signs. Results: The results showed three blended stages in the coining process, tactile exploration, information selection and negotiation of meaning. The coined signs were compared with their visual pair of NZSL [New Zealand Sign Language] finding several similarities in their components. This research had an expanding effect on the participant’s perspective of tactile communication and the possibility of integrating it to their daily life. Conclusion and Discussion: Further research on sign coining and tactile elements in the communication, including individual experiences, is essential for developing good practices for educators and support service providers in this field. It is recommended to use video analysis as a tool to observe the details in the handmovements. Some limitations in this study were the application in a case study, therefore no generalisations can be made, and the access to technology limited the possibilities for other possible participants.

Effects of task demands on tactile vigilance.

A performance decline during sustained monitoring of unpredictable and occasional signals, the vigilance decrement, has been studied mostly in the visual and auditory modalities, but a tactile vigilance decrement also has been observed and has been associated with high perceived workload, declines in sensitivity and task engagement, and increases in distress. The primary aim of the current study was to determine whether task demands affect the vigilance decrement in the tactile modality and whether the effects are similar to those observed in the auditory and visual modalities. Participants completed a 40-min vigil in which they monitored vibrotactile stimuli generated by a tactor and had to discriminate between durations of bursts of vibrations. Task demand was varied by including low and high event rates. Although correct detections decreased over time (vigilance decrement) and sensitivity was greater for the slower event rate, there was not an interaction between period of watch and event rate. There also were no differences in workload and stress between event rates. Results indicate that mean performance in tactile vigilance tasks is negatively impacted by increases in event rate, indicating that a typical source of task demand known to affect visual and auditory vigilance also affects tactile vigilance. Results could be explained by either an underload or overload theory of the vigilance decrement.

Hierarchical unimodal processing within the primary somatosensory cortex during a bimodal detection task

Do sensory cortices process more than one sensory modality?To answer these questions, scientists have generated a wide variety of studies at distinct space-time scales in different animal models, and often shown contradictory conclusions. Some conclude that this process occurs in early sensory cortices, but others that this occurs in areas central to sensory cortices. Here, we sought to determine whether sensory neurons process and encode physical stimulus properties of different modalities (tactile and acoustic). Forthis, we designed a bimodal detection task where the senses of touch and hearing compete from trial to trial. Two Rhesus monkeys performed this novel task, while neural activity was recorded in areas 3b and 1 of the primary somatosensory cortex (S1). Weanalyzed neurons' coding properties and variability, organizing them by their receptive field's position relative to the stimulation zone. Our results indicate that neurons of areas 3b and 1 are unimodal, encoding only the tactile modality in both the firing rate and variability. Moreover, we found that neurons in area 3b carried more information about the periodic stimulus structure than those in area 1, possessed lower response and coding latencies, and had a lower intrinsic time scale. In sum, these differences reveal a hidden processing-based hierarchy. Finally, using a powerful nonlinear dimensionality reduction algorithm, we show that the activity from areas 3b and 1 can be separated, establishing a clear division in the functionality of these two subareas of S1.

Tactile expectancy modulates occipital alpha oscillations in early blindness

Alpha oscillatory activity is thought to contribute to visual expectancy through the engagement of task-relevant occipital regions. In early blindness, occipital alpha oscillations are systematically reduced, suggesting that occipital alpha depends on visual experience. However, it remains possible that alpha activity could serve expectancy in non-visual modalities in blind people, especially considering that previous research has shown the recruitment of the occipital cortex for non-visual processing. To test this idea, we used electroencephalography to examine whether alpha oscillations reflected a differential recruitment of task-relevant regions between expected and unexpected conditions in two haptic tasks (texture and shape discrimination). As expected, sensor-level analyses showed that alpha suppression in parieto-occipital sites was significantly reduced in early blind individuals compared with sighted participants. The source reconstruction analysis revealed that group differences originated in the middle occipital cortex. In that region, expected trials evoked higher alpha desynchronization than unexpected trials in the early blind group only. Our results support the role of alpha rhythms in the recruitment of occipital areas in early blind participants, and for the first time we show that although posterior alpha activity is reduced in blindness, it remains sensitive to expectancy factors. Our findings therefore suggest that occipital alpha activity is involved in tactile expectancy in blind individuals, serving a similar function to visual anticipation in sighted populations but switched to the tactile modality. Altogether, our results indicate that expectancy-dependent modulation of alpha oscillatory activity does not depend on visual experience. Significance statementAre posterior alpha oscillations and their role in expectancy and anticipation dependent on visual experience? Our results show that tactile expectancy can modulate posterior alpha activity in blind (but not sighted) individuals through the engagement of occipital regions, suggesting that in early blindness, alpha oscillations maintain their proposed role in visual anticipation but subserve tactile processing. Our findings bring a new understanding of the role that alpha oscillatory activity plays in blindness, contrasting with the view that alpha activity is task unspecific in blind populations.

CNN‐Transformer for visual‐tactile fusion applied in road recognition of autonomous vehicles

Reliable autonomous driving requires comprehensive environment perception, among which the road recognition is critical for autonomous vehicles to achieve adaptability, reliability, and safety. Existing equipped sensors such as cameras, LiDAR, and accelerometers have been adopted widely for road recognition. However, single sensor based recognition methods present challenges in balancing high accuracy and adaptability. In this study, we propose a visual-tactile fusion road recognition system for autonomous vehicles. The visual modality is derived from the captured road images, and the tactile modality information comes from the designed intelligent tire system, containing a low-cost piezoelectric sensor. For accurate road recognition, we propose a multimodal fusion recognition network based on the CNN-transformer architecture. The visual and tactile modalities are fed into modality-specific SE-CNNs, which emphasize valuable input information to obtain weighted features. These features are subsequently input to "bottleneck" based fusion transformer encoders and output the recognition results. We design a fusion feature extractor to enhance the fusion representation capability and improve accuracy. The vehicle field experiments are conducted to build the dataset consisting of four road surfaces, and the results show that the proposed network achieves an accuracy of 99.48% in road recognition task.

The mechanisms involved in rocking toddlers

If the tactile modality is fundamental in theinfant's experience, there is a particular situation in which it is mobilized: rocking. Carrying a baby in one's arms or trying to soothe him will spontaneously lead the adult to rock him, whatever his culture. Theways of carrying out this rhythmic movement are varied, but all are characterized by a synchronous combination of somesthesic and vestibular stimulations repeated at regular intervals. Thespecificities ofthetactile, vestibular and temporal modalities involved in rocking highlight the way in which rocking provides an essential support to the sensory and social development of the infant.

Visual-Tactile Fused Graph Learning for Object Clustering.

Object clustering has received considerable research attention most recently. However, 1) most existing object clustering methods utilize visual information while ignoring important tactile modality, which would inevitably lead to model performance degradation and 2) simply concatenating visual and tactile information via multiview clustering method can make complementary information to not be fully explored, since there are many differences between vision and touch. To address these issues, we put forward a graph-based visual-tactile fused object clustering framework with two modules: 1) a modality-specific representation learning module MR and 2) a unified affinity graph learning module MU . Specifically, MR focuses on learning modality-specific representations for visual-tactile data, where deep non-negative matrix factorization (NMF) is adopted to extract the hidden information behind each modality. Meanwhile, we employ an autoencoder-like structure to enhance the robustness of the learned representations, and two graphs to improve its compactness. Furthermore, MU highlights how to mitigate the differences between vision and touch, and further maximize the mutual information, which adopts a minimizing disagreement scheme to guide the modality-specific representations toward a unified affinity graph. To achieve ideal clustering performance, a Laplacian rank constraint is imposed to regularize the learned graph with ideal connected components, where noises that caused wrong connections are removed and clustering labels can be obtained directly. Finally, we propose an efficient alternating iterative minimization updating strategy, followed by a theoretical proof to prove framework convergence. Comprehensive experiments on five public datasets demonstrate the superiority of the proposed framework.

Design and realization of a novel haptic graspable interface for augmenting touch sensations.

A novel haptic grasper that renders touch sensations to the user in 3-DoF (degrees of freedom), namely linear, rotary, and grasping motions, is presented. The touch sensations of the grasper include the combination of kinesthetic and tactile modalities such as stiffness, texture, and shape. The device is equipped with two swappable modular segments that provide stiffness and shape sensations. To increase the haptic fidelity, the textural surfaces that surround the outer surface of the segments are equipped with vibro-actuators underneath them. These vibro-actuators contribute to increasing the number of perceivable textures by varying amplitude, frequency, duration, and envelope of vibrations. The proposed device is characterized in terms of stiffness, shape and texture rendering capabilities. The experimental results validate the effectiveness of the developed haptic grasper in virtual/remote interactions. Also, the user studies and statistical analysis demonstrate that the users could perceive the high-fidelity haptic feedback with the unified sensations of kinesthetic and tactile cues.