Vibrotactile Signals Research Articles

Tactile Sensitivity to Missing tones in Complex Vibrotactile Signals.

Haptic interactions with objects induce complex vibrotactile signals that are central to tactile perception. Despite the broad literature on vibrotactile perception, surprisingly little is known about the sensory processing of complex tactile signals made of multiple pure tones. To fill this gap, the study reported here investigated the impact of the constitutive pure tones of a complex vibrotactile signal on its perception. Participants completed a 3-AFC task, in which they were asked to identify an odd signal among two complex references. The odd signal was created by removing one pure tone from the reference, which varied in spectral composition, harmonicity, and inter-frequency intervals. Each reference signal was made of either two, three, or four pure tones. The results revealed that the removed pure tone's value as well as the inter-frequency interval play a significant role on participants' performance whereas changes in harmonicity and complexity have little impact. The smaller the ratio between the removed frequency and the lowest one of the reference signals, the better the participants' capacity to identify the signal with the missing tone. As this ratio correlates with that of pure tone's perceived intensity, participants' performance can be linked to either of them. Analysis of a subset of complex signals made of pure tones perceived with roughly equal intensity showed that the correlation still holds but slightly decreases. Overall, these results suggest that perception of complex vibrations might be mediated by tactile mechanisms related to both frequency selectivity and to pure tones perceived intensity.

A multimodal multitask deep learning framework for vibrotactile feedback and sound rendering

Data-driven approaches are often utilized to model and generate vibrotactile feedback and sounds for rigid stylus-based interaction. Nevertheless, in prior research, these two modalities were typically addressed separately due to challenges related to synchronization and design complexity. To this end, we introduce a novel multimodal multitask deep learning framework. In this paper, we developed a comprehensive end-to-end data-driven system that encompasses the capture of contact acceleration signals and sound data from various texture surfaces. This framework introduces novel encoder-decoder networks for modeling and rendering vibrotactile feedback through an actuator while routing sound to headphones. The proposed encoder-decoder networks incorporate stacked transformers with convolutional layers to capture both local variability and overall trends within the data. To the best of our knowledge, this is the first attempt to apply transformer-based data-driven approach for modeling and rendering of vibrotactile signals as well as sounds during tool-surface interactions. In numerical evaluations, the proposed framework demonstrates a lower RMS error compared to state-of-the-art models for both vibrotactile signals and sound data. Additionally, subjective similarity evaluation also confirm the superiority of proposed method over state-of-the-art.

Helping Blind People Grasp: Evaluating a Tactile Bracelet for Remotely Guiding Grasping Movements.

The problem of supporting visually impaired and blind people in meaningful interactions with objects is often neglected. To address this issue, we adapted a tactile belt for enhanced spatial navigation into a bracelet worn on the wrist that allows visually impaired people to grasp target objects. Participants' performance in locating and grasping target items when guided using the bracelet, which provides direction commands via vibrotactile signals, was compared to their performance when receiving auditory instructions. While participants were faster with the auditory commands, they also performed well with the bracelet, encouraging future development of this system and similar systems.

The high/low frequency balance drives tactile perception of noisy vibrations.

Noisy vibrotactile signals transmitted during tactile explorations of an object provide precious information on the nature of its surface. Understanding the link between signal properties and how they are interpreted by the tactile sensory system remains challenging. In this paper, we investigated human perception of broadband, stationary vibrations recorded during exploration of textures and reproduced using a vibrotactile actuator. Since intensity is a well-established perceptual attribute, we here focused on the relevance of the spectral content. The stimuli were first equalized in perceived intensity and subsequently used to identify the most salient spectral features using dissimilarity estimations between pairs of successive vibration. Based on dimensionally reduced spectral representations, models of dissimilarity ratings showed that the balance between low and high frequencies was the most important cue. Formal validation of this result was achieved through a Mushra experiment, in which participants assessed the fidelity of resynthesized vibrations with various distorted frequency balances. These findings offer valuable insights into human vibrotactile perception and establish a computational framework for analyzing vibrations as humans do. Moreover, they pave the way for signal synthesis and compression based on sparse representations, holding significance for applications involving complex vibratory feedback.

HapticPilot

The emergence of vibrotactile feedback in hand wearables enables immersive virtual reality (VR) experience with whole-hand haptic rendering. However, existing haptic rendering neglects inconsistent sensations caused by hand postures. In our study, we observed that changing hand postures alters the distribution of vibrotactile signals which might degrade one's haptic perception. To address the issues, we present HapticPilot which allows an in-situ haptic experience design for hand wearables in VR. We developed an in-situ authoring system supporting instant haptic design. In the authoring tool, we applied our posture-adaptive haptic rendering algorithm with a novel haptic design abstraction called phantom grid. The algorithm adapts phantom grid to the target posture and incorporates 1D & 2D phantom sensation with a unique actuator arrangement to provide a whole-hand experience. With this method, HapticPilot provides a consistent haptic experience across various hand postures is available. Through measuring perceptual haptic performance and collecting qualitative feedback, we validated the usability of the system. In the end, we demonstrated our system with prospective VR scenarios showing how it enables an intuitive, empowering, and responsive haptic authoring framework.

Speech-derived haptic stimulation enhances speech recognition in a multi-talker background

Speech understanding, while effortless in quiet conditions, is challenging in noisy environments. Previous studies have revealed that a feasible approach to supplement speech-in-noise (SiN) perception consists in presenting speech-derived signals as haptic input. In the current study, we investigated whether the presentation of a vibrotactile signal derived from the speech temporal envelope can improve SiN intelligibility in a multi-talker background for untrained, normal-hearing listeners. We also determined if vibrotactile sensitivity, evaluated using vibrotactile detection thresholds, modulates the extent of audio-tactile SiN improvement. In practice, we measured participants’ speech recognition in a multi-talker noise without (audio-only) and with (audio-tactile) concurrent vibrotactile stimulation delivered in three schemes: to the left or right palm, or to both. Averaged across the three stimulation delivery schemes, the vibrotactile stimulation led to a significant improvement of 0.41 dB in SiN recognition when compared to the audio-only condition. Notably, there were no significant differences observed between the improvements in these delivery schemes. In addition, audio-tactile SiN benefit was significantly predicted by participants’ vibrotactile threshold levels and unimodal (audio-only) SiN performance. The extent of the improvement afforded by speech-envelope-derived vibrotactile stimulation was in line with previously uncovered vibrotactile enhancements of SiN perception in untrained listeners with no known hearing impairment. Overall, these results highlight the potential of concurrent vibrotactile stimulation to improve SiN recognition, especially in individuals with poor SiN perception abilities, and tentatively more so with increasing tactile sensitivity. Moreover, they lend support to the multimodal accounts of speech perception and research on tactile speech aid devices.

Vibrotactile Thresholds on the Wrist for Vibrations Normal to the Skin.

In order to facilitate the development of wrist-worn vibrotactile devices, detailed knowledge about how vibrations are perceived by the users is needed. In particular, perceptual thresholds in amplitude are really important. Thresholds have been measured in the literature for other areas of the body, but given the variability reported between areas (shape of the threshold curve, position of maximum sensitivity), thresholds on the wrist can not be inferred from previous measurements and must be measured. The amplitude thresholds for vibrations normal to the skin surface were evaluated on 28 participants, with a three interval forced choice method. They were measured for 7 frequencies that are classical in the literature about vibrotactile perception (25, 40, 80, 160, 250, 320, and 640 Hz). The classical U-shape of the amplitude-threshold curve is observed, with a maximum sensitivity at around 160 Hz, which differs from other body areas, but confirms recent results obtained for vibrations parallel to the skin surface of the same body area. The sensitivity thresholds of vibrotactile signals appear to be in the micrometer range.

Vibrotactile Alerting to Prevent Accidents in Highway Construction Work Zones: An Exploratory Study.

Struck-by accidents are the leading cause of injuries in highway construction work zones. Despite numerous safety interventions, injury rates remain high. As workers' exposure to traffic is sometimes unavoidable, providing warnings can be an effective way to prevent imminent threats. Such warnings should consider work zone conditions that can hinder the timely perception of alerts, e.g., poor visibility and high noise level. This study proposes a vibrotactile system integrated into workers' conventional personal protective equipment (PPE), i.e., safety vests. Three experiments were conducted to assess the feasibility of using vibrotactile signals to warn workers in highway environments, the perception and performance of vibrotactile signals at different body locations, and the usability of various warning strategies. The results revealed vibrotactile signals had a 43.6% faster reaction time than audio signals, and the perceived intensity and urgency levels on the sternum, shoulders, and upper back were significantly higher than the waist. Among different notification strategies used, providing a moving direction imposed significantly lower mental workloads and higher usability scores than providing a hazard direction. Further research should be conducted to reveal factors that affect alerting strategy preference towards a customizable system to elicit higher usability among users.

High Efficiency Vibrotactile Codec Based on Gate Recurrent Network

The multimedia has achieved dominant positions in both local storage and internet bandwidth, which inevitably promotes the compression of audio, image and video information. Nowadays, the emerging haptic technology, which enhances the immersion in virtual reality and remote control, has also brought new challenges in its codec design. It is thus imperative to develop haptic codecs, including kinesthetic and vibrotactile codecs, with high efficiency and low delay. In this paper, we exploit statistical features of vibrotactile data to develop a Recurrent-Network-based Vibrotactile Codec (RNVC) with high compression efficiency and low coding delay. The proposed encoder consists of vibrotactile estimation by Gate Recurrent Unit (GRU), non-uniform quantization/compensation of residuals and an entropy encoder. In particular, the GRU-based recurrent network is utilized for its high efficiency to predict signals and low complexity to converge. The decoder consists of all counterparts of encoder. Experimental results show the proposed RNVC significantly reduces of original bitrates with negligible encoding delay, which achieves the state-of-the-art coding performance of vibrotactile signal.

Effect of audio-tactile congruence on vibrotactile music enhancement.

Music listening experiences can be enhanced with tactile vibrations. However, it is not known which parameters of the tactile vibration must be congruent with the music to enhance it. Devices that aim to enhance music with tactile vibrations often require coding an acoustic signal into a congruent vibrotactile signal. Therefore, understanding which of these audio-tactile congruences are important is crucial. Participants were presented with a simple sine wave melody through supra-aural headphones and a haptic actuator held between the thumb and forefinger. Incongruent versions of the stimuli were made by randomizing physical parameters of the tactile stimulus independently of the auditory stimulus. Participants were instructed to rate the stimuli against the incongruent stimuli based on preference. It was found making the intensity of the tactile stimulus incongruent with the intensity of the auditory stimulus, as well as misaligning the two modalities in time, had the biggest negative effect on ratings for the melody used. Future vibrotactile music enhancement devices can use time alignment and intensity congruence as a baseline coding strategy, which improved strategies can be tested against.

Temporally Local Tactile Codes Can Be Stored in Working Memory.

Tactile exploration often involves sequential touches interspersed with stimulus-free durations (e.g., the time during which the hand moves from one textured surface to the other). Whereas it is obvious that texture-related perceptual variables, irrespective of the encoding strategy, must be stored in memory for comparison, it is rather unclear which of those variables are held in memory. There are two established variables—“intensity” and “frequency”, which are “temporally global” variables because of the long stimulus integration interval required to average the signal or derive spectral components, respectively; on the other hand, a recently established third contender is the “temporally local” variable that codes for kinematic profiles of very short, suprathreshold events in the vibrotactile signal. Here, we present the first psychophysical evidence that temporally local variables can be stored in memory. To that end, we asked participants to detect changes in pulsatile indentation stimuli at their fingertips with and without a gap of 1 s between stimulus presentations. The stimuli either contained global variables alone (change of pulse rate), or a mix of local and global variables (change of pulse shape). We found, first, that humans are much better at detecting a change in stimuli when local variables are available rather than global ones alone—as evident by the fact that 21 compared to only 6 participants out of 25 yielded a valid psychophysical curve, respectively. Second, this observation persists even when there is a gap between the stimuli, implying local variables must be stored in memory. Third, an extensive array of relevant intensity definitions failed to explain participants’ performance in any consistent manner, which implies that perceptual decisions were less likely to be driven by intensity coding. Taken together, our results suggest that humans perform pulsatile change detection utilizing local pulse shape, and to a lesser degree global pulse rate, and that both parameters can be stored in memory.

Multi-channel FM transmission of vibrotactile signals on 2-D communication textile

Multi-channel FM transmission of vibrotactile signals on 2-D communication textile

Enhancement of speech-in-noise comprehension through vibrotactile stimulation at the syllabic rate

SignificanceSyllables are important building blocks of speech. They occur at a rate between 4 and 8 Hz, corresponding to the theta frequency range of neural activity in the cerebral cortex. When listening to speech, the theta activity becomes aligned to the syllabic rhythm, presumably aiding in parsing a speech signal into distinct syllables. However, this neural activity cannot only be influenced by sound, but also by somatosensory information. Here, we show that the presentation of vibrotactile signals at the syllabic rate can enhance the comprehension of speech in background noise. We further provide evidence that this multisensory enhancement of speech comprehension reflects the multisensory integration of auditory and tactile information in the auditory cortex.

In-Vehicle Human Machine Interface: Investigating the Effects of Tactile Displays on Information Presentation in Automated Vehicles

<bold xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">Background:</b> Semi-autonomous vehicles still require human drivers to take over when the automated systems can no longer perform the driving task. <bold xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">Objective:</b> The goal of this study was to design and test the effects of six meaningful tactile signal types, representing six driving scenarios (i.e., navigation, speed, surrounding vehicles, over the speed limit, headway reductions, and pedestrian status) respectively, and two pattern durations (lower and higher urgencies), on drivers’ perception and performance during automated driving. <bold xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">Methods:</b> Sixteen volunteers participated in an experiment utilizing a medium-fidelity driving simulator presenting vibrotactile signals via 20 tactors embedded in the seat back, pan, and belt. Participants completed four separate driving sessions with 30 tactile signals presented randomly throughout each drive. Reaction times (RT), interpretation accuracy, and subjective ratings were measured. <bold xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">Results:</b> Results illustrated shorter RTs and higher intuitive ratings for higher urgency patterns than lower urgency patterns. Pedestrian status and headway reduction signals were associated with shorter RTs and increased confidence ratings, compared to other tactile signal types. Lastly, among six tactile signals, surrounding vehicle and navigation signal types had the highest interpretation accuracy. <bold xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">Conclusion:</b> These results will be used as preliminary data for future studies that aim to investigate the effects of meaningful tactile displays on automated vehicle takeover performance in complex situations (e.g., urban areas) where actual takeovers are required. The findings of this study will inform the design of next-generation in-vehicle human-machine interfaces.

Improving postural stability among people with lower-limb amputations by tactile sensory substitution

BackgroundFor people with lower-limb amputations, wearing a prosthetic limb helps restore their motor abilities for daily activities. However, the prosthesis's potential benefits are hindered by limited somatosensory feedback from the affected limb and its prosthesis. Previous studies have examined various sensory substitution systems to alleviate this problem; the prominent approach is to convert foot–ground interaction to tactile stimulations. However, positive outcomes for improving their postural stability are still rare. We hypothesized that the sensory substiution system based on surrogated tactile stimulus is capable of improving the standing stability among people with lower-limb amputations.MethodsWe designed a wearable device consisting of four pressure sensors and two vibrators and tested it among people with unilateral transtibial amputations (n = 7) and non-disabled participants (n = 8). The real-time measurements of foot pressure were fused into a single representation of foot–ground interaction force, which was encoded by varying vibration intensity of the two vibrators attached to the participants’ forearm. The vibration intensity followed a logarithmic function of the force representation, in keeping with principles of tactile psychophysics. The participants were tested with a classical postural stability task in which visual disturbances perturbed their quiet standing.ResultsWith a brief familiarization of the system, the participants exhibited better postural stability against visual disturbances when switching on sensory substitution than without. The body sway was substantially reduced, as shown in head movements and excursions of the center of pressure. The improvement was present for both groups of participants and was particularly pronounced in more challenging conditions with larger visual disturbances.ConclusionsSubstituting otherwise missing foot pressure feedback with vibrotactile signals can improve postural stability for people with lower-limb amputations. The design of the mapping between the foot–ground interaction force and the tactile signals is essential for the user to utilize the surrogated tactile signals for postural control, especially for situations that their postural control is challenged.

Which signal modalities do cyclists prefer based on experiences in road traffic?

Objective: On-bike systems warning cyclists about critical situations are a promising approach to improve safety. The chosen warning modality might strongly influence whether the cyclist accepts the system. So far, cyclists’ warning preferences have not been analyzed based on field data. They were only analyzed through web-based surveys or a simulator study without including the three most promising signal types (i.e., visual, auditory and vibro-tactile). This study aims to evaluate the signal preferences for transmitting information to cyclists based on experience of the signals in the field. Method: We conducted a field study where participants received signals of different signal types, i. e. visual, auditory and vibro-tactile signals, while cycling. After completing the course, all participants answered a questionnaire on their subjective experiences of the signals. The participants separately cycled a 10 km long route in road traffic. All participants received 12 signals per modality on predefined GPS coordinates. The course covered different environmental conditions like loud ambient noise, gravel roads or high visual load. Results: The data of 55 participants was analyzed. The participants chose the auditory and vibro-tactile signal over the visual signal. When asked, they significantly preferred an auditory warning to the other two signal types. The participants rated the auditory signal as most urgent and frequently associated it with warnings. Participants reported the visual signal as distracting from the cycling task and the vibro-tactile signals as difficult to distinguish from surface related vibrations. Conclusion: The advantages of different signal modalities can be applied to develop information transmission systems in the cycling context. Our results show that the signal types have inherent qualities which may fit into different areas of application. This study highlights that the choice of a warning modality needs to be balanced on a combination of noticeability, criticality and personal preference.

Research on visual‐tactile cross‐modality based on generative adversarial network

Aiming at the research of assisted blind technology, a generative adversarial network model was proposed to complete the transformation of the mode from vision to touch. Firstly, two key representations of visual to tactile sense are identified: the texture image of the object and the audio frequency that generates vibrotactile. It is essentially a matter of generating audio from images. The authors propose a cross-modal network framework that generates corresponding vibrotactile signals based on texture images. More importantly, the network structure is an end-to-end, which eliminates the traditional intermediate form of converting texture image to spectrum image, and can directly carry out the transformation from visual to tactile. A quantitative evaluation system is proposed in this study, which can evaluate the performance of the network model. The experimental results show that the network can complete the conversion of visual information to tactile signals. The proposed method is proved to be superior to the existing method of indirectly generating vibrotactile signals, and the applicability of the model is verified.

Quality Enhancement of Compressed Vibrotactile Signals Using Recurrent Neural Networks and Residual Learning.

We present a neural network-based compression artifact removal technique for vibrotactile signals. The proposed decoder-side quality enhancement approach is based on recurrent neural networks (RNNs) and the principle of residual learning. We use a total of 8 nonlinear RNN layers trained to first estimate the difference between the original and the compressed signal. The estimated difference signal is then added to the compressed signal, followed by further linear processing steps to construct the enhanced signal. With our approach, we are able to enhance signals at almost all compression ratios by up to $1.25\ \mathrm {dB}$. For the signals in our data set, rougly 86% are enhanced in their quality. Through an ablation study, we show that every block of our network is functioning as intended and contributes to the compression artifact removal. Additionally, we show that the chosen network parameters maximize performance.

Perceptual Quality Assessment of Compressed Vibrotactile Signals Through Comparative Judgment.

In this article, we present a comprehensive scheme for the quality assessment of compressed vibrotactile signals with human assessors. Inspired by the multiple stimulus test with hidden reference and anchors (MUSHRA) from the audio domain, we designed a method in which each compressed signal is compared to its original signal and rated on a numerical scale. For each signal tested, the hidden reference and two anchor signals are used to validate the results and provide assessor screening criteria. Differing from previous approaches, our method is hierarchically structured and strictly timed in a sequential manner to avoid experimental confounds and provide precise psychophysical assessments. We validated our method in an experiment with 20 human participants in which we compared two state-of-the-art lossy codecs. The results show that, with our approach, the performance of different codecs can be compared effectively. Furthermore, the method also provides a measure of subjective quality at different data compression rates. The proposed procedure can be easily adapted to evaluate other vibrotactile codecs.

PVC-SLP: Perceptual Vibrotactile-Signal Compression Based-on Sparse Linear Prediction

Developing a signal compression technique that is able to achieve a low bit rate while maintaining high perceptual signal quality is a classical signal processing problem vigorously studied for audio, speech, image, and video type of signals. Yet, until recently, there has been limited effort directed toward the compression of vibrotactile signals, which represent a crucial element of rich touch (haptic) information. A vibrotactile signal; produced when stroking a textured surface with a tool-tip or bare-finger; like other signals contains a great deal of redundant and imperceptible information that can be exploited for efficient compression. This paper presents PVC-SLP, a vibrotactile perceptual coding approach. PVC-SLP employs a model of tactile sensitivity; called ASF (Acceleration Sensitivity Function); for perceptual coding. The ASF is inspired by the four channels model that mediate the perception of vibrotactile stimuli in the glabrous skin. The compression algorithm introduces sparsity constraints in a linear prediction scheme both on the residual and the predictor coefficients. The perceptual quantization of the residual is developed through the use of ASF. The quantization parameters of the residual and the predictor coefficients were jointly optimized; by means of both squared error and perceptual quality measures; to find the sweet spot of the rate-distortion curve. PVC-SLP coding performance is evaluated using two publicly available databases that collectively comprise 1281 vibrotactile signals covering 193 material classes. Furthermore, we compare PVC-SLP with a recent vibrotactile compression method and show that PVC-SLP perceptually outperforms existing method by a sizable margin. Most recently, PVC-SLP has been selected to become part of the haptic codec standard currently under preparation by IEEE P1918.1.1, aka Haptic Codecs for the Tactile Internet.