Vibrotactile Device Research Articles

Stereo-Vision-Based Human-Computer Interaction with Tactile Stimulation

If a virtual object in a virtual environment represented by a stereo vision system could be touched by a user with some tactile feeling on his/her fingertip, the sense of reality would be heightened. To create a visual impression as if the user were directly pointing to a desired point on a virtual object with his/her own finger, we need to align virtual space coordinates and physical space coordinates. Also, if there is no tactile feeling when the user touches a virtual object, the virtual object would seem to be a ghost. Therefore, a haptic interface device is required to give some tactile sensation to the user. We have constructed such a human-computer interaction system in the form of a simple virtual reality game using a stereo vision system, a vibro-tactile device module, and two position/orientation sensors.

Design and evaluation of a tactile memory game for visually impaired children

Visually impaired people have a lack of proper user interfaces to allow them to easily make use of modern technology. This problem may be solved with multimodal user interfaces that should be designed taking into account the type and degree of disability. The purpose of the study presented in this article was to create usable games for visually impaired children making use of low-cost vibro-tactile devices in multimodal applications. A tactile memory game using multimodal navigation support with high-contrast visual feedback and audio cues was implemented. The game was designed to be played with a tactile gamepad. Different vibrations were to be remembered instead of sounds or embossed pictures that are common in memory games for blind children. The usability and playability of the game was tested with a group of seven 12–13-year-old visually impaired children. The results showed that the game design was successful and a tactile gamepad was usable. The game got a positive response from the focus group.

A pneumatic vibrotactile stimulation device for fMRI.

Mapping the functional response of the somatosensory cortex is useful both for characterizing normal brain activity and for determining the functional integrity of damaged cortex compromised by stroke or other neurological insults. A variety of stimulators have been used to produce somatosensory cortex activation in functional brain imaging, including brushes and swabs operated manually, pneumatically and mechanically powered mechanical vibrators, air puffs, and vibrating ceramic piezoelectric wafers and benders. A closed-system, pneumatically driven rubber diaphragm is reported that overcomes many of the limitations of existing vibrotactile devices and produces robust sensory cortex activation in an fMRI experiment.

Tactile stimulus predictability modulates activity in a tactile-motor cortical network.

Manipulating objects in the hand requires the continuous transformation of sensory input into appropriate motor behaviour. Using a novel vibrotactile device combined with fMRI, the cortical network associated with tactile sensorimotor transformations was investigated. Continuous tactile stimuli were delivered in a random or predictable pattern to the second digit on the right hand of all subjects. To better distinguish sensory and motor processes, subjects were instructed to make proportionate motor gripping responses with their left hand. A consistent cortical network of activation was revealed that included the supplementary motor, dorsal and ventral premotor, posterior parietal, primary and secondary somatosensory and primary motor cortex. Tracking the unpredictable versus predictable tactile stimulus led to greater delays in motor responses and to increased performance errors. Cortical effects due to stimulus predictability were observed in several components of the network, though it was most evident as increased cortical activation in frontal motor regions during tracking of unpredictable tactile stimuli. In contrast to the proposed hypotheses, primary and secondary somatosensory cortices contralateral to tactile input did not reveal enhanced responses during unpredictable tracking. Facilitation during unpredictable tracking was also observed in primary somatosensory cortex contralateral to motor responses, the receptive site for movement-related afference. The present study provides a novel and controlled approach to investigate the loci associated with tactile-motor processing and to measure the task-specific effect of stimulus predictability on network components.

History of bioresponse to vibration in the Acoustical Society of America

Human response to vibratory stimulation of the skin surface has long been considered an aspect of the sense of touch; however, the debate over whether vibration was one aspect of pressure sensation, as espoused by von Frey in the late 1800s, or a separate sense, as argued by Katz (1925), focused attention on this mode of stimulation. Experimental investigations from the 1920s to the 1960s by Knudsen, Geldard, Sherrick, Verrillo, Mountcastle, and others provided basic data on vibrotactile perception and the neural transduction of vibratory stimulation. Within the Acoustical Society of America, work on bioresponse to vibration has included not only basic investigations of vibrotactile perception and physiology, but also studies of the loss of sensitivity resulting from intense hand-arm vibration induced by occupational use of chainsaws and jackhammers, studies of human response to whole-body vibration, and evaluations of the utility of vibrotactile devices for communication of speech to hearing-impaired persons. Contributions in each of these areas, as well as future research directions, are discussed.

New devices to deliver somatosensory stimuli during functional MRI.

A new class of devices are described for improving investigation of somatosensory neuronal activation using fMRI. Dubbed magnetomechanical vibrotactile devices (MVDs), the principle of operation involves driving wire coils with small oscillatory currents in the large static magnetic field inherent to MRI scanners. The resulting Lorentz forces can be oriented to generate large vibrations that are easily converted to translational motions as large as several centimeters. Representative data demonstrate the flexibility of MVDs to generate different well-controlled vibratory and tactile stimuli to activate special proprioceptive and cutaneous somatosensory afferent pathways. The implications of these data are discussed with respect to the literature on existing devices for producing sensorimotor activation, as well as expanding the scope of current fMRI investigations.

Phonetic structure is similar across auditory and vibrotactile speech perception

Vibrotactile vocoders have long been viewed as a possible method for conveying speech information to deaf perceivers. However, modest levels of speech transmission have led to numerous alternative notions concerning limitations of vibrotactile devices. This study investigated whether perceived acoustic phonetic structure is preserved in the transformation of the F2 speech region via a linear vibrotactile vocoder array. In experiment 1, naive hearing subjects discriminated pairs of vibrotactile word stimuli immediately before and after 2 h of practice. At both times, relative vibrotactile discrimination levels were well predicted by identification of phonemes presented by an analogous F2 acoustic vocoder, suggesting that structure is preserved. In experiment 2, subjects separately rated auditory and vibrotactile consonant similarity of stimuli presented by auditory and vibrotactile vocoders, respectively. Analysis of phonological feature information showed similarities and differences in perceptual structure across modalities. Vibrotactile stimuli were rated to be more similar to each other than were auditory stimuli, suggesting that phonetic information was degraded relative to auditory information. Based on these results, current investigations are directed at methods for inducing perceptual learning of vibrotactile phonetic information. [Work supported by NIH/NIDCD 00695.]

Haptic speech perception: Are the possible pay-offs worth the effort?

Episodically throughout the 20th century, practical and scientific inquiries into speech perception by touch have been undertaken. While some extraordinary examples of communication by the natural (uninstrumented) method of Tadoma have suggested that touch has great potential for speech perception, artificial vibrotactile devices have produced relatively modest results. Why at this time, in the light of success (at times verging on spectacular) of auditory implants for people with profound hearing losses, should additional resources be invested in development and testing or sensory aids for speech perception via touch? In answer to this question, this presentation will summarize research on speech perception involving touch, outline important practical and clinical justifications for continued interest, and discuss some theoretical issues in speech perception to which studies involving touch can contribute. [Work supported by NIH DC00695 and DC01577.]

Early vocal development in tactually aided children with severe-profound hearing loss.

The vocalizations of six children with severe-profound hearing loss were audio-recorded in two conditions during individual speech-language intervention sessions: (a) auditory amplification alone, and (b) auditory amplification plus the Tactaid II, a two-channel vibrotactile device (Franklin, 1986). Utterances were categorized according to the infraphonological framework described by Oller (1980, 1986) and Oller and Lynch (1992). Vocalizations were categorized in a developmental framework relative to mature speech. Those utterances containing well-formed consonant-vowel syllables were transcribed with broad phonetic transcription and analyzed at both the syllabic and segmental levels. Statistically significant differences were found between the two conditions for vocal volubility (i.e., quantity of vocalizations produced); subjects vocalized more when using both auditory amplification and the Tactaid II together than with auditory amplification alone. Trends in the early vocal development of these children with severe-profound hearing loss are described at the infraphonologic, segmental, and syllabic levels.

The Effect of a Vibrotactile Device, Somatron , on Physiological and Psychological Responses: Musicians versus Nonmusicians

Thirty musicians and 30 nonmusicians participated in this study. Each subject lay on a Somatron™ (a vibrotactile couch) and was presented with two music conditions (Saint-Saens Organ Symphony, 2nd Movement and Dvorak 9th Symphony, 4th Movement) separated by 1 minute of silence. Additionally, half of the subjects received altered renditions of the music with the tempo gradually increased or decreased throughout each example by 20%. Heart rate was recorded continually across 15-second intervals, and each subject completed a questionnaire concerning affective responses. Affective measures indicated highly positive results. Subjects wanted to repeat the exercise, enjoyed the experience, did not want to get up. and found the experience stimulating as well as relaxing. Physiological data indicated no significant differences attributable to any of the experimental conditions, and no relationship between heart rate and any of the affective verbal reports. Interestingly, no subject within the altered music condition realized that the tempo had been changed. Therefore, an additional study was conducted to determine if musicians who were specifically told, “Something has been changed with the music selections you are going to hear” could isolate the alteration. Again, results indicated that subjects did not perceive the tempo change in either direction.

Evaluation of a wearable multichannel tactile aid

In designing a tactile aid, the most important considerations are that a device provide useful information and that it be wearable. While electrotactile rather than vibrotactile devices are more easily made wearable, results with these devices to date have been disappointing. One exception to this has been the Tickle Talker (Univ. of Melbourne). The Tickle Talker is an eight‐channel device, worn as rings on the fingers of one hand, with a electrode placed on either side of each finger. The processor performs a formant extraction, displaying an estimated fundamental frequency, second formant frequency, and the amplitude of the speech signal. In an attempt to corroborate early promising results [Cowan et al., J. Acoust. Soc. Am. 85, 2593–2607 (1989)], this device was evaluated with three normally hearing adult subjects. In initial testing with closed set tasks done under a tactile aid alone condition, subjects on the average were able to identify the manner of production of stimuli 76% of the time, presence or absence of voicing 74% of the time, and medial vowel 81% of the time. Wearing the tactile aid in conjunction with lipreading raised subjects' scores on a subsequent closed set identification task from 57% lipreading alone to 72% with the device. Results to date with connected discourse tracking also show improvements with the device. [Work supported by NIH.]

Speech Perception with a Single-Channel Cochlear Implant

Suprasegmental and segmental speech perception tasks were administered to 8 patients with single-channel cochlear implants. Suprasegmental tasks included the recognition of syllable number, syllabic stress, and intonation. Segmental tasks included the recognition of vowels and consonants in three modalities: visual only, implant only, and visual + implant. Results were compared to those obtained from artificially deafened adults using a single-channel vibrotactile device. The patterns of responses for both suprasegmental and segmental tasks were highly similar for both groups of subjects, despite differences between the characteristics of the subject samples. These results suggest that single-channel sensory devices, whether they be cochlear implants or vibrotactile aids, produce similar patterns of speech perception errors, even when differences are observed in overall performance level.

Separating signal and noise in vibrotactile devices for the deaf

This paper briefly reviews a series of experiments demonstrating that a sound-to-touch transform of signal and noise may be perceptually separated by providing tactile cues for sound localization. Following this, three experiments are reported in which it is shown that such stereotactile cues are available if two vibrotactile devices (Minifonators, produced by Siemens Co. Ltd.), are used instead of one. With this system, subjects were able to attend to a target signal (male voice) in white noise and when a female voice was presented simultaneously. This ability depended upon a sufficient spatial separation of the target and noise in acoustic space. These findings appear to have applications in the design of tactile hearing aids and cochlear implants when used in noisy environments or when more than one sound is present.

Clinical aspects of vibrotactile devices with deaf children

Longitudinal data will be presented on the speech perception and production abilities of 15 deaf children who use a vibrotactile aid (Tactaid II). Their performance is being compared to that of deaf children who use either a single- or multichannel implant. Initial findings indicate that the performance of the subjects with implants, even those with single-channel devices, is superior to that of the subjects who use the tactile aid. The subjects who achieved the highest scores with the tactile aid tended to have more sophisticated cognitive and linguistic skills that permitted them to develop strategies to decode the vibratory patterns. The importance of prior knowledge of the language in decoding information perceived via the skin is also supported by the data collected from postlingually deaf adults using the same tactile aid. The implications of generalizing data collected on adults to prelingually deaf children will be discussed. [Work supported by NIH.]

Vibrotactile Stimulation: An Educational Program for Spatial Concept Development

The potential of vibrotactile stimulation to enrich the perceptual world of congenitally and early blinded children was evaluated through an educational program developed to introduce spatial concepts typically associated with visual perception. The vibrotactile device used was a modified Optacon of Telesensory Systems, Incorporated. The program was evaluated using 30 blind children, ranging in age from 8 to 14 years, who possessed no more than minimal light perception. It has been suggested that vibrotactile stimulation is perceptually informative and could serve congenitally and early blinded children as a potential basis for an extended understanding of three-dimensional space.

Lipreading with single‐channel vibrotactile presentation of voice fundamental frequency

A study evaluated several transformations of voice fundamental frequency (F0) for use by a single‐channel vibrotactile device to supplement lipreading. The experimental procedure involved lipreading stimuli from a corpus of over 1500 sentences stored on a video laserdisk [L. E. Bernstein and S. P. Eberhardt, 1986, Johns Hopkins Lipreading Corpus]. There were five experimental conditions. Three transformations of voice F0 were tested: two transformed F0 to a pulse rate code, the third employed two sine waves that were amplitude modulated as a function of F0. Condition 4 provided both F0 and a second stimulator indicating high‐frequency speech energy. Condition 5 was visual alone. Subjects were 15 normal‐hearing adults in a multiple single‐subject design with alternating baseline and test sessions. Analyses of covariance for words correct, words per stimulus sentence, and syllables per sentence showed that all subjects improved in lipreading. Subjects who received a single vibrator for voice F0 lipread more successfully than subjects who did not receive the vibrotactile supplement or who received the dual tactile stimulation.

Judgments of intonation and contrastive stress during lipreading

As part of work to develop vibrotactile devices to convey voice fundamental frequency to hearing-impaired lipreaders, an experiment was conducted to investigate the visibility of contrastive stress and question-versus-statement intonation contours. Four sentences (“We will weigh you,” “We owe you a yoyo,” “Chuck caught two cats,” and “Pat cooked Pete's breakfast”) with contrastive stress on one of the first three words and spoken as either statements or questions by a male and a female were presented from videodisk. Sentences were chosen to minimize indexical or affective information that might be used to judge stress or intonation. Subjects were tested in a six alternative forced-choice procedure with response alternatives labeled “question” or “statement” and stress position “1,” “2,” or “3.” Results suggest that contrastive stress and intonation can be judged visually at levels significantly above chance. Judgments of stress were significantly more accurate than judgments of intonation. Results will be discussed in terms of loglinear models to assess the relative independence of stress versus intonation in visual judgments. [Work supported by NIH.]

Transformations of voice fundamental frequency for a vibrotactile device to aid lipreading

As part of the development of a wearable vibrotactile device to convey voice fundamental frequency F0 to hearing-impaired lipreaders, a study was conducted to compare transforms of F0. Rothenberg and Moliter [J. Acoust. Soc. Am. 66, 1029–1038 (1979)] reported that most errors in a tactile-alone task of judging stress and intonation in sentences were mislocations of the stressed word. We found that contrastive stress can be detected visually. The present experiment compared F0 transforms in a tactual-visual task. Transforms included: (1) direct glottal to tactile pulse; (2) linear transform with lowering of center frequency and shift in range; (3) similar to (2) with log of the normalized glottal period; (4) similar to (2) with the difference of logs of the current period and a weighted sum of previous periods; and (5) a constant pulse train during voicing. Subjects received tactile stimulation to the index finger by an AV-6 vibrator, and responses were labeled “question” or “statement” and one of three stress positions. Effects of the transforms and conditions of vision-only versus tactual-visual presentation will be reported. [Work supported by NIH.]

The case for tactile aids

Devices that convert sound information into vibrotactile stimulation have been shown in laboratory and clinical studies to enable persons to appreciate many aspects of the acoustic environment, and thus are of potential benefit to deaf persons. These vibrotactile devices provide information comparable to that obtained with surgical cochlear implants, but appear to have some advantages, particularly in use with children. These advantages include no invasive surgery, the ability to accommodate growth of the user, the potential to use the device for a trial period to assess its usefulness, the possibility of shared use by more than one person, and considerably lower cost. At CID, a number of vibrotactile aids ranging from single-channel commercially available aids to multichannel tactile vocoders are being evaluated. Results from studies in our own and other laboratories will be used to address such issues as the features of the acoustic environment that can or should be transmitted through a tactile aid, the differences in the information transmitted by commercially available versus experimental laboratory devices, and the technical problems that remain to be overcome in the design and evaluation of improved vibrotactile aids. [Work supported by NIH and NSF.]

Clinical results with a hearing aid and a single-channel vibrotactile device for profoundly deaf adults.

The speech perception of a group of 19 adults with post-lingual profound to total hearing loss was tested with nine closed-set speech tests without lipreading, two open-set tests without lipreading and two open-set speech tests with lipreading. The subjects were all prospective cochlear implant patients participating in a clinical trial of the implant and the results reported here were obtained as part of the pre-operative assessment. They were divided into groups on the basis of their prior experience with the aid(s), their speech detection thresholds with the two aids and their personal preferences. Seven of the subjects used a hand-held single-channel vibrotactile device and the other 12 used a powerful conventional hearing aid. Subjects from each group scored significantly better than chance on the closed set tests without lipreading. Training or regular hearing aid use was correlated with good performance on the closed-set tests. No subject showed a significant improvement of the lipreading score when the aid was used as a supplement. The use of sophisticated wearable tactile devices and extensive training may allow a better result, but in this clinical program, neither a hearing aid nor a single-channel vibrotactile device greatly benefited the postlingually profoundly deaf adults.