Vibrotactile music systems for co-located and telematic performance

Abstract

Building on previous findings, the project reported here expands on the idea of how the modality of touch contributes to the sensory perception of sound in the presence of vibro-tactile events. The SenseAble1.0, a vibrotactile interface, was developed to transform the auditory parameter space into one that is adequate for haptic perception by the means of eight individually controllable actuators and machine learning algorithms. Due to the process of sensory substitution, people with hearing impairments can use the vibrotactile display to “listen” to music and perceive exterior acoustic stimuli through the sense of touch. A psychophysical pilot study investigated (i) frequency discrimination and pitch direction as well as (ii) interval size recognition of the haptic modality, and (iii) adaptation effects of the somatosensory system. Live performances using the SenseAble as a vibrotactile music system illustrate to what extent musical communication between ensemble members and a hearing-impaired musician i...