New haptic systems for elicit emotions in audio-visual events for hearing impaired people.

Abstract

In this research we have developed devices for hearing impaired people that can transmit the emotions evoked by music. In this way, we will all achieve the full inclusion of impairment people in all social activities. As this is an issue that goes beyond pure engineering research, it has been necessary to create a multidisciplinary group within the Spanish Centre for Captioning and Audio Description (CESyA), in direct collaboration with the Display and Photonic Applications Research Group (GDAF) and the Human Language and Accessibility Technologies Group (HULAT) of the Carlos III University (UC3M) and the Psychiatry and Applied Communication Departments of the Complutense University of Madrid. For this research, a pair of haptic gloves were developed, which transmit different characteristics extracted from music by means of vibrational sequences, intended to drive an emotional reinforcement. Another device, a “music box”, a vibration box, was also developed from the idea of using vibration as an instrument.Overall, the acceptance of the glove was positive, with an 80% of users accepting the negative video, repeatedly stating in the surveys that they "felt more when the glove vibrated". The music box was also well received. Volunteers felt that the system gave them a pleasant sensation and that they could feel the music. However, some participants associated the vibration with the sensation of an alarm clock, which was expectedly negative.Considering the obtained EEG results, we can conclude that the integration of simple multimodal stimuli, visual and vibrotactile, helps to enhance the activation of emotional processes linked with musical stimulation. It can also be concluded that multimodal visual and vibrotactile stimulation enhances both attentional and emotional processes. These results pave the way for future developments that will allow a hearing-impaired person to fully enjoy an audiovisual event, which will have a great impact on this group.