Emotional information affects fission illusion induced by audio-visual interactions

Yasuhiro Takeshima

doi:10.1038/s41598-020-57719-y

Abstract

Multisensory integration is affected by various types of information coming from different sensory stimuli. It has been suggested that emotional information also influences the multisensory integration process. The perceptual phenomena induced by audio-visual integration are modulated by emotional signals through changing individuals’ emotional states. However, the direct effects of emotional information, without changing emotional states on the multisensory integration process have not yet been examined. The present study investigated the effects of an emotional signal on audio-visual integration. The experiments compared the magnitude of audio-visual fission and fusion illusions using facial expression stimuli and simple geometric shapes. Facial expression stimuli altered the criterion difference for discerning the number of flashes when two beeps were simultaneously presented in Experiment 1. These stimuli did not affect the fission illusion’s magnitude. For simple geometric shapes, emotional shapes perceptually induced a larger fission illusion in Experiment 2. The present study found that the emotional valence included in simple geometric shapes induced a larger fission illusion. Moreover, current results suggest that emotional faces modulate response criterion for fission illusion in discernment of the number of flashes. Future studies should elucidate in detail the mechanism of emotional valence effects on audio-visual integration.

Highlights

Multisensory integration is affected by various types of information coming from different sensory stimuli
Several fMRI studies have reported that the activation of primary visual cortex (V1) during the fission illusion is similar to the response elicited by the presentation of two physical flashes and the response during the fusion illusion is similar to that elicited by the presentation of one physical flash20,21
Visual stimulus was presented below the fixation point

Summary

Introduction

Multisensory integration is affected by various types of information coming from different sensory stimuli. By using these types of emotional stimuli, the present study attempted to widely examine the effects of emotional information with task-relevant stimuli on the audio-visual integration process that occur during fission and fusion illusions. Three types of simple geometric shapes (upward-pointing triangle, downward-pointing triangle, and circle) were presented instead of facial expression stimuli in Experiment 2.

Results

Conclusion