Studying possible influences of situational, stimulus driven and pre- stimulus brain state factors on multisensory interaction in the double flash illusion

Abstract

Multisensory integration is essential for our perception of the world. Multisensory illusion provide a unique opportunity to study the integration process. In particular, the double-flash paradigm – in which a single flash, paired with two auditory beeps causes perception of an illusory second flash – has often been used as a test bed for multisensory integration. The research reported here examines the effect of situational, stimulus driven and pre-stimulus brain state influence on the phenomenon to cast light on (i) whether we can have control over the integration (ii) what mechanisms account for the dominance of one modality (in this case auditory) and can the dominance change (iii) do ongoing changes in brain activity not related to the stimuli predict the rate of the illusion. Chapter 3 tests whether impairment of cognitive influences through working memory load alters illusion rates. In chapter 4 the reliability of the auditory stimuli were reduced to see if modality dominance might switch. Chapter 5 examines whether incentives for correct responses using paid feedback can alter illusion rates. Working memory load had no effect on rates of illusion. Paid feedback did, however, produce a resistance to the illusion. The resistance appeared to be learned. When auditory reliability was reduced, an auditory illusion was detected, alongside the standard visual illusion. EEG data for the visual illusion with near threshold sound was consistent with previous literature for full intensity sound. For the auditory illusion, EEG interaction was found to occur later than in the standard paradigm. These findings suggest that, firstly, multisensory interaction takes place early in sensory processing, too early to be affected by top down attentional processes. However, it does not appear to be immune to adaption or learning, as evidenced by the reduction in illusions and change in multisensory ERP components when participants are paid for correct answers. Secondly, reduction in auditory reliability does not appear to change the visual illusion in terms of illusion rate or ERP profile. It does however produce a simultaneous auditory illusion with apparently different neural mechanisms which occur later. Finally, ongoing changes in pre-stimulus activity appear to be associated with perception of the illusion.