Abstract

Incongruent audiovisual speech stimuli can lead to perceptual illusions such as fusions or combinations. Here, we investigated the underlying audiovisual integration process by measuring ERPs. We observed that visual speech‐induced suppression of P2 amplitude (which is generally taken as a measure of audiovisual integration) for fusions was similar to suppression obtained with fully congruent stimuli, whereas P2 suppression for combinations was larger. We argue that these effects arise because the phonetic incongruency is solved differently for both types of stimuli.

Highlights

  • When a speech sound (A, for auditory speech) is accompanied by the speaker’s articulatory gestures (V, for visual speech), the listener’s brain integrates the unimodal signals

  • We sought to determine whether the electrophysiological correlates of AV integration at the N1/P2 are different for McGurk fusions and combinations

  • V-induced suppression of the N1/P2 is generally interpreted as an effect of AV integration, and from that perspective, it is evident that A and V are integrated in both types of McGurk stimuli

Read more

Summary

Introduction

When a speech sound (A, for auditory speech) is accompanied by the speaker’s articulatory gestures (V, for visual speech), the listener’s brain integrates the unimodal signals. Tuomainen et al, 2005; Saint-Amour et al, 2007; Alsius et al, 2014), but can lead to percepts that are different from either A or V This is evident from a highly influential paper by McGurk & MacDonald (1976) who showed that seeing ‘g’ while the actual speech sound is a ‘b’ (i.e. AbVg) may yield illusory ‘d’ percepts. Green et al, 1991; Sekiyama & Tohkura, 1991; van Wassenhove et al, 2005; Schwartz, 2010; van Wassenhove, 2013; Tiippana, 2014), as the brain solves the phonetic AV conflict by fusing the place of articulation cues Such fusions do not always occur; changing the modality of the conflicting consonants can produce a combination percept in Received 8 May 2017, revised 27 September 2017, accepted 27 September 2017. We explored whether the electrophysiological correlates of AV integration are different for McGurk fusion and combination stimuli

Methods
Results
Conclusion

Talk to us

Join us for a 30 min session where you can share your feedback and ask us any queries you have

Schedule a call

Disclaimer: All third-party content on this website/platform is and will remain the property of their respective owners and is provided on "as is" basis without any warranties, express or implied. Use of third-party content does not indicate any affiliation, sponsorship with or endorsement by them. Any references to third-party content is to identify the corresponding services and shall be considered fair use under The CopyrightLaw.