Auditory–visual integration during nonconscious perception

Abstract

Our study proposes a test of a key assumption of the most prominent model of consciousness – the global workspace (GWS) model (e.g., Baars, 2002, 2005, 2007; Dehaene & Naccache, 2001; Mudrik, Faivre, & Koch, 2014). This assumption is that multimodal integration requires consciousness; however, few studies have explicitly tested if integration can occur between nonconscious information from different modalities. The proposed study examined whether a classic indicator of multimodal integration – the McGurk effect – can be elicited with subliminal auditory–visual speech stimuli. We used a masked speech priming paradigm developed by Kouider and Dupoux (2005) in conjunction with continuous flash suppression (CFS; Tsuchiya & Koch, 2005), a binocular rivalry technique for presenting video stimuli subliminally. Applying these techniques together, we carried out two experiments in which participants categorised auditory syllable targets which were preceded by subliminal auditory–visual (AV) speech primes. Subliminal AV primes were either illusion-inducing (McGurk) or illusion-neutral (Incongruent) combinations of speech stimuli. In Experiment 1, the categorisation of the syllable target (“pa”) was facilitated by the same syllable prime when it was part of a McGurk combination (auditory “pa” and visual “ka”) but not when part of an Incongruent combination (auditory “pa” and visual “wa”). This dependency on specific AV combinations indicated a nonconscious AV interaction. Experiment 2 presented a different syllable target (“ta”) which matched the predicted illusory outcome of the McGurk combination – here, both the McGurk combination (auditory “pa” and visual “ka”) and the Incongruent combination (auditory “ta” and visual “ka”) failed to facilitate target categorisation. The combined results of both Experiments demonstrate a type of nonconscious multimodal interaction that is distinct from integration – it allows unimodal information that is compatible for integration (i.e., McGurk combinations) to persist and influence later processes, but does not actually combine and alter that information. As the GWS model does not account for non-integrative multimodal interactions, this places some pressure on such models of consciousness.