Unconscious imitation as a metric for crossmodal influences: Visibility of the mouth can enhance alignment to audiovisual speech.

Abstract

Human perceivers unconsciously imitate the behavioral characteristics of other humans (e.g., Iacoboni, 2009). Imitated characteristics include body postures (Condon & Ogston, 1967), facial expressions (Hale & Burgoon, 1984), and subtle acoustical characteristics of perceived speech, a phenomenon known as speech alignment (e.g. Goldinger, 1998). Recent evidence shows crossmodal influences on speech alignment (Dias & Rosenblum, 2011, in review; Miller, Sanchez, & Rosenblum, 2010). Our laboratory has shown that integrated audio and visual (lipread) information for a talker’s utterance can influence the nuances of perceivers’ articulations of that same utterance. Further, these changes in perceivers’ articulations seem more sensitive to crossmodal influences than identification responses would reveal. One question raised from these results concerns the facial information that influences alignment to auditory speech. In the current investigation, 10 female undergraduates shadowed (said out-loud) 120 audiovisual words uttered by a talker, presented in auditory noise at a level where all words were easily identifiable (+10 SNR). The shadowed tokens provided either full view of the face, or the face with the mouth area blurred out. Each participant shadowed clear and blurred tokens in block order, counterbalanced between groups. When shadowing blurred stimuli prior to clear stimuli, the degree of speech alignment is enhanced when visibility of the mouth is available, MD = .027, SE = .015, t(24) = 1.854, p <.05, r = .354. No such advantage was found when shadowing clear stimuli prior to shadowing blurred stimuli, MD = -.019, SE = .016, t(24) = -1.201, p = .241, r = .238. The results suggest alignment to auditory speech can be crossmodally enhanced by visibility of the mouth, even when words are easily identifiable. These results support the possibly that in some contexts, inadvertent imitation can provide a more sensitive measure of crossmodal influences than simple identification responses. Meeting abstract presented at VSS 2013