A word to the eyes: Visual cues benefit lexical segmentation in noise

Abstract

In English, most word-initial syllables are stressed. Listeners use these syllable strength cues to identify word boundaries in degraded acoustic conditions. This is evident in the types of lexical boundary errors they make when tasked with parsing a continuous stream of degraded speech: word boundaries are more often inserted before strong syllables than before weak syllables. Listeners also use visual cues from the talker's face to glean both phonemic and prosodic speech information in degraded listening conditions. While the benefits of lipreading have received much attention, the role of visual cues in lexical segmentation remains largely unexplored. The present study examined the effect of auditory-visual cues on lexical boundary decisions. Normal-hearing listeners identified target phrases degraded by multi-talker babble. Responses in auditory-only and auditory-visual conditions were analyzed for percent words correct and lexical boundary error type. Results indicate large inter-individual variability, but overall an increase in word identification accuracy and a decrease in lexical boundary errors in the auditory-visual condition. Further, some listeners made a greater proportion of lexical boundary insertions before strong syllables, suggesting that the addition of visual cues increased their use of syllable strength to identify word boundaries. Implications for clinical populations will be discussed.