Word Recognition in Speechreading

Abstract

The customary approaches to research on human speechreading are to study phonetic perception with identification of phonemes in nonsense syllables, and to study perception of connected discourse with identification of words in isolated sentences. These two approaches imply a theory that accounts for speechreading in terms of bottom-up phoneme perception and top-down syntactic/semantic processes. However, in auditory spoken language understanding research, word recognition is widely accepted as the critical interface between phonetic input processing and semantic/syntactic processing. In this chapter, a general theoretical approach to word recognition based on auditory speech perception is described, and it is argued that the same model should hold for speechreading. Analyses of two speechreading databases (Bernstein, Demorest, & Tucker, 1995; Demorest, Bernstein, & DeHaven, 1995) were examined for evidence that word recognition does play a critical role in speechreading. The results show high associations between the ability to speechread isolated words and words in sentences, but only low-to-moderate associations between phoneme and word identification (i.e., for words in isolation or in sentences). One implication of these results is that the course of mental events in phoneme identification is at best a subset of those events that result in word recognition. More importantly, the results support the suggestion that word recognition deserves increased attention in efforts to understand speechreading.