More information is more information: Working memory and visual context

Abstract

Listeners with cochlear implants demonstrate diminished auditory-verbal working memory capacities, possibly due to a lack of durable codes in the memory buffer. Earlier studies suggest that the context provided by lip-read information should enhance those codes, with both the phonological form and dynamic nature of lip-read signals contributing to this facilitative effect. The purpose of this study was to test the hypothesis that lip-read signals would make uniquely beneficial contributions to the recognition of degraded speech. To test this hypothesis, three kinds of signals were used: unprocessed words, vocoded words, and nonspeech environmental sounds. Two kinds of visual enhancements were applied: (1) dynamic signals specifying the event that generated the signal or (2) pictures representing the object named or creating the signal. Eighty young adults with normal hearing were asked to recall order of eight-item lists in a closed-set format. All listeners heard lists in all three signal conditions (unprocessed, vocoded, environmental sounds) but half recalled order in each visual-enhancement condition. Adding lip-read information improved accuracy and eased cognitive demands for recall of vocoded words, but other visual information provided benefits as well, calling into question previous claims of the specialness of dynamic facial movements.Listeners with cochlear implants demonstrate diminished auditory-verbal working memory capacities, possibly due to a lack of durable codes in the memory buffer. Earlier studies suggest that the context provided by lip-read information should enhance those codes, with both the phonological form and dynamic nature of lip-read signals contributing to this facilitative effect. The purpose of this study was to test the hypothesis that lip-read signals would make uniquely beneficial contributions to the recognition of degraded speech. To test this hypothesis, three kinds of signals were used: unprocessed words, vocoded words, and nonspeech environmental sounds. Two kinds of visual enhancements were applied: (1) dynamic signals specifying the event that generated the signal or (2) pictures representing the object named or creating the signal. Eighty young adults with normal hearing were asked to recall order of eight-item lists in a closed-set format. All listeners heard lists in all three signal conditions (unp...