Audiovisual Temporal Processing in Postlingually Deafened Adults with Cochlear Implants

Iliza M Butera,Brannon D Mangus,René H Gifford,Ryan A Stevenson,Mark T Wallace,Tiffany G Woynaroski

doi:10.1038/s41598-018-29598-x

Abstract

For many cochlear implant (CI) users, visual cues are vitally important for interpreting the impoverished auditory speech information that an implant conveys. Although the temporal relationship between auditory and visual stimuli is crucial for how this information is integrated, audiovisual temporal processing in CI users is poorly understood. In this study, we tested unisensory (auditory alone, visual alone) and multisensory (audiovisual) temporal processing in postlingually deafened CI users (n = 48) and normal-hearing controls (n = 54) using simultaneity judgment (SJ) and temporal order judgment (TOJ) tasks. We varied the timing onsets between the auditory and visual components of either a syllable/viseme or a simple flash/beep pairing, and participants indicated either which stimulus appeared first (TOJ) or if the pair occurred simultaneously (SJ). Results indicate that temporal binding windows—the interval within which stimuli are likely to be perceptually ‘bound’—are not significantly different between groups for either speech or non-speech stimuli. However, the point of subjective simultaneity for speech was less visually leading in CI users, who interestingly, also had improved visual-only TOJ thresholds. Further signal detection analysis suggests that this SJ shift may be due to greater visual bias within the CI group, perhaps reflecting heightened attentional allocation to visual cues.

Highlights

Speech is typically an audiovisual (AV) experience wherein coincident orofacial articulations can considerably boost perceptual accuracy over that observed with auditory-alone stimulation[4]
Group differences in age and nonverbal IQ prompted us to explore these indices as covariates; these factors were retained in all models wherein they accounted for significant variance
A key finding in this study is a shift in the point of subjective simultaneity (PSS) for making temporal judgments regarding audiovisual speech in postlingually deafened adults with cochlear implant (CI) compared to NH controls

Summary

Introduction

Speech is typically an audiovisual (AV) experience wherein coincident orofacial articulations can considerably boost perceptual accuracy over that observed with auditory-alone stimulation[4]. A great deal of modeling work suggests that ambiguous information stemming from unreliable sensory estimates is optimally integrated in the brain by weighting the relative reliability of the different sources of sensory evidence[5,6,7] This process results in a more robust multisensory percept with specific advantages including increased stimulus saliency[8], decreased detection thresholds[9,10], reduced reaction times[11], and enhanced efficiency in neural processing[12]. We expected this result to be specific for speech stimuli and not for simple flashbeep stimuli on account of the greater ecological validity of speech signals We drew this prediction in part from prior work investigating the maturation of temporal binding windows in normal development[32], and reasoned that reduced auditory experience during deafness might result in less mature (i.e., broader and more symmetric) temporal binding windows to be evident well into adulthood for CI users

Methods

Results

Conclusion