Evaluating the Effort Expended to Understand Speech in Noise Using a Dual-Task Paradigm: The Effects of Providing Visual Speech Cues

Abstract

Using a dual-task paradigm, 2 experiments (Experiments 1 and 2) were conducted to assess differences in the amount of listening effort expended to understand speech in noise in audiovisual (AV) and audio-only (A-only) modalities. Experiment 1 had equivalent noise levels in both modalities, and Experiment 2 equated speech recognition performance levels by increasing the noise in the AV versus A-only modality. Sixty adults were randomly assigned to Experiment 1 or Experiment 2. Participants performed speech and tactile recognition tasks separately (single task) and concurrently (dual task). The speech tasks were performed in both modalities. Accuracy and reaction time data were collected as well as ratings of perceived accuracy and effort. In Experiment 1, the AV modality speech recognition was rated as less effortful, and accuracy scores were higher than A only. In Experiment 2, reaction times were slower, tactile task performance was poorer, and listening effort increased, in the AV versus the A-only modality. At equivalent noise levels, speech recognition performance was enhanced and subjectively less effortful in the AV than A-only modality. At equivalent accuracy levels, the dual-task performance decrements (for both tasks) suggest that the noisier AV modality was more effortful than the A-only modality.