Visual speech benefit provided by realistic sentences in noise

Abstract

Understanding speech in background noise presents a significant challenge to listeners with hearing loss. Even though hearing aids are very successful in restoring listening in quieter conditions, current devices provide only limited benefit in noise. While seeing a talker can improve intelligibility, visual benefit has typically been quantified using rather unrealistic audio-visual speech materials. Here, we investigated visual benefit using realistic effortful speech from the ECO-SiN corpus. Audio-only, audio-visual, and visual-only sentences were presented in three simulated real-world environments at their realistic sound levels in a 3-D loudspeaker array equipped with integrated high-resolution video projectors. Ten young normal-hearing (NH) listeners as well as 18 older listeners with sensorineural hearing loss participated in the experiment with and without their own hearing aids. While the NH listeners rapidly hit ceiling, listeners with hearing loss struggled to understand the audio-only sentences in the louder background noise—even when aided. There was, however, demonstrable visual benefit of up to 16% in the audio-visual condition for both groups, which is significantly less than commonly reported. Future research will explore if the individual outcomes of the audio-visual ECO-SIN test better reflects real-world hearing experience and how far this is maintained when using virtual reality glasses.