A visually guided beamformer to aid listening in complex acoustic environments

Abstract

The purpose of this paper is to provide an overview of recent work on the visually guided hearing aid (VGHA; Kidd et al. JASA 133, EL202). The latest prototype VGHA consists of a lightweight, head-mounted, 18-microphone array combined with PC-based software to create a highly-tuned acoustic beamformer steered in real time by an integrated binocular eye tracker. Recent work on the two main functional sub-components of the VGHA (beamforming and eye-gaze control) focused on the benefits that may be obtained by listeners with hearing loss (and in some cases by listeners with normal hearing) attempting to understand speech masked by one or more competing sounds (such as Gaussian noise or additional speakers). These benefits include improved signal-to-noise ratio under different masked conditions and faster steering (compared to fixed directional amplification steered by head turns). The VGHA may be simulated over headphones by PC-based implementations using HRTFs and an optional external eye-tracker. This has allowed testing a number of variations of beamformer properties and the evaluation of multiple types of sound processing strategies including the incorporation of natural binaural cues. Overall, the VGHA holds considerable promise for improving selective listening in a variety of complex listening conditions.