Binaural Enhancement Research Articles

Multilingual non-intrusive binaural intelligibility prediction based on phone classification

Speech intelligibility (SI) prediction models are a valuable tool for the development of speech processing algorithms for hearing aids or consumer electronics. For the use in realistic environments it is desirable that the SI model is non-intrusive (does not require separate input of original and degraded speech, transcripts or a-priori knowledge about the signals) and does a binaural processing of the audio signals. Most of the existing SI models do not fulfill all of these criteria. In this study, we propose an SI model based on phone probabilities obtained from a deep neural net. The model comprises a binaural enhancement stage for prediction of the speech recognition threshold (SRT) in realistic acoustic scenes. In the first part of the study, SRT predictions in different spatial configurations are compared to the results from normal-hearing listeners. On average, our approach produces lower errors and higher correlations compared to three intrusive baseline models. In the second part, we explore if measures relevant in spatial hearing, i.e., the intelligibility level difference (ILD) and the binaural ILD (BILD), can be predicted with our modeling approach. We also investigate if a language mismatch between training and testing the model plays a role when predicting ILD and BILD. This point is especially important for low-resource languages, where not thousands of hours of language material are available for training. Binaural benefits are predicted by our model with an error of 1.5 dB. This is slightly higher than the error with a competitive baseline MBSTOI (1.1 dB), but does not require separate input of original and degraded speech. We also find that good binaural predictions can be obtained with models that are not specifically trained with the target language.

Optimized Binaural Enhancement via attention masking network-based speech separation framework in digital hearing aids

Earlier studies have utilized microphone signal processing for performing the target speech evaluation and separation-like feature recognition by involving a huge amount of training data along with supervised machine learning. These approaches are most appropriate for stationary noise suppression; however, it is challenging for non-stationary noise, and it does not satisfy the practical processing requirement. To overcome the former issue, a system is presented for “joint speaker separation, and noise suppression”, referred to as the Optimized Binaural Enhancement via Attention Masking Network (OBEAMNET). In this paper, a speech separation model for a hearing aid is suggested. Here, the noise speech is separated into two different speeches through the proposed OBEAMNET approach. Initially, the standard input audio signals are gathered from standard benchmark datasets. Further, the cepstral features are gathered from the input data for getting the essential information. A novel Fitness Ordered Black Widow Optimization (FO-BWO) algorithm is developed for optimizing the features to increase the PSNR as well as decrease the RMSE. The feature selection is carried out by FO-BWO to get the most noteworthy information from audio signals. The suggested FO-BWO, along with the OBEAMNET technique, is used for efficiently separating the signals. Finally, the optimal features are fed to the speech separation using the OBEAMNET approach for improving the effectiveness of the proposed model that is observed in decoding auditory attention in noisy practical environments.

Speaker separation in realistic noise environments with applications to a cognitively-controlled hearing aid

Future wearable technology may provide for enhanced communication in noisy environments and for the ability to pick out a single talker of interest in a crowded room simply by the listener shifting their attentional focus. Such a system relies on two components, speaker separation and decoding the listener’s attention to acoustic streams in the environment. To address the former, we present a system for joint speaker separation and noise suppression, referred to as the Binaural Enhancement via Attention Masking Network (BEAMNET). The BEAMNET system is an end-to-end neural network architecture based on self-attention. Binaural input waveforms are mapped to a joint embedding space via a learned encoder, and separate multiplicative masking mechanisms are included for noise suppression and speaker separation. Pairs of output binaural waveforms are then synthesized using learned decoders, each capturing a separated speaker while maintaining spatial cues. A key contribution of BEAMNET is that the architecture contains a separation path, an enhancement path, and an autoencoder path. This paper proposes a novel loss function which simultaneously trains these paths, so that disabling the masking mechanisms during inference causes BEAMNET to reconstruct the input speech signals. This allows dynamic control of the level of suppression applied by BEAMNET via a minimum gain level, which is not possible in other state-of-the-art approaches to end-to-end speaker separation. This paper also proposes a perceptually-motivated waveform distance measure. Using objective speech quality metrics, the proposed system is demonstrated to perform well at separating two equal-energy talkers, even in high levels of background noise. Subjective testing shows an improvement in speech intelligibility across a range of noise levels, for signals with artificially added head-related transfer functions and background noise. Finally, when used as part of an auditory attention decoder (AAD) system using existing electroencephalogram (EEG) data, BEAMNET is found to maintain the decoding accuracy achieved with ideal speaker separation, even in severe acoustic conditions. These results suggest that this enhancement system is highly effective at decoding auditory attention in realistic noise environments, and could possibly lead to improved speech perception in a cognitively controlled hearing aid.

Evaluation of a method for enhancing interaural level differences at low frequencies.

A method (called binaural enhancement) for enhancing interaural level differences at low frequencies, based on estimates of interaural time differences, was developed and evaluated. Five conditions were compared, all using simulated hearing-aid processing: (1) Linear amplification with frequency-response shaping; (2) binaural enhancement combined with linear amplification and frequency-response shaping; (3) slow-acting four-channel amplitude compression with independent compression at the two ears (AGC4CH); (4) binaural enhancement combined with four-channel compression (BE-AGC4CH); and (5) four-channel compression but with the compression gains synchronized across ears. Ten hearing-impaired listeners were tested, and gains and compression ratios for each listener were set to match targets prescribed by the CAM2 fitting method. Stimuli were presented via headphones, using virtualization methods to simulate listening in a moderately reverberant room. The intelligibility of speech at ±60° azimuth in the presence of competing speech on the opposite side of the head at ±60° azimuth was not affected by the binaural enhancement processing. Sound localization was significantly better for condition BE-AGC4CH than for condition AGC4CH for a sentence, but not for broadband noise, lowpass noise, or lowpass amplitude-modulated noise. The results suggest that the binaural enhancement processing can improve localization for sounds with distinct envelope fluctuations.

Parameter-based binaural hearing aid algorithms to improve speech intelligibility and localization in complex environments.

This paper presents new binaural enhancement and noise suppression algorithms for binaural hearing aids. To enhance interaural level difference (ILD) cues at low frequencies, which are usually small, interaural time difference (ITD) cues are estimated and transformed to ILDs. The binaural noise suppression algorithm consists of adaptive beamforming and a coherence-based suppression filter. The estimated phase and signal-to-noise ratio (SNR) at each hearing aid are used to perform the processing. The performance of the proposed methods was assessed using perceptual evaluation with hearing-impaired listeners and objective evaluation.

Functional Properties of Human Auditory Cortical Fields

While auditory cortex in non-human primates has been subdivided into multiple functionally specialized auditory cortical fields (ACFs), the boundaries and functional specialization of human ACFs have not been defined. In the current study, we evaluated whether a widely accepted primate model of auditory cortex could explain regional tuning properties of fMRI activations on the cortical surface to attended and non-attended tones of different frequency, location, and intensity. The limits of auditory cortex were defined by voxels that showed significant activations to non-attended sounds. Three centrally located fields with mirror-symmetric tonotopic organization were identified and assigned to the three core fields of the primate model while surrounding activations were assigned to belt fields following procedures similar to those used in macaque fMRI studies. The functional properties of core, medial belt, and lateral belt field groups were then analyzed. Field groups were distinguished by tonotopic organization, frequency selectivity, intensity sensitivity, contralaterality, binaural enhancement, attentional modulation, and hemispheric asymmetry. In general, core fields showed greater sensitivity to sound properties than did belt fields, while belt fields showed greater attentional modulation than core fields. Significant distinctions in intensity sensitivity and contralaterality were seen between adjacent core fields A1 and R, while multiple differences in tuning properties were evident at boundaries between adjacent core and belt fields. The reliable differences in functional properties between fields and field groups suggest that the basic primate pattern of auditory cortex organization is preserved in humans. A comparison of the sizes of functionally defined ACFs in humans and macaques reveals a significant relative expansion in human lateral belt fields implicated in the processing of speech.

An Electrophysiological Measure of Binaural Hearing in Noise

A common complaint of patients with (central) auditory processing disorder is difficulty understanding speech in noise. Because binaural hearing improves speech understanding in compromised listening situations, quantifying this ability in different levels of noise may yield a measure with high clinical utility. To examine binaural enhancement (BE) and binaural interaction (BI) in different levels of noise for the auditory brainstem response (ABR) and middle latency response (MLR) in a normal hearing population. An experimental study in which subjects were exposed to a repeated measures design. Fifteen normal hearing female adults served as subjects. Normal hearing was assessed by pure-tone audiometry and otoacoustic emissions. All subjects were exposed to 0, 20, and 35 dB effective masking (EM) of white noise during monotic and diotic click stimulation. ABR and MLR responses were simultaneously acquired. Peak amplitudes and latencies were recorded and compared across conditions using a repeated measures analysis of variance (ANOVA). For BE, ABR results showed enhancement at 0 and 20 dB EM, but not at 35 dB EM. The MLR showed BE at all noise levels, but the degree of BE decreased with increasing noise level. For BI, both the ABR and MLR showed BI at all noise levels. However, the degree of BI again decreased with increasing noise level for the MLR. The results demonstrate the ability to measure BE simultaneously in the ABR and MLR in up to 20 dB of EM noise and BI in up to 35 dB EM of noise. Results also suggest that ABR neural generators may respond to noise differently than MLR generators.

Binaural enhancement of speech intelligibility metrics

Efficient methods of quantifying speech intelligibility are needed for designing and understanding functional architectural spaces. All current measures of speech intelligibility are based on monaural impulse response, which excludes consideration of important binaural aspects of human hearing, including dereverberation and decoloration. Acquisition of binaural data has increasingly become common practice, using dummy heads or in‐ear microphones, making the development of binaural intelligibility measures especially timely. The need for such a measure is illustrated by calculations of speech transmission index (STI) with a single energy‐based impulse obtained through combination of binaural data channels. Each method of combination produces significantly different STI values that illustrate the impact of orientation and location on intelligibility calculations. The relationship between such estimation variation and subjective experience must be studied to determine the research direction for a much needed...

The effect of broad-band noise on the binaural interaction components of human auditory brainstem-evoked potentials.

Three-channel Lissajous trajectories (3-CLTs) of binaural interaction components (BI) of auditory brainstem potentials (ABEPs) were derived from 13 normally hearing adults by subtracting the response to binaural clicks from the algebraic sum of monaurally evoked responses to clicks. ABEPs were recorded in response to 65 dB nHL, alternating-polarity clicks, presented at a rate of 11/s. The procedure was repeated with clicks alone as well as with clicks with broad-band masking noise. Noise was presented at 25 and 45 dB nHL, producing a signal-to-noise ratio of +40 and +20 dB, respectively. All BI 3-CLTs included 6 planar segments (labeled BdI, BdII, BdIII, BeI, BeII and Bf) whose apex latencies, except Bf, increased with increasing noise level above 25 dB nHL, and whose durations, sizes, shapes and orientations did not change across noise levels. There were also significant increases in peak latencies of the BI from single channels vertex-mastoid and vertex-neck with increasing noise level. No significant change was found in the trajectory amplitude of apices, with the exception of apices BdIII and Bf whose amplitudes increased with increasing noise level. We suggest that the paradoxical increase in BI amplitude with masking noise may reflect a binaural enhancement of the effect of noise. The effects observed indicate that, whereas the response to clicks displays occlusion, the response to noise displays spatial facilitation at the brainstem level.

Stethoscope having pseudostereophonic binaural enhancement

First Page

Binaural and Monaural Speech Discrimination Under Reverberation

This study investigated the effects of reverberation upon the speech discrimination performance of 30 normally hearing subjects and 30 persons with bilateral sensorineural hearing loss. Following preliminary tests, the Modified Rhyme Test was administered monaurally and binaurally at reverberation times of 0, 1, 2 and 3 sec. All stimuli were administered under earphones. The lists and conditions were appropriately randomized and counterbalanced so that each subject heard a test list at each reverberation time (RT) monaurally and binaurally. The expected gain due to binaural summation of loudness was simulated in order to test the possibility that any binaural enhancement of intelligibility might be due to binaural loudness summation. For all conditions, the normally hearing subjects performed significantly better than the hearing-impaired group. However, in relative terms, the two groups were remarkably similar. Binaural scores were significantly higher than monaural scores at each RT for both groups (in spite of homophasic conditions). For both groups, monaural and binaural scores decreased with increasing RT. Monaural scores decreased at a significantly faster rate with RT than did the binaural scores (the binaural advantage, however, was larger for the normal group). Increasing the monaural presentation levels to simulate the binaural loudness gain did not result in higher scores. It was concluded that speech discrimination under reverberation is better binaurally than monaurally for both normally hearing and hearing-impaired persons. This is due, at least partly, to the ability of the binaural auditory system to squelch the effects of reverberation. A tentative model is suggested for the binaural squelch of reverberation. The current findings are compared with existing data; and suggestions are offered for clinical application.