Preservation Of Binaural Cues Research Articles

Robust parameter strategy for Wiener-based binaural noise reduction methods in hearing aids

This work presents a method for designing the weighting parameter required for Wiener-based binaural noise reduction methods. This parameter establishes the desired trade-off between noise reduction and binaural cue preservation in hearing aids. The proposed strategy was specially derived for the preservation of the interaural level difference, interaural time difference and interaural coherence binaural cues. It is defined as a function of the average input noise power at the microphones, providing robustness against the influence of joint changes in noise and speech power (Lombard effect), as well as signal-to-noise ratio variations. A theoretical framework based on the mathematical definition of the degree of homogeneity is presented and applied to a generic augmented Wiener-based cost function. The theoretical insights obtained are supported by computational simulations and psychoacoustic experiments using the multichannel Wiener filter with a noise interaural transfer function preservation technique (MWF-ITFv), as a case study. Statistical analysis indicates that, compared to a purely fixed weighting parameter, the proposed structure for the weighting parameter and the design method of its fixed part provide significant robustness against changes in the noise binaural cues at the cost of a small noise reduction performance decrease and speech binaural cue distortion.

A Parametric Unconstrained Beamformer Based Binaural Noise Reduction for Assistive Hearing

For hearing-impaired listeners, it is required not only to enhance the target speech by suppressing ambient noises, but also to preserve the binaural cues of important directional sources, such that a complete spatial awareness of the acoustic scene is obtained. It was shown that the binaural minimum variance distortionless response (BMVDR) filter enables a joint enhancement and binaural cue preservation of the target source. By adding more linear constraints associated with the acoustic transfer functions of noise sources to BMVDR, the binaural linearly constrained minimum variance (BLCMV) filter is obtained, which can further preserve the spatial cues of interfering sources at the cost of sacrificing the noise reduction performance. Both BMVDR and BLCMV beamformers enforce a <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">hard decision</i> on the spatial awareness. In this paper, we therefore propose a generalization for BMVDR and BLCMV, namely parametric unconstrained binaural (PUB) beamformer, which can achieve a controllable <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">soft decision</i> on noise reduction and binaural cues preservation. The involved parameters are chosen to control the distortion level of the target source and the preservation of interfering sources, respectively. Theoretical analysis on the performance comparison shows that the conventional BMVDR and BLCMV beamformers are special cases of the PUB beamformer. Experiments using both synthetic data and real audio measurements validate the superiority of the proposed method.

A Class of Pareto Optimal Binaural Beamformers

The objective of binaural multi-microphone speech enhancement algorithms can be viewed as a multi-criteria design problem as there are several requirements to be met. The objective is not only to extract the target speaker without distortion, but also to suppress interfering sources (e.g., competing speakers) and ambient background noise, while preserving the auditory impression of the complete acoustic scene. Such a multi-objective problem (MOP) can be solved using a Pareto frontier, which provides a useful trade-off between the different criteria. In this paper, we propose a unified Pareto optimization framework, which is achieved by defining a generalized mean squared error (MSE) cost function, derived from a MOP. The solution to the multi-criteria problem is grounded on a solid mathematical foundation. The MSE cost function consists of a weighted sum of speech distortion (SD), partial interference reduction (IR), and partial noise reduction (NR) terms with scaling parameters that control the amount of IR and NR. The filter minimizing this generalized cost function, denoted Pareto optimal binaural multichannel Wiener filter (Pareto-BMWF), constitutes a generalization of various binaural MWF-based and binaural MVDR-based beamformers. This solution is optimal for any set of parameters. The improved speech enhancement capabilities are experimentally demonstrated using real-signal recordings when estimation errors are present and the binaural cue preservation capabilities are analyzed.

Performance Analysis of the Extended Binaural MVDR Beamformer With Partial Noise Estimation

Besides reducing undesired noise sources and limiting speech distortion, another important objective of a binaural noise reduction algorithm is the preservation of the binaural cues of all sound sources in the acoustic scene. In this paper, we consider the binaural minimum variance distortionless response beamformer with partial noise estimation (BMVDR-N), which allows to trade off between noise reduction performance and binaural cue preservation of the noise component by mixing the output signals of the BMVDR beamformer with the noisy reference microphone signals. For a directional noise source, it has been shown that incorporating an external microphone in addition to the head-mounted microphones enables both the noise reduction performance as well as the interaural time and level difference cues of the noise component to be improved in the output signals. In this paper, we consider an arbitrary noise field and analytically show that incorporating an external microphone in the BMVDR-N beamformer enables 1) a larger output signal-to-noise ratio (SNR) for the same mixing parameter, 2) the same output SNR for a larger mixing parameter, and 3) the same desired output magnitude squared coherence (MSC) of the noise component for a smaller mixing parameter to be obtained. The derived analytical expressions are firstly validated using simulated anechoic acoustic transfer functions, where the listener’s head is modelled as a rigid sphere. Experimental results using recorded signals for a binaural hearing device setup in a reverberant environment also show that in a realistic scenario incorporating an external microphone in the BMVDR-N beamformer significantly improves the output SNR and reduces the mixing parameter that is required to obtain a desired output MSC of the noise component compared to using only the head-mounted microphones.

Binaural LCMV Beamforming With Partial Noise Estimation

Besides reducing undesired sources (interfering sources and background noise), another important objective of a binaural beamforming algorithm is to preserve the spatial impression of the acoustic scene, which can be achieved by preserving the binaural cues of all sound sources. While the binaural minimum variance distortionless response (BMVDR) beamformer provides a good noise reduction performance and preserves the binaural cues of the desired source, it does not allow to control the reduction of the interfering sources and distorts the binaural cues of the interfering sources and the background noise. Hence, several extensions have been proposed. First, the binaural linearly constrained minimum variance (BLCMV) beamformer uses additional constraints, enabling to control the reduction of the interfering sources while preserving their binaural cues. Second, the BMVDR with partial noise estimation (BMVDR-N) mixes the output signals of the BMVDR with the noisy reference microphone signals, enabling to control the binaural cues of the background noise. Merging the advantages of both extensions, in this paper we propose the BLCMV with partial noise estimation (BLCMV-N). We show that the output signals of the BLCMV-N can be interpreted as a mixture of the noisy reference microphone signals and the output signals of a BLCMV using an adjusted interference scaling parameter. We provide a theoretical comparison between the BMVDR, the BLCMV, the BMVDR-N and the proposed BLCMV-N in terms of noise and interference reduction performance and binaural cue preservation. Experimental results using recorded signals as well as the results of a perceptual listening test show that the BLCMV-N is able to preserve the binaural cues of an interfering source (like the BLCMV), while enabling to trade off between noise reduction performance and binaural cue preservation of the background noise (like the BMVDR-N).

Binaural Codebook-Based Speech Enhancement With Atomic Speech Presence Probability

In this work, we present a universal codebook-based speech enhancement framework that relies on a single codebook to encode both speech and noise components. The atomic speech presence probability (ASPP) is defined as the probability that a given codebook atom encodes speech at a given point in time. We develop ASPP estimators based on binaural cues including the interaural phase and level difference (IPD and ILD), the interaural coherence magnitude (ICM), as well as a combined version leveraging the full interaural transfer function (ITF). We evaluate the performance of the resulting ASPP-based speech enhancement algorithms on binaural mixtures of reverberant speech and real-world noise. The proposed approach improves both objective speech quality and intelligibility over a wide range of input SNR, as measured with PESQ and binaural STOI metrics, outperforming two binaural speech enhancement benchmark methods. We show that the proposed ITF-based ASPP approach achieves a good balance of the trade-off between binaural noise reduction and binaural cue preservation.

A Robust Target Linearly Constrained Minimum Variance Beamformer With Spatial Cues Preservation for Binaural Hearing Aids

In this paper, a binaural beamforming algorithm for hearing aid applications is introduced.The beamforming algorithm is designed to be robust to some error in the estimate of the target speaker direction. The algorithm has two main components: a robust target linearly constrained minimum variance (TLCMV) algorithm based on imposing two constraints around the estimated direction of the target signal, and a post-processor to help with the preservation of binaural cues. The robust TLCMV provides a good level of noise reduction and low level of target distortion under realistic conditions. The post-processor enhances the beamformer abilities to preserve the binaural cues for both diffuse-like background noise and directional interferers (competing speakers), while keeping a good level of noise reduction. The introduced algorithm does not require knowledge or estimation of the directional interferers' directions nor the second-order statistics of noise-only components. The introduced algorithm requires an estimate of the target speaker direction, but it is designed to be robust to some deviation from the estimated direction. Compared with recently proposed state-of-the-art methods, comprehensive evaluations are performed under complex realistic acoustic scenarios generated in both anechoic and mildly reverberant environments, considering a mismatch between estimated and true sources direction of arrival. Mismatch between the anechoic propagation models used for the design of the beamformers and the mildly reverberant propagation models used to generate the simulated directional signals is also considered. The results illustrate the robustness of the proposed algorithm to such mismatches.

A Convex Approximation of the Relaxed Binaural Beamforming Optimization Problem

The recently proposed relaxed binaural beamforming (RBB) optimization problem provides a flexible tradeoff between noise suppression and binaural-cue preservation of the sound sources in the acoustic scene. It minimizes the output noise power, under the constraints, which guarantee that the target remains unchanged after processing and the binaural-cue distortions of the acoustic sources will be less than a user-defined threshold. However, the RBB problem is a computationally demanding non convex optimization problem. The only existing suboptimal method which approximately solves the RBB is a successive convex optimization (SCO) method which, typically, requires to solve multiple convex optimization problems per frequency bin, in order to converge. Convergence is achieved when all constraints of the RBB optimization problem are satisfied. In this paper, we propose a semidefinite convex relaxation (SDCR) of the RBB optimization problem. The proposed suboptimal SDCR method solves a single convex optimization problem per frequency bin, resulting in a much lower computational complexity than the SCO method. Unlike the SCO method, the SDCR method does not guarantee user-controlled upper-bounded binaural-cue distortions. To tackle this problem, we also propose a suboptimal hybrid method that combines the SDCR and SCO methods. Instrumental measures combined with a listening test show that the SDCR and hybrid methods achieve significantly lower computational complexity than the SCO method, and in most cases better tradeoff between predicted intelligibility and binaural-cue preservation than the SCO method.

Influence of Multi-microphone Signal Enhancement Algorithms on the Acoustics and Detectability of Angular and Radial Source Movements.

Hearing-impaired listeners are known to have difficulties not only with understanding speech in noise but also with judging source distance and movement, and these deficits are related to perceived handicap. It is possible that the perception of spatially dynamic sounds can be improved with hearing aids (HAs), but so far this has not been investigated. In a previous study, older hearing-impaired listeners showed poorer detectability for virtual left-right (angular) and near-far (radial) source movements due to lateral interfering sounds and reverberation, respectively. In the current study, potential ways of improving these deficits with HAs were explored. Using stimuli very similar to before, detailed acoustic analyses were carried out to examine the influence of different HA algorithms for suppressing noise and reverberation on the acoustic cues previously shown to be associated with source movement detectability. For an algorithm that combined unilateral directional microphones with binaural coherence-based noise reduction and for a bilateral beamformer with binaural cue preservation, movement-induced changes in spectral coloration, signal-to-noise ratio, and direct-to-reverberant energy ratio were greater compared with no HA processing. To evaluate these two algorithms perceptually, aided measurements of angular and radial source movement detectability were performed with 20 older hearing-impaired listeners. The analyses showed that, in the presence of concurrent interfering sounds and reverberation, the bilateral beamformer could restore source movement detectability in both spatial dimensions, whereas the other algorithm only improved detectability in the near-far dimension. Together, these results provide a basis for improving the detectability of spatially dynamic sounds with HAs.

Speech reception with different bilateral directional processing schemes: Influence of binaural hearing, audiometric asymmetry, and acoustic scenario

Hearing aid (HA) users can differ markedly in their benefit from directional processing (or beamforming) algorithms. The current study therefore investigated candidacy for different bilateral directional processing schemes. Groups of elderly listeners with symmetric (N = 20) or asymmetric (N = 19) hearing thresholds for frequencies below 2 kHz, a large spread in the binaural intelligibility level difference (BILD), and no difference in age, overall degree of hearing loss, or performance on a measure of selective attention took part. Aided speech reception was measured using virtual acoustics together with a simulation of a linked pair of completely occluding behind-the-ear HAs. Five processing schemes and three acoustic scenarios were used. The processing schemes differed in the tradeoff between signal-to-noise ratio (SNR) improvement and binaural cue preservation. The acoustic scenarios consisted of a frontal target talker presented against two speech maskers from ±60° azimuth or spatially diffuse cafeteria noise. For both groups, a significant interaction between BILD, processing scheme, and acoustic scenario was found. This interaction implied that, in situations with lateral speech maskers, HA users with BILDs larger than about 2 dB profited more from preserved low-frequency binaural cues than from greater SNR improvement, whereas for smaller BILDs the opposite was true. Audiometric asymmetry reduced the influence of binaural hearing. In spatially diffuse noise, the maximal SNR improvement was generally beneficial. N0Sπ detection performance at 500 Hz predicted the benefit from low-frequency binaural cues. Together, these findings provide a basis for adapting bilateral directional processing to individual and situational influences. Further research is needed to investigate their generalizability to more realistic HA conditions (e.g., with low-frequency vent-transmitted sound).

Efficacy of Directional Microphones in Hearing Aids Equipped with Wireless Synchronization Technology.

This study aimed to evaluate the use of directionality in hearing aids with wireless synchronization on localization and speech intelligibility in noise. This study included 25 individuals with bilateral mild to moderate flat sensorineural hearing loss. For the localization experiment, eight loudspeakers (Genelec 8020B) arranged in a circle covering a 0-360° angle and the Cubase 6 software were used for presenting the stimulus. A car horn of 260 ms was presented from these loudspeakers, one at a time, randomly. The listener was instructed to point to the direction of the source. The degree of the localization error was obtained with and without directionality and wireless synchronization options. For speech perception in a noise experiment, signal to noise ratio-50 (SNR-50) was obtained using sentences played through a speaker at a fixed angle of 0°. A calibrated eight-talker speech babble was used as noise and the babble was routed either through 0°, 90°, 270° (through one speaker at a time) or through both 90° and 270° speakers. The results revealed that the conditions where both the wireless synchronization and directionality were activated resulted in a significantly better performance in both localization and speech perception in noise tasks. It can be concluded that the directionality in the wireless synchronization hearing aids coordinates with each other binaurally for better preservation of binaural cues, thus reducing the localization errors and improving speech perception in noise. The results of this study could be used to counsel and justify the selection of the directional wireless synchronization hearing aids.

Relaxed Binaural LCMV Beamforming

In this paper we propose a new binaural beamforming technique which can be seen as a relaxation of the linearly constrained minimum variance (LCMV) framework. The proposed method can achieve simultaneous noise reduction and exact binaural cue preservation of the target source, similar to the binaural minimum variance distortionless response (BMVDR) method. However, unlike BMVDR, the proposed method is also able to preserve the binaural cues of multiple interferers to a certain predefined accuracy. Specifically, it is able to control the trade-off between noise reduction and binaural cue preservation of the interferers by using a separate trade-off parameter per interferer. Moreover, we provide a robust way of selecting these trade-off parameters in such a way that the preservation accuracy for the binaural cues of the interferers is always better than the corresponding ones of the BMVDR. The relaxation of the constraints in the proposed method achieves approximate binaural cue preservation of more interferers than other previously presented LCMV-based binaural beamforming methods that use strict equality constraints.

Theoretical Analysis of Linearly Constrained Multi-Channel Wiener Filtering Algorithms for Combined Noise Reduction and Binaural Cue Preservation in Binaural Hearing Aids

Besides noise reduction, an important objective of binaural speech enhancement algorithms is the preservation of the binaural cues of all sound sources. For the desired speech source and the interfering sources, e.g., competing speakers, this can be achieved by preserving their relative transfer functions (RTFs). It has been shown that the binaural multi-channel Wiener filter (MWF) preserves the RTF of the desired speech source, but typically distorts the RTF of the interfering sources. To this end, in this paper we propose two extensions of the binaural MWF, i.e., the binaural MWF with RTF preservation (MWF-RTF) aiming to preserve the RTF of the interfering source and the binaural MWF with interference rejection (MWF-IR) aiming to completely suppress the interfering source. Analytical expressions for the performance of the binaural MWF, MWF-RTF and MWF-IR in terms of noise reduction, speech distortion and binaural cue preservation are derived, showing that the proposed extensions yield a better performance in terms of the signal-to-interference ratio and preservation of the binaural cues of the directional interference, while the overall noise reduction performance is degraded compared to the binaural MWF. Simulation results using binaural behind-the-ear impulse responses measured in a reverberant environment validate the derived analytical expressions for the theoretically achievable performance of the binaural MWF, MWF-RTF, and MWF-IR, showing that the performance highly depends on the position of the interfering source and the number of microphones. Furthermore, the simulation results show that the MWF-RTF yields a very similar overall noise reduction performance as the binaural MWF, while preserving the binaural cues of both the speech and the interfering source.

Interaural Coherence Preservation in Multi-Channel Wiener Filtering-Based Noise Reduction for Binaural Hearing Aids

Besides noise reduction an important objective of binaural speech enhancement algorithms is the preservation of the binaural cues of all sound sources. To this end, an extension of the binaural multi-channel Wiener filter (MWF), namely the MWF-ITF, has been proposed, which aims to preserve the Interaural Transfer Function (ITF) of the noise sources. However, the MWF-ITF is well-suited only for directional noise sources but not for, e.g., spatially isotropic noise, whose spatial characteristics cannot be properly described by the ITF but rather by the Interaural Coherence (IC). Hence, another extension of the binaural MWF, namely the MWF-IC, has been recently proposed, which aims to preserve the IC of the noise component. Since for the MWF-IC a substantial tradeoff between noise reduction and IC preservation exists, in this paper we propose a perceptually constrained version of the MWF-IC, where the amount of IC preservation is controlled based on the IC discrimination ability of the human auditory system. In addition, a theoretical analysis of the binaural cue preservation capabilities of the binaural MWF and the MWF-ITF for spatially isotropic noise fields is provided. Several simulations in diffuse noise scenarios show that the perceptually constrained MWF-IC yields a controllable preservation of the IC without significantly degrading the output SNR compared to the binaural MWF and the MWF-ITF. Furthermore, contrary to the binaural MWF and MWF-ITF, the proposed algorithm retains the spatial separation between the output speech and noise components while the binaural cues of the speech component are only slightly distorted, such that the binaural hearing advantage for speech intelligibility can still be exploited.

Sound Processing for Better Coding of Monaural and Binaural Cues in Auditory Prostheses

Despite many considerable technical advances in the field of hearing aids and cochlear implants, people using auditory prostheses still have major problems with speech understanding in the presence of interfering sounds and with directional hearing. Both abilities are dependent on sound stream segregation in real-world listening environments. In this paper, two timely and important issues related to sound stream segregation in auditory prostheses are addressed, namely, the coding of monaural and binaural cues. Several state-of-the-art signal processing algorithms used in cochlear implants (CIs) and in hearing aids (HAs) are introduced. A review is given of some recent proposals to improve temporal coding in monaural CIs, and of recent work to improve the transmission of binaural cues in both HAs, CIs, and combined acoustic and electric hearing (bimodal hearing). The ultimate aim is to improve speech and music perception, and, additionally, the preservation of binaural cues to preserve directional hearing.

Two-stage binaural speech enhancement with Wiener filter for high-quality speech communication

Speech enhancement has been researched extensively for many years to provide high-quality speech communication in the presence of background noise and concurrent interference signals. Human listening is robust against these acoustic interferences using only two ears, but state-of-the-art two-channel algorithms function poorly. Motivated by psychoacoustic studies of binaural hearing (equalization–cancellation (EC) theory), in this paper, we propose a two-stage binaural speech enhancement with Wiener filter (TS-BASE/WF) approach that is a two-input two-output system. In this proposed TS-BASE/WF, interference signals are first estimated by equalizing and cancelling the target signal in a way inspired by the EC theory, a time-variant Wiener filter is then applied to enhance the target signal given the noisy mixture signals. The main advantages of the proposed TS-BASE/WF are (1) effectiveness in dealing with non-stationary multiple-source interference signals, and (2) success in preserving binaural cues after processing. These advantages were confirmed according to the comprehensive objective and subjective evaluations in different acoustical spatial configurations in terms of speech enhancement and binaural cue preservation.

Theoretical Analysis of Binaural Multimicrophone Noise Reduction Techniques

Binaural hearing aids use microphone signals from both left and right hearing aid to generate an output signal for each ear. The microphone signals can be processed by a procedure based on speech distortion weighted multichannel Wiener filtering (SDW-MWF) to achieve significant noise reduction in a speech + noise scenario. In binaural procedures, it is also desirable to preserve binaural cues, in particular the interaural time difference (ITD) and interaural level difference (ILD), which are used to localize sounds. It has been shown in previous work that the binaural SDW-MWF procedure only preserves these binaural cues for the desired speech source, but distorts the noise binaural cues. Two extensions of the binaural SDW-MWF have therefore been proposed to improve the binaural cue preservation, namely the MWF with partial noise estimation (MWF-eta) and MWF with interaural transfer function extension (MWF-ITF). In this paper, the binaural cue preservation of these extensions is analyzed theoretically and tested based on objective performance measures. Both extensions are able to preserve binaural cues for the speech and noise sources, while still achieving significant noise reduction performance.

Role of binaural hearing in speech intelligibility and spatial release from masking using vocoded speech

A cochlear implant vocoder was used to evaluate relative contributions of spectral and binaural temporal fine-structure cues to speech intelligibility. In Study I, stimuli were vocoded, and then convolved through head related transfer functions (HRTFs) to remove speech temporal fine structure but preserve the binaural temporal fine-structure cues. In Study II, the order of processing was reversed to remove both speech and binaural temporal fine-structure cues. Speech reception thresholds (SRTs) were measured adaptively in quiet, and with interfering speech, for unprocessed and vocoded speech (16, 8, and 4 frequency bands), under binaural or monaural (right-ear) conditions. Under binaural conditions, as the number of bands decreased, SRTs increased. With decreasing number of frequency bands, greater benefit from spatial separation of target and interferer was observed, especially in the 8-band condition. The present results demonstrate a strong role of the binaural cues in spectrally degraded speech, when the target and interfering speech are more likely to be confused. The nearly normal binaural benefits under present simulation conditions and the lack of order of processing effect further suggest that preservation of binaural cues is likely to improve performance in bilaterally implanted recipients.

Speech enhancement with multichannel Wiener filter techniques in multimicrophone binaural hearing aids

This paper evaluates speech enhancement in binaural multimicrophone hearing aids by noise reduction algorithms based on the multichannel Wiener filter (MWF) and the MWF with partial noise estimate (MWF-N). Both algorithms are specifically developed to combine noise reduction with the preservation of binaural cues. Objective and perceptual evaluations were performed with different speech-in-multitalker-babble configurations in two different acoustic environments. The main conclusions are as follows: (a) A bilateral MWF with perfect voice activity detection equals or outperforms a bilateral adaptive directional microphone in terms of speech enhancement while preserving the binaural cues of the speech component. (b) A significant gain in speech enhancement is found when transmitting one contralateral microphone signal to the MWF active at the ipsilateral hearing aid. Adding a second contralateral microphone showed a significant improvement during the objective evaluations but not in the subset of scenarios tested during the perceptual evaluations. (c) Adding the partial noise estimate to the MWF, done to improve the spatial awareness of the hearing aid user, reduces the amount of speech enhancement in a limited way. In some conditions the MWF-N even outperformed the MWF possibly due to an improved spatial release from masking.