Binaural Hearing Aids Research Articles

Spatially Correct Rate-Constrained Noise Reduction for Binaural Hearing Aids in Wireless Acoustic Sensor Networks

Compared to monaural hearing aids (HAs), binaural hearing aid systems, in which there is a communication link between the two devices, have improved noise reduction capabilities and the ability to preserve binaural spatial information. However, the limited HA battery lifetime puts constraints on the amount of information that can be shared between the two devices. In other words, the rate of transmission between the devices is an important constraint that needs to be considered, while preserving the spatial information. In this article, a linearly constrained noise reduction problem is proposed, which jointly finds the optimal rate allocation and the optimal estimation (beamforming) weights across all sensors and frequencies, while preserving the binaural spatial cues of point sources. The proposed method considers a rate constraint together with linear constraints to preserve the binaural spatial cues of point sources. Minimizing the mean square error on the estimated target speech at the left and the right side beamformers, the optimal weights are found to be rate-constrained linearly constrained minimum variance (LCMV) filters, and the optimal rates are found to be the solutions to a set of reverse water filling problems. The performance of the proposed method is evaluated using the averaged binaural signal-to-noise ratio (SNR), the interaural level difference (ILD) error and the interaural time difference (ITD) error. The results show that the proposed method outperforms spatially correct noise reduction approaches that use naive/random rate allocation strategies.

A Smart Binaural Hearing Aid Architecture Leveraging a Smartphone APP With Deep-Learning Speech Enhancement

This paper presents a smartphone-based binaural hearing aid architecture for improving the speech intelligibility of hearing aid users. The proposed system consists of an earpiece, a smartphone and an application that performs real-time speech enhancement. The speaker’s voice, which is picked up by the microphone of the earpiece that is worn on the ear, is transmitted to the smartphone via wireless technology. After the speech intelligibility is improved in real time by the deep learning speech enhancement application, it is returned to the earpiece and generates sound. Deep learning speech enhancement algorithms can be used without performing burdensome calculations on the processors in the hearing aid. The results showed that the average usage of the central processing unit in the smartphone was approximately 26%, and the signal-to-noise ratios improve by at least 20%. The presented objective and subjective results show that the proposed method achieves comparatively more noise suppression without distorting the audio.

Binaural Direct-to-Reverberant Energy Ratio and Speaker Distance Estimation

This article addresses the problem of distance estimation using binaural hearing aid microphones in reverberant rooms. Among several distance indicators, the direct-to-reverberant energy ratio (DRR) has been shown to be more effective than other features. Therefore, we present two novel approaches to estimate the DRR of binaural signals. The first method is based on the interaural magnitude-squared coherence whereas the second approach uses stochastic maximum likelihood beamforming to estimate the power of the direct and reverberant components. The proposed DRR estimation algorithms are integrated into a distance estimation technique. When based solely on DRR, the distance estimation algorithm requires calibration where naturally the critical distance is a good calibration point. We thus propose two approaches for the calibration of the distance estimation algorithm: Informed calibration using the critical distance of the reverberant room and blind calibration using the listener's own voice. Results across various acoustical environments show the benefit of the proposed algorithms for the estimation of sound source distances up to 3 m with an estimation error of about 35 cm using informed calibration and about 1 m using the fully blind calibration strategy.

The effects of wireless audio microphones on sentences recognition threshold for hearing aid users in noise

Objective:This study was aimed to observe the effects of wireless audio microphone on hearing aids effect in noise at different listening distances. Method:Twenty-three subjects with bilateral sensorineural hearing loss, including 17 males and 6 females were fitted with binaural hearing aids. These patients did sentences recognition tests at two different listening distance（1.5 and 3 meters） in noise. The subjects were tested under three conditions, ①with hearing aids alone; ②with the wireless audio microphone alone; ③with hearing aid microphone and mini audio microphone simultaneously. Result:The sentence recognition threshold at 3 meters listening distance is significantly higher than it at 1.5 meters listening distance with hearing aids alone（P<0.05）. There is no significant difference in the sentence recognition threshold between two listening distances when the wireless audio microphone was switched on（P>0.05）. Conclusion:Mini audio microphone can significantly improve hearing aids effect in long distance listening in noise.

A high-fidelity multi-channel portable platform for development of novel algorithms for assistive listening wearables

The NIDCD has funded a number of projects to develop portable signal processing tools that enable real-time processing of the acoustic environment. The overarching goal is to provide a large group of researchers with the means to efficiently develop and evaluate novel signal processing schemes, individualized fitting procedures, and technical solutions and services for hearing apparatus such as hearing aids and assistive listening devices. We report here on a development done in the SBIR Phase II Project R44DC016247. This project builds on the software being concurrently developed in R01DC015429 to provide a complete portable and wearable software-hardware master hearing aid device needed for development of new solutions for assisted hearing. We will present and demonstrate the portable platform, currently in the Beta launch, that consists of a Cortex A8 based processing unit and a codec set able to support hearing aid architecture of up to 6 microphones and 4 speakers. It is currently accompanied by a binaural 2-microphone BTE hearing aid set, but will also support different headset form-factors of our partners. Additionally it features stereo line in and line out connections. The device can be remotely controlled with a smart phone.

A Robust Target Linearly Constrained Minimum Variance Beamformer With Spatial Cues Preservation for Binaural Hearing Aids

In this paper, a binaural beamforming algorithm for hearing aid applications is introduced.The beamforming algorithm is designed to be robust to some error in the estimate of the target speaker direction. The algorithm has two main components: a robust target linearly constrained minimum variance (TLCMV) algorithm based on imposing two constraints around the estimated direction of the target signal, and a post-processor to help with the preservation of binaural cues. The robust TLCMV provides a good level of noise reduction and low level of target distortion under realistic conditions. The post-processor enhances the beamformer abilities to preserve the binaural cues for both diffuse-like background noise and directional interferers (competing speakers), while keeping a good level of noise reduction. The introduced algorithm does not require knowledge or estimation of the directional interferers' directions nor the second-order statistics of noise-only components. The introduced algorithm requires an estimate of the target speaker direction, but it is designed to be robust to some deviation from the estimated direction. Compared with recently proposed state-of-the-art methods, comprehensive evaluations are performed under complex realistic acoustic scenarios generated in both anechoic and mildly reverberant environments, considering a mismatch between estimated and true sources direction of arrival. Mismatch between the anechoic propagation models used for the design of the beamformers and the mildly reverberant propagation models used to generate the simulated directional signals is also considered. The results illustrate the robustness of the proposed algorithm to such mismatches.

CABE: A Cloud-Based Acoustic Beamforming Emulator for FPGA-Based Sound Source Localization.

Microphone arrays are gaining in popularity thanks to the availability of low-cost microphones. Applications including sonar, binaural hearing aid devices, acoustic indoor localization techniques and speech recognition are proposed by several research groups and companies. In most of the available implementations, the microphones utilized are assumed to offer an ideal response in a given frequency domain. Several toolboxes and software can be used to obtain a theoretical response of a microphone array with a given beamforming algorithm. However, a tool facilitating the design of a microphone array taking into account the non-ideal characteristics could not be found. Moreover, generating packages facilitating the implementation on Field Programmable Gate Arrays has, to our knowledge, not been carried out yet. Visualizing the responses in 2D and 3D also poses an engineering challenge. To alleviate these shortcomings, a scalable Cloud-based Acoustic Beamforming Emulator (CABE) is proposed. The non-ideal characteristics of microphones are considered during the computations and results are validated with acoustic data captured from microphones. It is also possible to generate hardware description language packages containing delay tables facilitating the implementation of Delay-and-Sum beamformers in embedded hardware. Truncation error analysis can also be carried out for fixed-point signal processing. The effects of disabling a given group of microphones within the microphone array can also be calculated. Results and packages can be visualized with a dedicated client application. Users can create and configure several parameters of an emulation, including sound source placement, the shape of the microphone array and the required signal processing flow. Depending on the user configuration, 2D and 3D graphs showing the beamforming results, waterfall diagrams and performance metrics can be generated by the client application. The emulations are also validated with captured data from existing microphone arrays.

Personalized signal-independent beamforming for binaural hearing aids.

The effect of personalized microphone array calibration on the performance of hearing aid beamformers under noisy reverberant conditions is studied. The study makes use of a new, publicly available, database containing acoustic transfer function measurements from 29 loudspeakers arranged on a sphere to a pair of behind-the-ear hearing aids in a listening room when worn by 27 males, 14 females, and 4 mannequins. Bilateral and binaural beamformers are designed using each participant's hearing aid head-related impulse responses (HAHRIRs). The performance of these personalized beamformers is compared to that of mismatched beamformers, where the HAHRIR used for the design does not belong to the individual for whom performance is measured. The case where the mismatched HAHRIR is that of a mannequin is of particular interest since it represents current practice in commercially available hearing aids. The benefit of personalized beamforming is assessed using an intrusive binaural speech intelligibility metric and in a matrix speech intelligibility test. For binaural beamforming, both measures demonstrate a statistically signficant (p < 0.05) benefit of personalization. The benefit varies substantially between individuals with some predicted to benefit by as much as 1.5 dB.

Perceptually Relevant Preservation of Interaural Time Differences in Binaural Hearing Aids

This paper presents a noise reduction method with perceptually relevant preservation of the interaural time difference (ITD) of the residual noise in binaural hearing aids. The interaural coherence (IC) concept, previously applied to the multichannel Wiener filter (MWF) for preservation of the spatial subjective sensation of diffuse noise fields, is proposed here to both preserve and emphasize the ITD binaural cues of a directional acoustic noise source. It is demonstrated that the previously developed MWF-ITD technique may decrease the original IC magnitude of the processed noise, consequently increasing the variance of the interaural phase difference (IPD) of the output signals. It is shown that the MWF-IC technique simultaneously minimizes a nonlinear function of the difference between input and output IPD, which is strictly related to ITD, and preserves the natural coherence of directional noise captured by the reference microphones. Objective measures and psychoacoustic experiments corroborate the theoretical findings, showing that the MWF-IC technique provides relevant noise reduction, while preserving the original ITD subjective perception and original lateralization for a directional noise source. These results are especially relevant for hearing aid designers, since they indicate MWF-IC as a noise reduction technique that provides residual noise spatial preservation for both diffuse and directional noise sources in frequencies below 1.5 kHz.

Assessment of a feasible virtual acoustics method for testing hearing aids using the Hearing-Aid Speech Perception Index

Significant differences have been found in hearing aid (HA) performance between laboratory and real world test environments. Virtual sound environments provide a degree of control and reproducibility which is lacking in real world testing but may require an impractical number of loudspeakers. We assessed the accuracy of a simulation approach in which sources’ direct sound is delivered by single loudspeakers while room acoustics are reproduced using low-order Ambisonics and a small number of loudspeakers. In a large office, we recorded binaural hearing aid output in response to sentence targets and babble noise presented at various levels and from various combinations of four loudspeakers surrounding a manikin. We measured the loudspeakers’ room impulse responses (IRs) using a 32-channel spherical microphone array (Eigenmike), and split the IRs into "direct sound" and "room sound" portions. In an anechoic chamber, the original acoustics were simulated using Ambisonics or discrete loudspeakers for each source’s direct portion and Ambisonics for the room portion. Ambisonic order and/or number of playback loudspeakers were also varied. HA output in the simulations was recorded using the manikin and assessed by comparing Hearing-Aid Speech Perception Index (HASPI) values computed on the simulation recordings with those made in the original room.

Asymmetric Coding for Rate-Constrained Noise Reduction in Binaural Hearing Aids

Binaural hearing aids (HAs) can potentially perform advanced noise reduction algorithms, leading to an improvement over monaural/bilateral HAs. Due to the limited transmission capacities between the HAs and given knowledge of the complete joint noisy signal statistics, the optimal rate-constrained beamforming strategy is known from the literature. However, as these joint statistics are unknown in practice, sub-optimal strategies have been presented. In this paper, we present a unified framework to study the performance of these existing optimal and sub-optimal rate-constrained beamforming methods for binaural HAs. Moreover, we propose to use an asymmetric sequential coding scheme to estimate the joint statistics between the microphones in the two HAs. We show that under certain assumptions, this leads to sub-optimal performance in one HA but allows to obtain the truly optimal performance in the second HA. Based on the mean square error distortion measure, we evaluate the performance improvement between monaural beamforming (no communication) and the proposed scheme, as well as the optimal and the existing sub-optimal strategies in terms of the information bit-rate. The results show that the proposed method outperforms existing practical approaches in most scenarios, especially at middle rates and high rates, without having the prior knowledge of the joint statistics.

Model-Based Speech Enhancement for Intelligibility Improvement in Binaural Hearing Aids

Speech intelligibility is often severely degraded among hearing impaired individuals in situations such as the cocktail party scenario. The performance of the current hearing aid technology has been observed to be limited in these scenarios. In this paper, we propose a binaural speech enhancement framework that takes into consideration the speech production model. The enhancement framework proposed here is based on the Kalman filter that allows us to take the speech production dynamics into account during the enhancement process. The usage of a Kalman filter requires the estimation of clean speech and noise short term predictor (STP) parameters, and the clean speech pitch parameters. In this work, a binaural codebook-based method is proposed for estimating the STP parameters, and a directional pitch estimator based on the harmonic model and maximum likelihood principle is used to estimate the pitch parameters. The proposed method for estimating the STP and pitch parameters jointly uses the information from left and right ears, leading to a more robust estimation of the filter parameters. Objective measures such as PESQ and STOI have been used to evaluate the enhancement framework in different acoustic scenarios representative of the cocktail party scenario. We have also conducted subjective listening tests on a set of nine normal hearing subjects, to evaluate the performance in terms of intelligibility and quality improvement. The listening tests show that the proposed algorithm, even with access to only a single channel noisy observation, significantly improves the overall speech quality, and the speech intelligibility by up to $\text{15}{\%}$ .

Beamforming Designs Robust to Propagation Model Estimation Errors for Binaural Hearing Aids

This work introduces new binaural beamforming algorithms for hearing aids, with a good robustness to errors in the estimated target source propagation model. Two different binaural beamforming designs are proposed. One design is based on an adaptive null positioning scheme, and the second is based on a combination of the adaptive null positioning scheme and wider beampatterns. Simulations are performed using signals and propagation models obtained from multichannel binaural hearing aids recordings, including some in a mildly reverberant environment. Evaluations are done in terms of noise reduction and target distortion. Evaluation results illustrate the robustness of the two proposed designs to errors between the true and estimated directions of arrival for the target source, and to mismatch between the anechoic propagation models used for the beamformers designs and the reverberant propagation models used to generate the signals at the sensors or beamformer inputs. Both designs surpass the performance of standard binaural Minimum Variance Distortionless Response (MVDR) and binaural Generalized-Side Lobe Canceler (GSC) beamformers.

Wireless binaural hearing aid technology for telephone use and listening in wind noise

Objective: To assess the speech perception benefits of binaural streaming technology for bilateral hearing aid users in two difficult listening conditions.Design: Two studies were conducted to compare hearing aid processing features relating to telephone use and wind noise. Speech perception testing was conducted in four different experimental conditions in each study.Study sample: Ten bilaterally-aided children in each study.Results: Significant improvements in speech perception were obtained with a wireless feature for telephone use. Significant speech perception benefits were also obtained with wireless hearing aid features when listening to speech in simulated wind noise.Conclusions: Binaural signal processing algorithms can significantly improve speech perception for bilateral hearing aid users in challenging listening situations.

Low-delay wind noise cancellation with preservation of spatial information for binaural hearing aids

Low-delay wind noise cancellation with preservation of spatial information for binaural hearing aids

Effect of Binaural Hearing Aids on the Improvement of the Maximum Discrimination Score

両耳聴の意義に対する認識が高まり, 補聴器も人工内耳も可能であれば両耳聴を目指す方向に向かっている. 今回, 装用前の語音弁別能は不良であったが, 両耳装用後に実用レベルに達した症例について検討した. 耳鳴を主訴とした両側水平性感音難聴の2症例で, 語音弁別能は初診時左右それぞれ30～55%であったが, 1年後には両耳装用下の音場検査にて各々80,95%という結果が得られた. これは周波数ごとに圧縮を変更しながら十分なファンクショナルゲインを確保したことと, 優れた両耳聴効果に起因するところが大きいと思われる. そして, 補聴器の調整を適正に行い十分な聴能訓練を行うことで, 補聴器の効果をより発揮できると考える.

A Wearable Platform for Research in Augmented Hearing.

We have previously reported a realtime, open-source speech-processing platform (OSP) for hearing aids (HAs) research. In this contribution, we describe a wearable version of this platform to facilitate audiological studies in the lab and in the field. The system is based on smartphone chipsets to leverage power efficiency in terms of FLOPS/watt and economies of scale. We present the system architecture and discuss salient design elements in support of HA research. The ear-level assemblies support up to 4 microphones on each ear, with 96 kHz, 24 bit codecs. The wearable unit runs OSP Release 2018c on top of 64-bit Debian Linux for binaural HA with an overall latency of 5.6 ms. The wearable unit also hosts an embedded web server (EWS) to monitor and control the HA state in realtime. We describe three example web apps in support of typical audiological studies they enable. Finally, we describe a baseline speech enhancement module included with Release 2018c, and describe extensions to the algorithms as future work.

Scalp tapping-based protocol for adjusting the parameters of binaural hearing aids

Abstract Hearing aids (HAs) are used in everyday life. To improve the subjective satisfaction of HAs, a simple and comprehensive means to adjust the internal parameters to accommodate various situations is required. In this study, we propose a new scalp tapping-based protocol for controlling the parameters of binaural HAs. We then demonstrate the reliability, benefits, and clinical feasibility of the protocol via subjective evaluations from 11 volunteers. In the reliability test, the implemented scalp-tapping detection algorithm showed accuracies above 95.0% in various conditions and no false adjustment of internal parameters occurred from motions or other artifacts. In the benefit test, improvements of signal-to-noise ratio (SNR) and segmental SNR were 2.42–4.04 dB and 1.22–3.95 dB on average for serial connection of beamforming and noise-reduction algorithms. In the feasibility test, the range of control accuracy was 66.67–100% (88.06 ± 10.19%), and the range of latency time was 5.03–22.27 sec (7.25 ± 1.73 s). We expect that the proposed protocol can provide a simple and convenient way to adjust internal parameters of HAs without additional hardware changes and with short-term training.

Speech Enhancement Algorithm Based on Sound Source Localization and Scene Matching for Binaural Digital Hearing Aids

At present, the speech enhancement algorithm in binaural digital hearing aids is mainly based on adaptive beamforming algorithm. This algorithm strongly depends on the environment. And the enhancement performance is not satisfactory, which makes it difficult for hearing loss people to get high intelligibility and comfort speech. To solve this problem, a binaural speech enhancement algorithm based on sound source localization and scene matching is proposed in this paper. First, the spatial information of sound source is extracted by the sound source localization algorithm based on head-related transfer function and Gaussian mixture model classifier with Gammatone filter decomposition. Second, noise in different directions from the speaker is removed by spatial filter. Third, the type of noise in the same direction from the speaker is recognized by the scene recognition algorithm based on multi-feature and weighted minimum distance classifier. Finally, according to the type of noise, the optimal speech enhancement method is chosen to remove this noise. Experiment results show that the proposed algorithm has better robustness, better speech enhancement performance and lower complexity than the contrast algorithm.

Binaural Speaker Localization Integrated Into an Adaptive Beamformer for Hearing Aids

In this paper, we present and compare novel algorithms to localize simultaneous speakers using four microphones distributed on a pair of binaural hearing aids. The framework consists of two groups of localization algorithms, namely, beamforming-based and statistical model based localization algorithms. We first generalize our previously proposed methods based on beamforming techniques to the binaural configuration with 2 $\times$ 2 microphones. Next, we contribute two statistical model based methods for binaural localization using the maximum likelihood approach that also takes head-related transfer functions and unknown noise conditions into account. The methods enable the localization of multiple source positions for all azimuth angles and do not require prior training of binaural cues. The proposed localization algorithms are integrated into a generalized side-lobe canceller (GSC) to extract the desired speaker in the presence of competing speakers and background noise and when the head of the listener turns. The GSC components are adapted with the frequency-wise target presence probability and the frame-wise broadband direction-of-arrival (DOA) estimates that track the turns of the listener's head. We evaluate the performance of the localization algorithms individually and also in the context of the adaptive binaural beamformer in various noisy and reverberant conditions. Finally, we introduce a new adaptive beamformer, which combines the GSC with multichannel speech presence probability estimation and achieves superior source separation performance in noisy environment.