Hearing Aid Research Research Articles

Comparison of 2D and 3D multichannel audio rendering methods for hearing research applications using technical and perceptual measures

Virtual reality with multichannel audio playback is increasingly used in hearing aid research. The purpose of this study is to compare horizontal (2D) and periphonic (3D) rendering methods in terms of localization, minimum audible angle, and perceptual ratings related to spatial quality. Higher Order Ambisonics, Vector-Base Amplitude Panning, and Nearest Speaker Selection were used, with playback through 16, 29 and 45 speakers. The results show that an improvement in vertical localization can be obtained by using periphonic rendering instead of horizontal rendering. The perceptual advantage of periphonic rendering depends on the spatial complexity of the scene; it disappears in complex acoustic environments. Scenes with low acoustic complexity, such as a single primary sound source in a room, benefit from Nearest Speaker Selection rendering. For more complex scenes with multiple sound sources, such as a symphony orchestra in a concert hall with many primary sources, or traffic on a road with moving sources, horizontal rendering methods such as 2D Higher Order Ambisonics will provide similar or better performance.

A High-Performance, Low Power Research Hearing Aid featuring a High-Level Programmable Custom 22nm FDSOI SoC.

With advances in algorithmic hearing aid research, the need for high-level programmable, behind-the-ear (BTE) wearable and low-power research platforms is emerging. These can be used to test new algorithms in real-world scenarios. Although various groups are developing different portable solutions, they are not in a BTE form factor. For this reason, the devices must be worn around the neck or somewhere on the body, which causes limited mobility and can lead to inaccurate research results. Therefore, this work presents a fully integrated and functional hearing aid research platform that weighs only 5 grams and can be worn behind the ear. The platform is high-level programmable, features wireless technologies such as near-field magnetic induction (NFMI) and Bluetooth Low Energy (BLE), and integrates two micro-electro-mechanical systems (MEMS) microphones and an external speaker. The audio processor of the system is based on a new custom, low-power 22nm mixed-signal system on chip (SoC). Different real-world use cases, like a dynamic compressor, are used to evaluate the platform. With a total power consumption of 47 mW, the rechargeable device achieves a run-time of six hours. When the wireless interfaces are turned off, the power consumption drops to 31 mW, and the run-time increases to nine hours.Clinical relevance-The proposed research hearing aid demonstration platform can be used portable and outside the clinical setting for algorithmic research. With its behind-the-ear form factor and rechargeable battery, studies can be conducted for several hours without restricting patient movement in real-world scenarios.

Portable hearing laboaratory for developping and evaluating novel hearing aid algorithms

We report here on the further development of our portable hearing aid research platform achived in Phase III NIDCD project R44DC016247. We also report on a large number of academic studies using the platform around the world. The project builds on the openMHA distribution for ARM processors included in an optimized Linux system developed in our other project R01DC015429. Thus we provide a complete wearable software-hardware fast hearing aid device needed for development of new innovative solutions for assisted hearing. We refer to this device as the PHL, short for Portable Hearing Laboratory. Throught the project we have followed the requirement that the platform should be “acoustically perfect.” That is able to transmit and process signals without audible distortion or noise beyond what is required to compensate for hearing loss. Any new sophisticated algorithms (such as those based on the AI) if run on a platform of inferior quality will adapt for that inferior structire and fail to capitalize on the reach processing capabilities of the modern hearing aids. We also report on a family of devices that can be used either with PHL or with other processors to provide supperior sound; such as bunaural BTE multimicrophone headsets. Last but not least we will present new devices connecting these auxiliary devices to powefull desktop computers for laboratory studies.

Audio Classification Algorithm for Hearing Aids Based on Robust Band Entropy Information

Audio classification algorithms for hearing aids require excellent classification accuracy. To achieve effective performance, we first present a novel supervised method, involving a spectral entropy-based magnitude feature with a random forest classifier (SEM-RF). A novel-feature SEM based on the similarity and stability of band signals is introduced to improve the classification accuracy of each audio environment. The random forest (RF) model is applied to perform the classification process. Subsequently, to resolve the problem of decreasing classification accuracy of the SEM-RF algorithm in mixed speech environments, an improved algorithm, ImSEM-RF, is proposed. The SEM features and corresponding phase features are fused on multiple time resolutions to form a robust multi-time resolution magnitude and phase (multi-MP) feature, which improves the stability of the feature with which the speech signal interferes. The RF model is improved using the linear discriminant analysis (LDA) method to form a linear discriminant analysis-random forest (LDA-RF) joint classification model, which performs model acceleration. Through experiments on hearing aid research data sets for acoustic environment recognition, the effectiveness of the SEM-RF algorithm was confirmed on a background audio signal dataset. The classification accuracy increased by approximately 7% compared with the background noise classification algorithm using an RF tree classifier. The validity of the ImSEM-RF algorithm in speech-interference environments was confirmed using the speech in the background audio signal dataset. Compared with the SEM-RF algorithm, the classification accuracy was improved by approximately 2%. The LDA-RF reduced the program’s running time by >80% with multi-MP features compared with RF.

Fixed Point Analysis Workflow for efficient Design of Convolutional Neural Networks in Hearing Aids

Abstract Neural networks (NN) are a powerful tool to tackle complex problems in hearing aid research, but their use on hearing aid hardware is currently limited by memory and processing power. To enable the training with these constrains, a fixed point analysis and a memory friendly power of two quantization (replacing multiplications with shift operations) scheme has been implemented extending TensorFlow, a standard framework for training neural networks, and the Qkeras package [1, 2]. The implemented fixed point analysis detects quantization issues like overflows, underflows, precision problems and zero gradients. The analysis is done for each layer in every epoch for weights, biases and activations respectively. With this information the quantization can be optimized, e.g. by modifying the bit width, number of integer bits or the quantization scheme to a power of two quantization. To demonstrate the applicability of this method a case study has been conducted. Therefore a CNN has been trained to predict the Ideal Ratio Mask (IRM) for noise reduction in audio signals. The dataset consists of speech samples from the TIMIT dataset mixed with noise from the Urban Sound 8kand VAD-dataset at 0 dB SNR. The CNN was trained in floating point, fixed point and a power of two quantization. The CNN architecture consists of six convolutional layers followed by three dense layers. From initially 1.9 MB memory footprint for 468k float32 weights, the power of two quantized network is reduced to 236 kB, while the Short Term Objective Intelligibility (STOI) Improvement drops only from 0.074 to 0.067. Despite the quantization only a minimal drop in performance was observed, while saving up to 87.5 % of memory, thus being suited for employment in a hearing aid

A Step towards Neuro-Steered Hearing Aids: Integrated Portable Setup for Time- Synchronized Acoustic Stimuli Presentation and EEG Recording

Abstract Aiming to provide a portable research platform to develop algorithms for neuro-steered hearing aids, a joint hearing aid - EEG measurement setup was implemented in this work. The setup combines the miniaturized electroencephalography sensor technology cEEGrid with a portable hearing aid research platform - the Portable Hearing Laboratory. The different components of the system are connected wirelessly, using the lab streaming layer framework for synchronization of audio and EEG data streams. Our setup was shown to be suitable for simultaneous recording of audio and EEG signals used in a pilot study (n=5) to perform an auditory Oddball experiment. The analysis showed that the setup can reliably capture typical event-related potential responses. Furthermore, linear discriminant analysis was successfully applied for single-trial classification of P300 responses. The study showed that time-synchronized audio and EEG data acquisition is possible with the Portable Hearing Laboratory research platform.

Behavioral Validation of the Smartphone for Remote Microphone Technology.

As part of a National Institutes of Health-National Institute on Deafness and Other communication Disorders (NIH-NIDCD)-supported project to develop open-source research and smartphone-based apps for enhancing speech recognition in noise, an app called Smartphone Hearing Aid Research Project Version 2 (SHARP-2) was tested with persons with normal and impaired hearing when using three sets of hearing aids (HAs) with wireless connectivity to an iPhone. Participants were asked to type sentences presented from a speaker in front of them while hearing noise from behind in two conditions, HA alone and HA + SHARP-2 app running on the iPhone. The signal was presented at a constant level of 65 dBA and the signal-to-noise ratio varied from -10 to +10, so that the task was difficult when listening through the bilateral HAs alone. This was important to allow for improvement to be measured when the HAs were connected to the SHARP-2 app on the smartphone. Benefit was achieved for most listeners with all three manufacturer HAs with the greatest improvements recorded for persons with normal (33.56%) and impaired hearing (22.21%) when using the SHARP-2 app with one manufacturer's made-for-all phones HAs. These results support the continued development of smartphone-based apps as an economical solution for enhancing speech recognition in noise for both persons with normal and impaired hearing.

Open Speech Platform: Democratizing Hearing Aid Research.

Hearing aids help overcome the challenges associated with hearing loss, and thus greatly benefit and improve the lives of those living with hearing-impairment. Unfortunately, there is a lack of adoption of hearing aids among those that can benefit from hearing aids. Hearing researchers and audiologists are trying to address this problem through their research. However, the current proprietary hearing aid market makes it difficult for academic researchers to translate their findings into commercial use. In order to abridge this gap and accelerate research in hearing health care, we present the design and implementation of the Open Speech Platform (OSP), which consists of a co-design of open-source hardware and software. The hardware meets the industry standards and enables researchers to conduct experiments in the field. The software is designed with a systematic and modular approach to standardize algorithm implementation and simplify user interface development. We evaluate the performance of OSP regarding both its hardware and software, as well as demonstrate its usefulness via a self-fitting study involving human participants.

Modelling of Hearing Aid’s Digital Signal Processor

According to the World Health Organization (WHO), hearing loss (HL) is one of the six key contributors to worldwide disease rates. It is becoming a critical issue in society, not just affecting the aging population, but also negatively impacting young people who are spending more of their spare time performing activities that expose them to excessive noise levels. In this research, we intend to design a filter as a signal processing system in a hearing aid (HA). MATLAB is used to model the digital filter structure, while Simulink is used to capture the entire design. This study examines current critical concerns in hearing aid research from the perspectives of a variety of disciplines. The study proposes a filter and signal processor model based on hearing aid experience, but first, it provides an audiogram for numerous examples to determine if the suggested model would suit or not. The work thus provides an effective compensation of missed high-frequency sounds response in patient hearing by the digital signal processor.

Noise management features of the open speech platform

In this contribution, we present the noise management features of the Open Speech Platform (OSP) for hearing aid (HA) research. OSP includes basic HA modules (i) subband decomposition, (ii) wide dynamic range compression and (iii) adaptive feedback cancellation; a baseline single-channel speech enhancement (SE) based on Wiener filtering for noise subtraction. We extended OSP noise management to include a generalized sidelobe cancellation (GSC) beamforming between left and right channels, followed by the SE. Time is the scarcest resource, followed by central processing unit (CPU) resources in commercial HAs. With the proposed GSC + SE approach, we were both time and CPU limited. Release 2019a of OSP has 5.6 ms end-to-end latency and GSC requires an additional 5 ms, putting us above the 10 ms requirement. The wearable device of OSP has 4 cores, C0 – C3. C0 is used for all non-realtime tasks such as kernel, embedded web server, etc. and remaining cores are used for realtime tasks. Naive realization of GSC results in one or more cores not meeting the realtime constraints, resulting in audible artifacts. We present optimizations to meet time and CPU budgets; and preliminary objective and subjective results of the proposed system.

Open Master Hearing Aid (openMHA)—An integrated platform for hearing aid research

The project R01DC015429 “Open community platform for hearing aid algorithm research” provides a software platform for real-time, low-latency audio signal processing: the open Master Hearing Aid (openMHA). It contains a versatile set of basic and advanced methods for hearing aid processing, as well as tools and manuals enabling the design of own setups for algorithm development and evaluation. Documentation is provided for different user levels, in particular for audiologists, application engineers and algorithm designers. The software runs on various computer systems including lab setups and portable setups. Portable setups are of particular interest for the evaluation of new methods in real-word scenarios. In addition to standard off-the-shelf hardware, a portable, integrated research platform for openMHA is provided in conjunction with the SBIR project R44DC016247. This contribution introduces openMHA and discusses the usage and possible application scenarios of the portable openMHA setup in hearing research. The opportunity is given to try a smartphone-based self-fitting application for the portable openMHA, and to learn about the flexible configuration and remote control of openMHA running a typical hearing aid processing chain. Furthermore, a discussion and exchange of ideas on current challenges and future developments is offered.

A Toolbox for Rendering Virtual Acoustic Environments in the Context of Audiology

A toolbox for creation and rendering of dynamic virtual acoustic environments (TASCAR) that allows direct user interaction was developed for application in hearing aid research and audiology. This technical paper describes the general software structure and the time-domain simulation methods, i.e., transmission model, image source model, and render formats, used to produce virtual acoustic environments with moving objects. Implementation-specific properties are described, and the computational performance of the system was measured as a function of simulation complexity. Results show that on commercially available commonly used hardware the simulation of several hundred virtual sound sources is possible in the time domaina

CCi-MOBILE: Design and Evaluation of a Cochlear Implant and Hearing Aid Research Platform for Speech Scientists and Engineers.

Hearing loss is an increasingly prevalent condition resulting from damage to the inner ear which causes a reduction in speech intelligibility. The societal need for assistive hearing devices has increased exponentially over the past two decades; however, actual human performance with such devices has only seen modest gains relative to advancements in digital signal processing (DSP) technology. A major challenge with clinical hearing technologies is the limited ability to run complex signal processing algorithms requiring high computation power. The CCi-MOBILE platform, developed at UT-Dallas, provides the research community with an open-source, flexible, easy-to-use, software-mediated, powerful computing research interface to conduct a wide variety of listening experiments. The platform supports cochlear implants (CIs) and hearing aids (HAs) independently, as well as bimodal hearing (i.e., a CI in one ear and HA in the contralateral ear). The platform is ideally suited to address hearing research for: both quiet and naturalistic noisy conditions, sound localization, and lateralization. The platform uses commercially available smartphone/tablet devices as portable sound processors and can provide bilateral electric and acoustic stimulation. The hardware components, firmware, and software suite are presented to demonstrate safety to the speech scientist and CI/HA user, highlight user-specificity, and outline various applications of the platform for research.

Open portable platform for hearing aid research

The NIDCD has recently funded a number of projects to develop portable signal processing tools that enable real-time processing of the acoustic environment. The overarching goal is to provide a large group of researchers with the means to efficiently develop and evaluate, in collaborative multi-center environments, novel signal processing schemes, individualized fitting procedures, and technical solutions and services for hearing apparatus such as hearing aids and assistive listening devices. We report on the specific goals and results of two such projects. In one of them (R01DC015429), an open source software platform for real-time runtime environments is developed: The open Master Hearing Aid (openMHA). It provides an extendible set of algorithms for hearing aid signal processing and runs under Linux, Windows, and Mac operating systems on standard PC platforms and on small-scale ARM-based boards. An optimized version of openMHA is provided for the companion SBIR project (R44DC016247), which is a portable, rigid, versatile, and wearable platform featuring an ARM Cortex®-A8 processor. The resulting Portable Hearing Aid Community Platform consists of both hardware elements to provide the advanced desired functionality and software routines to provide for all the features that researchers may need to develop new algorithms.

Development of an open source audio processing platform

The Open-Source software movement has demonstrated the value and quality of open-source tools and includes well-established systems such the Android Operating System. This movement is expanding to hardware and when designed properly, open-source hardware combines the user-friendliness and dependability of commercial products with the high performance and flexibility of scientific tools developed by researchers in their own laboratories. In this research, we present a hardware and software platform entirely open-source, designed to spur innovation in sound and speech processing algorithms. The first prototype includes a Teensy 3.6 board which leverages the Arduino development environment while providing powerful computational capabilities. This processor board is paired with custom-designed electronics for audio control, power management and wireless communication. The software base is designed for multiple users: (1) “experts” who optimize new algorithms directly in firmware; and (2) researchers who interact with existing algorithms to modify parameters and evaluate their performance in different conditions. Hearing aid research and other audio signal processing fields will greatly benefit from the open sharing of algorithms on one common platform that is easily obtained from existing vendors. Here we present the current functionality of the platform, as well as how to access it and contribute to this exciting Open Source project.

Estimation of speech source direction for hearing aid application using smartphone

In our hearing aid research project funded by NIH-NIDCD we utilize smartphone and its features as a powerful platform to implement complex signal processing algorithms which would improve hearing aid applications. Among the required algorithms, is finding the direction of arrival of speech source signals which can improve the performance of hearing aid devices for the users. In this poster presentation, a signal processing algorithm for estimating the speech source direction and its real-time implementation on an Android smartphone are demonstrated. The visual display of source direction on the smartphone panel can be used to align the phone and its two microphones in the right direction of source to enhance the signal to noise ratio and improve the reception of speech signal. A demo unit will be presented showing the real-time operation of the proposed method on an Android smartphone.

Evaluation of Spatial Audio Reproduction Schemes for Application in Hearing Aid Research

Loudspeaker-based spatial audio reproduction schemes are increasingly used for evaluating hearing aids in complex acoustic conditions. To further establish the feasibility of this approach, this study investigated the interaction between spatial resolution of different reproduction methods and technical and perceptual hearing aid performance measures using computer simulations. Three spatial audio reproduction methods -- discrete speakers, vector base amplitude panning and higher order ambisonics -- were compared in regular circular loudspeaker arrays with 4 to 72 channels. The influence of reproduction method and array size on performance measures of representative multi-microphone hearing aid algorithm classes with spatially distributed microphones and a representative single channel noise-reduction algorithm was analyzed. Algorithm classes differed in their way of analyzing and exploiting spatial properties of the sound field, requiring different accuracy of sound field reproduction. Performance measures included beam pattern analysis, signal-to-noise ratio analysis, perceptual localization prediction, and quality modeling. The results show performance differences and interaction effects between reproduction method and algorithm class that may be used for guidance when selecting the appropriate method and number of speakers for specific tasks in hearing aid research.

Predicting the effect of hearing loss and audibility on amplified speech reception in a multi-talker listening scenario

Auditive and cognitive influences on speech perception in a complex situation were investigated in listeners with normal hearing (NH) and hearing loss (HL). The speech corpus used was the Nonsense-Syllable Response Measure [NSRM; Woods and Kalluri, (2010). International Hearing Aid Research Conference, pp. 40-41], a 12-talker corpus which combines 154 nonsense syllables with 8 different carrier phrases. Listeners heard NSRM sentences in quiet, background noise, and in background noise plus other "jammer" NSRM sentences. All stimuli were linearly amplified. A "proficiency" value, determined from the results in quiet and the quiet-condition speech intelligibility index (SII), was used with the SII in predicting results in the other conditions. Results for nine of ten NH subjects were well-predicted (within the limits of binomial variability) in the noise condition, as were eight of these subjects in the noise-plus-jammers condition. All 16 HL results were well-predicted in the noise condition, as were 9 of the HL in the noise-plus-jammers condition. Hierarchical regression partialling out the effects of age found proficiency in noise-plus-jammers significantly correlated with results of "trail-making" tests, thought to index processing speed and attention-deployment ability, and proficiency in quiet and noise was found significantly correlated with results from a backward digit-span memory test.

A Real-Time Hearing-Aid Research Platform (HARP): Realization, Calibration, and Evaluation

A Real-Time Hearing-Aid Research Platform (HARP): Realization, Calibration, and Evaluation

Evaluation of the Phase-Inversion Signal Separation Method When Using Nonlinear Hearing Aids

Using two measurements with simultaneous speech and noise presentation, Hagerman and Olofsson have suggested a time-domain method to estimate the speech and noise signals at the output of a hearing device. The method, which uses a simple phase-inversion scheme, has gained popularity in hearing-aid research, although receiving only limited validation. In this work, we present an evaluation of this signal-separation method using simulated measurements with different hearing aids and listening conditions. Estimates of the speech and noise spectra from the phase-inversion method are compared to those obtained using the coherence function. New measures of speech and noise distortion are proposed as tools to evaluate the phase-inversion method. Additionally, we analyze the intelligibility predictions computed from the recovered spectral estimates, while accounting for the proposed speech distortion measure. Under additive-noise conditions, the phase-inversion method provides ideal signal separation without suffering any biases at low signal-to-noise ratios. For conditions involving automatic gain control, compressive output limiting, and peak clipping, the intelligibility predictions based on the phase-inversion method are found to agree with relevant findings from the literature.