Binaural System Research Articles

When does binaural hearing benefit from ongoing envelope fluctuations?

A number of classic and recent studies suggest that listeners’ access to ongoing binaural information in sounds with steady envelopes is significantly poorer than for sounds with stochastic or slowly modulated ongoing envelopes. For some types of binaural cues—e.g., envelope interaural time differences (ITD)—that result is consistent with the requirement of salient envelope features for accurate cue encoding. For interaural level differences (ILD) and fine-structure ITD, however, no such requirement exists and the result is therefore more surprising. Regardless, the overall results suggest that binaural-cue encoding-in general-relies heavily upon the occurrence of infrequent envelope fluctuations, such as sound onsets and slow (e.g., syllabic-rate) amplitude modulations. Here, the results of relevant studies are reviewed, along with conceptual and physiological models of sampling binaural cues during envelope fluctuations. Potential advantages for binaural hearing in reverberation and competing backgrounds are considered, as is the impact of such mechanisms on the localization of strongly modulated targets such as speech. Overall results suggest a general degradation of binaural information-regardless of cue type-in the absence of envelope fluctuations. Instead, the binaural system emphasizes representations of temporally sparse, but highly localizable, onset-like events in the auditory scene. [Work supported by NIH R01-DC011548.]

Performance ratings for children using bilateral cochlear implants obtained with the Speech, Spatial, and Other Qualities of Hearing Scale for parents

Introduction Researchers at The University of Melbourne are evaluating the broader outcomes of bilateral implantation in early implanted school-age children, including effects on parent and child stress (Sarant and Garrard, 2013), language (Sarant et al., 2014), psychosocial functioning, academic achievement, and everyday listening. The Speech, Spatial, and Other Qualities of Hearing Scale (SSQ) is a three-part adult questionnaire (http://www.ihr.mrc.ac.uk/products/ display/ssq; Gatehouse and Noble, 2004). The SSQ focusses mainly on hearing functions requiring the binaural system and is therefore suitable for evaluating everyday listening performance with bilateral implants. The SSQ-P is an adapted version for use with parents of children with hearing impairment (Galvin and Noble, 2013). The first section of the SSQ focusses on speech perception in different listening scenarios, such as in group conversation in a noisy environment, or a single speaker in a reverberant environment. The second section focusses on spatial hearing, in this case the direction of movement and location of sound sources, including the speaker in a group conversation. The final section focusses on other qualities of hearing, including sound segregation, listening effort, and sound and object identification.

Rapid binaural processing for source segregation and lateralization

For realistic listening conditions, interaural cues will fluctuate due to the presence of multiple active sources. If it is assumed that the binaural system is sluggish, then the perceived location of the sound input would be an average of the varying interaural cues. If, however, the binaural system is fast enough to assess the rapidly changing interaural differences, then it could be possible for the binaural system to properly identify the spatial position of a target source. Using a continuous, broadband noise stimulus that contained periodically alternating interaural time differences (ITD) and, notably, no monaural cues, we investigated the binaural system's ability to lateralize brief durations of the target ITD. Results show that listeners can lateralize targets for durations of 3–6 ms indicating that the binaural system allows for a segregation and lateralization of the target and interfering noise streams. Furthermore, results indicate that the binaural system mediates the buildup of a modulated stream. A second experiment investigating whether the salience of the target ITD in the aforementioned stimulus depends on the temporal position of the target within the phase of an amplitude modulated envelope revealed that this was not the case.

AUDIO SYSTEM WITH FEEDBACK DETECTION MEANS

The invention concerns in a first aspect an audio system comprising a microphone, audio signal processing means, an output transducer and means for detecting a possible feedback tone and the corresponding frequency of the feedback tone in the audio system between the output transducer and the microphone. According to the invention means for counteracting feedback are provided. Further, means are provided for changing the phase of the audio signal at a given frequency. In a further aspect, a binaural hearing aid system comprising first and second hearing instruments according to the first aspect, each hearing instrument comprising transceiver circuitry allowing an exchange of signals between the two hearing instruments, and wherein the binaural hearing aid system is adapted to provide that a phase change introduced in the first audio signal by a controller of the first hearing instrument is or can be introduced in the second audio signal of the second hearing instrument via said transceiver circuitry.

Rate-constrained source separation for speech enhancement in wireless-communicated binaural hearing aids

A recent trend in hearing aids is the connection of the left and right devices to collaborate between them. Binaural systems can provide natural binaural hearing and support the improvement of speech intelligibility in noise, but they require data transmission between both devices, which increases the power consumption. This paper presents a novel sound source separation algorithm for binaural speech enhancement based on supervised machine learning and time-frequency masking. The system is designed considering the power restrictions in hearing aids, constraining both the computational cost of the algorithm and the transmission bit rate. The transmission schema is optimized using a tailored evolutionary algorithm that assigns a different number of bits to each frequency band. The proposed algorithm requires less than 10% of the available computational resources for signal processing and obtains good separation performance using bit rates lower than 64 kbps.

Computational Audiovisual Scene Analysis in Online Adaptation of Audio-Motor Maps

For sound localization, the binaural auditory system of a robot needs audio-motor maps, which represent the relationship between certain audio features and the position of the sound source. This mapping is normally learned during an offline calibration in controlled environments, but we show that using computational audiovisual scene analysis (CAVSA), it can be adapted online in free interaction with a number of a priori unknown speakers. CAVSA enables a robot to understand dynamic dialog scenarios, such as the number and position of speakers, as well as who is the current speaker. Our system does not require specific robot motions and thus can work during other tasks. The performance of online-adapted maps is continuously monitored by computing the difference between online-adapted and offline-calibrated maps and also comparing sound localization results with ground truth data (if available). We show that our approach is more robust in multiperson scenarios than the state of the art in terms of learning progress. We also show that our system is able to bootstrap with a randomized audio-motor map and adapt to hardware modifications that induce a change in audio-motor maps.

Lateralization, discrimination, detection, and Ervin

Erv Hafter’s early work focused on the development of a comprehensive understanding of the binaural system, combining a characterization of the subjective phenomena associated with binaural lateralization with the results of objective experiments measuring interaural discrimination and binaural detection. To a large extent, this author’s attempts to characterize a broad range of auditory phenomena based on a Jeffress-Colburn cross-correlation-based model can be viewed as a reformulation of the stimulus-based lateralization model that Hafter developed to provide a unified theoretical framework for his own findings. Erv Hafter has a gift both for designing clever and revealing experimental stimuli and paradigms, as well as for developing provocative interpretations of his experimental results. This paper will discuss a selection of complex binaural phenomena revealed by Erv’s research in the context of their impact on contemporary theories of binaural interaction, as well as their impact on computational models of speech processing for robust speech recognition. [Work supported by DARPA and Cisco Research.]

Adaptation of the speech, spatial, and qualities of hearing scale for use with children, parents, and teachers

Subjective assessment of hearing ability in everyday life complements more objective forms of evaluation. A broad evaluation of the additional benefit provided to children by a second bilateral cochlear implant required such an assessment. As no paediatric tool provided detailed evaluation of performance in the areas of daily listening in which benefit was likely to be demonstrated, an adult questionnaire was adapted. Items of the Speech, Spatial and Qualities of Hearing Scale (SSQ) focused mainly, although not exclusively, on hearing functions requiring the binaural system. The adapted child, parent, and teacher versions of the SSQ retained the structure of rating listening performance in everyday scenarios across the domains of speech perception, spatial hearing, and other qualities of hearing. Modifications were minimized, although deletion of some items and wording changes were required, and some subdomains could not be included. Observation periods were introduced so that parents and teachers observe performance prior to providing ratings. The suggested minimum age is 11 years for the child version and 5 years for the parent and teacher versions. Instructions indicate interview-style administration in which interpretation of the described listening scenarios can be clarified and use of the ruler-style response format demonstrated. Researchers applying the SSQ for parents have reported higher performance ratings for bilateral over unilateral cochlear implants, particularly in the spatial hearing domain. Further research should provide evidence for the target age range, compare child and parent responses, and evaluate modifications for use with younger children.

Applications of models of binaural hearing

Models of binaural hearing have been proposed for more than 60 decades, but it is only recently that they are ready for technological application. Recognizing this situation, 15 laboratories in Europe and in the United States have founded a research group (AABBA) with the aim of setting in place “Aural Assessment by Means of Binaural Algorithm.” Now, after its first 4-years term of activity, the group has published their recent results in a chapter book (Springer)—including a software package for building customized binaural models. In this talk, the group’s most relevant results will be described. The results concern the following areas of model application, among others: binaural analysis of aural scenes, binaural de-reverberation, binaural quality assessment of audio channels, loudspeakers and performance spaces, binaural perceptual coding, binaural processing in hearing aids and cochlea implants, binaural systems in robots, binaural/tactile human–machine interfaces, speech-intelligibility prediction in rooms, and/or multi-speaker scenarios. Current members of AABBA are research labs at Universities/Techn. Universities in Helsinki, Boston, Cardiff, Oldenburg, Lyon, Troy, NY (Rensselaer), Bochum, Berlin, Copenhagen, Dresden, Eindhoven, Munich, Paris (Pierre et Marie Curie), Toulouse, and at the Austrian Academy’s Acoustics Research Inst. in Vienna.

Speech intelligibility among modulated and spatially distributed noise sources

At a cocktail party, listeners are faced with multiple, spatially distributed interfering voices. The dominant interfering voice may change from moment to moment and, consequently, change in spatial location. The ability of the binaural system to deal with such a dynamic scene has not been systematically analyzed. Spatial release from masking (SRM) was measured in simple spatial scenes, simulated over headphones with a frontal speech source. For a single noise at 105°, SRM was reduced if that noise modulated (10 Hz square wave, 50% duty cycle, 20 dB modulation depth), but, for two noises in symmetrical locations, SRM increased if the noises were modulated in alternation, suggesting that the binaural system can "switch" between exploiting different spatial configurations. Experiment 2 assessed the contributions of interaural time and level differences as a function of modulation rate (1-20 Hz). Scenes were created using the original head-related impulse responses and ones that had been manipulated to isolate each cue. SRM decreased steeply with modulation rate. The combined effects of interaural time and level differences were consistent with additive contributions. The results indicate that binaural sluggishness limits the contribution of binaural switching to speech understanding at a cocktail party.

HRTF Adjustments with Audio Quality Assessments

Abstract There are an increasing number of binaural systems embedded with head-related transfer functions (HRTFs), so listeners can experience virtual environments via conventional stereo loudspeakers or head- phones. As HRTFs vary from person to person, it is difficult to select appropriated HRTFs from already existing databases for users. Once the HRTFs in a binaural audio device hardly match the real ones of the users, poor localization happens especially on the cone of confusion. The most accurate way to obtain personalized HRTFs might be doing practical measurements. It is, however, expensive and time consuming. Modifying non-individualized HRTFs may be an effort-saving way, though the modifications are always accompanied by undesired audio distortion. This paper proposes a flexible HRTF adjustment system for users to define their own HRTFs. Also, the system can keep sounds from suffering intolerable distortion based on an objective measurement tool for evaluating the quality of processed audio.

METHOD FOR OPERATING A BINAURAL HEARING SYSTEM AS WELL AS A BINAURAL HEARING SYSTEM

A hearing system and a method for operating a binaural hearing system are disclosed that comprises at least two hearing devices (1, 2) to be at least partly inserted into or to be worn behind the left and right ear of a user. Each hearing device (1, 2) comprises at least one microphone (2a, 2b) to generate an electrical signal corresponding to an acoustic signal. The inventive method comprises the step of : - determining contra-lateral information (CLD) based on an acoustic signal recorded by a microphone (2a, 2b) of the contra-lateral hearing device (1, 2), - determining ipsi-lateral information (ILD) based on an acoustic signal recorded by a microphone (2b, 2a) of the ipsi-lateral hearing device (2, 1), - providing a coordination level, the coordination level being indicative of a degree of synchronization of the two hearing devices (1, 2), and - adjusting processes in the ipsi-lateral hearing device (2, 1) in accordance with the coordination level, wherein the coordination level either being determined from the contra-lateral information and/or from the ipsi-lateral information, or being obtained from an external device (8). The inventive hearing system has the advantage that the hearing devices (1, 2) can be operated at different coordination levels resulting in improved hearing ability and inconspicuousness for the user.

METHOD AND APPARATUS FOR A BINAURAL HEARING ASSISTANCE SYSTEM USING MONAURAL AUDIO SIGNALS

The present application provides method and apparatus for a binaural hearing assistance system using a monaural audio signal input. The system, in various examples, provides adjustable delay/phase adjustment and sound level adjustment. Different embodiments are provided for receiving the monaural signal and distributing it to a plurality of hearing assistance devices. Different relaying modes are provided. Special functions are supported, such as telecoil functions. The system also has examples that account for a head-related transfer function in providing advanced sound processing for the wearer. Other examples are provided that are described in the detailed description.

A spatial encoding method for measured room impulse responses

The spatial information contained in measured room impulse responses can be used to explain some of the acoustical properties of performance spaces. This paper presents a spatial encoding method, which can extract accurate spatial information from impulse responses that are measured with at least four microphones in an open 3-D array. The method is based on decomposing a spatial room impulse response into a set of image-sources, i.e., every single sample in the impulse response is considered as an image source. Each of the image-sources is localized with an acoustic source localization method, which depends on the applied microphone array and the acoustic conditions. Due to the image-source presentation, the presented method can be applied to any compact array and used in conjunction with variety of current spatial loudspeaker reproduction systems to create convolution reverb-type spatial sound reproduction. The method allows static and interactive binaural reproduction via virtual loudspeaker arrays. The presentation includes demonstrations with a binaural reproduction system.

The effect of interaural differences in envelope shape on the perceived location of sounds (L)

Users of bilateral cochlear implants and a cochlear implant combined with a contralateral hearing aid are sensitive to interaural time differences (ITDs). The way cochlear implant speech processors work and differences between modalities may result in interaural differences in shape of the temporal envelope presented to the binaural system. The effect of interaural differences in envelope shape on ITD sensitivity was investigated with normal-hearing listeners using a 4 kHz pure tone modulated with a periodic envelope with a trapezoid shape in each cycle. In one ear the onset segment of the trapezoid was transformed by a power function. No effect on the just noticeable difference in ITD was found with an interaural difference in envelope shape, but the ITD for a centered percept was significantly different across envelope shape conditions.

Detection of simultaneous modulation of interaural time and level differences: Effects of modulation rate and relative phase (L)

The binaural system is known to be sluggish, i.e., unable to track modulations in interaural parameters even at a relatively slow rate. The present study evaluated the binaural system's sensitivity to modulation phase rather than to modulation magnitude. The detectability of simultaneous modulations in interaural time and level differences with various relative phases were measured. It was found that for modulation rates up to 10-20 Hz, the detectability varied with the relative phase. This indicates that information about higher rates is lost at or below the level of cue integration.

Binaural prediction of speech intelligibility in reverberant rooms with multiple noise sources

When speech is in competition with interfering sources in rooms, monaural indicators of intelligibility fail to take account of the listener's abilities to separate target speech from interfering sounds using the binaural system. In order to incorporate these segregation abilities and their susceptibility to reverberation, Lavandier and Culling [J. Acoust. Soc. Am. 127, 387-399 (2010)] proposed a model which combines effects of better-ear listening and binaural unmasking. A computationally efficient version of this model is evaluated here under more realistic conditions that include head shadow, multiple stationary noise sources, and real-room acoustics. Three experiments are presented in which speech reception thresholds were measured in the presence of one to three interferers using real-room listening over headphones, simulated by convolving anechoic stimuli with binaural room impulse-responses measured with dummy-head transducers in five rooms. Without fitting any parameter of the model, there was close correspondence between measured and predicted differences in threshold across all tested conditions. The model's components of better-ear listening and binaural unmasking were validated both in isolation and in combination. The computational efficiency of this prediction method allows the generation of complex "intelligibility maps" from room designs.

Binaural masking level difference in skilled reading children and children with dyslexia

Combining stimuli arriving at both ears makes it possible to locate sounds in the environment and to better detect signals or understand speech in noise when the sound sources are separated spatially. The sensitivity of the binaural system to interaural differences in time and amplitude can be investigated by means of the binaural masking level difference (BMLD). The age at which the BMLD reaches adult levels appears to depend partly upon masker bandwidth. Less is known about the effect of masker's level on the development of BMLD in children. In the present study we assessed the effect of masker level on the BMLD of 3rd and 5th grade skilled reading children. In view of possible binaural hearing effects in dyslexia, the BMLD of a group of 5th grade children with reading difficulties was measured. Detection thresholds of 500 Hz pure tone were measured at noise levels of 40 dBHL, 50 dBHL and 60 dBHL. All subjects presented increased MLD values with the rise of noise intensity between 40 dBHL and 60 dBHL. Among the skilled readers the results showed that younger children had smaller BMLDs than older children at all masker levels. However, a significant group-by-intensity interaction indicated that although the reading disabled group had reduced BMLD values than older skilled readers at noise levels of 50 dBHL and 60 dBHL, no difference was found between their BMLD values and those of the young skilled readers at noise levels of 50 dBHL and 60 dBHL. Moreover, their BMLD values at noise level of 40 dBHL were higher than those of the 3rd grade typically reading students while no difference was found between them and 5th grade efficient readers. These results support the existence of both quantitative and qualitative differences in binaural hearing of children with developmental dyslexia.

Towards quantifying cochlear implant localization performance in complex acoustic environments

Cochlear implant (CI) users frequently report listening difficulties in reverberant and noisy spaces. While it is common to assess speech understanding with implants in background noise, binaural hearing performance has rarely been quantified in the presence of other sources, although the binaural system is a major contributor to the robustness of speech understanding in noisy situations with normal hearing. Here, a pointing task was used to measure horizontal localization ability of a bilateral CI user in quiet and in a continuous diffuse noise interferer at a signal-to-noise ratio of 0 dB. Results were compared to localization performance of six normal hearing listeners. The average localization error of the normal hearing listeners was within normal ranges reported previously and only increased by 1.8° when the interfering noise was introduced. In contrast, the bilateral CI user showed a localization error of 22° in quiet which rose to 31° in noise. This increase was partly due to target sounds being inaudible when presented from frontal locations between −20° and +20°. With the noise present, the implant user was only able to reliably hear target sounds presented from locations well off the median plane. The results give support to the informal complaints raised by CI users and can help to define targets for the design of, e.g., noise reduction algorithms for implant processors.

Resolution of ITD detection: stochastic vs. exquisite system

Differences in arrival times of a sound at the two ears (Interaural Time Differences, ITDs) are utilized by mammals to determine the location of low-frequency sound source in the horizontal dimension [4]. Simple calculation shows that to achieve psychophysically revealed accuracy of sound localization, binaural auditory system have to detect ITDs in the order of just a few tens of microseconds. It has long been assumed that such remarkable temporal resolution is the result of ITD encoding by systematically organized arrays of fast neurons - “coincidence detectors” that receive inputs from the two ears via axons of variable length - “delay lines” and create a neural “map” of physiological ITDs. However, despite its wide acceptance, an increasing number of recent morphological and physiological studies in mammals report findings that can not be interpreted in the framework of this classical coincidence detector concept (reviewed in [2]). Therefore, neural mechanisms of the mammalian auditory system underlying high-resolution ITD detection are still not fully understood. Here, we continue to consider an alternative ITD detection mechanism based on a population coding strategy [5], according to which ITDs are encoded by the relative “activity” levels within bilaterally located populations of binaural sluggish (slow) neurons. To implement studied ITD detection mechanism we employ biologically inspired computational model that simulates the behavior of two symmetric neural populations of the auditory system. Every single neuron model of each population has membrane time constant of >1 ms and receives the balanced excitatory-inhibitory (EI) synaptic inputs carrying ITD. The parameters of neuron [3] and synapse [1] models are turned to mimic pure trough-type activity of EI neurons, i.e., no (or minimal) responses occur when “ipsilateral” excitation and “contralateral” inhibition are near coincidence. Our previous theoretical and modeling studies have shown that precise distribution (oppositely directed linear gradients) of the conductance value of excitatory and inhibitory synapses within each population allows the proposed model to detect ITD with submillisecond precision. It was also revealed that presence of jitter (related to the dynamic travel time deviation of ipsi- and contralateral signals) or neuronal noise from the physiological range does not have dramatic influence on ITD detection accuracy. In this work, we focus on the further analysis of stochastic and noise processes in the auditory system and present a computational evidence indicating the necessity to reassess the role of stochasticity in ITD processing. We demonstrate that microsecond order resolution of ITD detection, based on the population coding mechanism, can be accomplished by the auditory system without exquisite tunings of synapses or/and axon lengths. It is also shown that biologically relevant levels of both intrinsic neuronal noise and random time jitter in the synaptic input underlie ITD detection acuity similar to that observed in the model with a precise as well as a random balance of excitatory and inhibitory synaptic inputs. Thus, the results suggest that the considered stochastic processes can play crucial role as complementary/replaceable sources of “perturbation” for the sluggish neuron based populations; that is needed for high-resolution ITD detection.