Interaural Time Differences Research Articles

Lateralization of binaural envelope cues measured with a mobile cochlear-implant research processora).

Bilateral cochlear implant (BICI) listeners do not have full access to the binaural cues that normal hearing (NH) listeners use for spatial hearing tasks such as localization. When using their unsynchronized everyday processors, BICI listeners demonstrate sensitivity to interaural level differences (ILDs) in the envelopes of sounds, but interaural time differences (ITDs) are less reliably available. It is unclear how BICI listeners use combinations of ILDs and envelope ITDs, and how much each cue contributes to perceived sound location. The CCi-MOBILE is a bilaterally synchronized research processor with the untested potential to provide spatial cues to BICI listeners. In the present study, the CCi-MOBILE was used to measure the ability of BICI listeners to perceive lateralized sound sources when single pairs of electrodes were presented amplitude-modulated stimuli with combinations of ILDs and envelope ITDs. Young NH listeners were also tested using amplitude-modulated high-frequency tones. A cue weighting analysis with six BICI and ten NH listeners revealed that ILDs contributed more than envelope ITDs to lateralization for both groups. Moreover, envelope ITDs contributed to lateralization for NH listeners but had negligible contribution for BICI listeners. These results suggest that the CCi-MOBILE is suitable for binaural testing and developing bilateral processing strategies.

Closed-loop sound source localization in neuromorphic systems

Sound source localization (SSL) is used in various applications such as industrial noise-control, speech detection in mobile phones, speech enhancement in hearing aids and many more. Newest video conferencing setups use SSL. The position of a speaker is detected from the difference in the audio waves received by a microphone array. After detection the camera focuses onto the location of the speaker. The human brain is also able to detect the location of a speaker from auditory signals. It uses, among other cues, the difference in amplitude and arrival time of the sound wave at the two ears, called interaural level and time difference. However, the substrate and computational primitives of our brain are different from classical digital computing. Due to its low power consumption of around 20 W and its performance in real time the human brain has become a great source of inspiration for emerging technologies. One of these technologies is neuromorphic hardware which implements the fundamental principles of brain computing identified until today using complementary metal-oxide-semiconductor technologies and new devices. In this work we propose the first neuromorphic closed-loop robotic system that uses the interaural time difference for SSL in real time. Our system can successfully locate sound sources such as human speech. In a closed-loop experiment, the robotic platform turned immediately into the direction of the sound source with a turning velocity linearly proportional to the angle difference between sound source and binaural microphones. After this initial turn, the robotic platform remains at the direction of the sound source. Even though the system only uses very few resources of the available hardware, consumes around 1 W, and was only tuned by hand, meaning it does not contain any learning at all, it already reaches performances comparable to other neuromorphic approaches. The SSL system presented in this article brings us one step closer towards neuromorphic event-based systems for robotics and embodied computing.

Binaural Processing and Auditory Working Memory in Individuals with Tinnitus Having Normal Hearing Sensitivity.

Tinnitus is a sound perceived in the ears or head without any external or internal sound source. It can be hypothesized that the auditory processing at the different levels of the auditory pathway and working memory may get affected due to the additional sound contributed by tinnitus. The objective of our study is to evaluate binaural processes and working memory capacity in individual with tinnitus having normal hearing sensitivity. The binaural interaction process was evaluated using the interaural time difference and interaural level difference. Similarly, the binaural integration process was assessed using the dichotic consonant-vowel test, and the working memory was assessed using the audio descending span test and Audio 2N-Back span test. Mann-Whitney U test was done to see the significant difference between the groups. The Mann-Whitney U test showed significantly poorer interaural level difference scores in the tinnitus group. Also, dichotic consonantvowel test scores and auditory working memory test scores were found to be significantly lower in individuals with tinnitus. The binaural processes, along with the working memory capacity, are found to be affected in an individual with tinnitus, which in turn may affect the speech perception ability of the individual.

Antlion larvae localize long distant preys by a mechanism based on time difference.

Pit building antlions Euroleon nostras have been submitted to artificial cues in order to delineate their faculty to localize a prey. Series of propagating pulses in sand have been created from an extended source made of 10 piezoelectric transducers equally spaced on a line and located at a large distance from the pit. The envelope of each pulse encompasses six oscillations at a carrier frequency of 1250 Hz and up to eight oscillations at 1666 Hz. In one set of experiments, the first wave front is followed by similar wave fronts and the antlions respond to the cue by throwing sand in the opposite direction of the wave front propagation direction. In another set of experiments, the first wave front is randomly spatially structured while the propagation of the wave fronts inside the envelope of the pulse are not. In that case, the antlions respond less to the cue by throwing sand, and when they do, their sand throwing is more randomly distributed in direction. The finding shows that the localization of vibration signal by antlions are based on the equivalent for hearing animals of interaural time difference in which the onset has more significance than the interaural phase difference.

Development of frequency tuning shaped by spatial cue reliability in the barn owl's auditory midbrain.

Sensory systems preferentially strengthen responses to stimuli based on their reliability at conveying accurate information. While previous reports demonstrate that the brain reweighs cues based on dynamic changes in reliability, how the brain may learn and maintain neural responses to sensory statistics expected to be stable over time is unknown. The barn owl's midbrain features a map of auditory space where neurons compute horizontal sound location from the interaural time difference (ITD). Frequency tuning of midbrain map neurons correlates with the most reliable frequencies for the neurons' preferred ITD (Cazettes et al., 2014). Removal of the facial ruff led to a specific decrease in the reliability of high frequencies from frontal space. To directly test whether permanent changes in ITD reliability drive frequency tuning, midbrain map neurons were recorded from adult owls, with the facial ruff removed during development, and juvenile owls, before facial ruff development. In both groups, frontally tuned neurons were tuned to frequencies lower than in normal adult owls, consistent with the change in ITD reliability. In addition, juvenile owls exhibited more heterogeneous frequency tuning, suggesting normal developmental processes refine tuning to match ITD reliability. These results indicate causality of long-term statistics of spatial cues in the development of midbrain frequency tuning properties, implementing probabilistic coding for sound localization.

Structure and dynamics that specialize neurons for high-frequency coincidence detection in the barn owl nucleus laminaris.

A principal cue for sound source localization is the difference in arrival times of sounds at an animal's two ears (interaural time difference, ITD). Neurons that process ITDs are specialized to compare the timing of inputs with submillisecond precision. In the barn owl, ITD processing begins in the nucleus laminaris (NL) region of the auditory brain stem. Remarkably, NL neurons are sensitive to ITDs in high-frequency sounds (kilohertz-range). This contrasts with ITD-based sound localization in analogous regions in mammals where ITD sensitivity is typically restricted to lower-frequency sounds. Guided by previous experiments and modeling studies of tone-evoked responses of NL neurons, we propose NL neurons achieve high-frequency ITD sensitivity if they respond selectively to the small-amplitude, high-frequency oscillations in their inputs, and remain relatively non-responsive to mean input level. We use a biophysically based model to study the effects of soma-axon coupling on dynamics and function in NL neurons. First, we show that electrical separation of the soma from the axon region in the neuron enhances high-frequency ITD sensitivity. This soma-axon coupling configuration promotes linear subthreshold dynamics and rapid spike initiation, making the model more responsive to input oscillations, rather than mean input level. Second, we provide new evidence for the essential role of phasic dynamics for high-frequency neural coincidence detection. Transforming our model to the phasic firing mode further tunes the model to respond selectively to the oscillating inputs that carry ITD information. Similar structural and dynamical mechanisms specialize mammalian auditory brain stem neurons for ITD sensitivity, and thus, our work identifies common principles of ITD processing and neural coincidence detection across species and for sounds at widely different frequencies.

Principal neuron diversity in the murine lateral superior olive supports multiple sound localization strategies and segregation of information in higher processing centers

Principal neurons (PNs) of the lateral superior olive nucleus (LSO) in the brainstem of mammals compare information between the two ears and enable sound localization on the horizontal plane. The classical view of the LSO is that it extracts ongoing interaural level differences (ILDs). Although it has been known for some time that LSO PNs have intrinsic relative timing sensitivity, recent reports further challenge conventional thinking, suggesting the major function of the LSO is detection of interaural time differences (ITDs). LSO PNs include inhibitory (glycinergic) and excitatory (glutamatergic) neurons which differ in their projection patterns to higher processing centers. Despite these distinctions, intrinsic property differences between LSO PN types have not been explored. The intrinsic cellular properties of LSO PNs are fundamental to how they process and encode information, and ILD/ITD extraction places disparate demands on neuronal properties. Here we examine the ex vivo electrophysiology and cell morphology of inhibitory and excitatory LSO PNs in mice. Although overlapping, properties of inhibitory LSO PNs favor time coding functions while those of excitatory LSO PNs favor integrative level coding. Inhibitory and excitatory LSO PNs exhibit different activation thresholds, potentially providing further means to segregate information in higher processing centers. Near activation threshold, which may be physiologically similar to the sensitive transition point in sound source location for LSO, all LSO PNs exhibit single-spike onset responses that can provide optimal time encoding ability. As stimulus intensity increases, LSO PN firing patterns diverge into onset-burst cells, which can continue to encode timing effectively regardless of stimulus duration, and multi-spiking cells, which can provide robust individually integrable level information. This bimodal response pattern may produce a multi-functional LSO which can encode timing with maximum sensitivity and respond effectively to a wide range of sound durations and relative levels.

Interaural time difference sensitivity under binaural cochlear implant stimulation persists at high pulse rates up to 900 pps

Spatial hearing remains one of the major challenges for bilateral cochlear implant (biCI) users, and early deaf patients in particular are often completely insensitive to interaural time differences (ITDs) delivered through biCIs. One popular hypothesis is that this may be due to a lack of early binaural experience. However, we have recently shown that neonatally deafened rats fitted with biCIs in adulthood quickly learn to discriminate ITDs as well as their normal hearing litter mates, and perform an order of magnitude better than human biCI users. Our unique behaving biCI rat model allows us to investigate other possible limiting factors of prosthetic binaural hearing, such as the effect of stimulus pulse rate and envelope shape. Previous work has indicated that ITD sensitivity may decline substantially at the high pulse rates often used in clinical practice. We therefore measured behavioral ITD thresholds in neonatally deafened, adult implanted biCI rats to pulse trains of 50, 300, 900 and 1800 pulses per second (pps), with either rectangular or Hanning window envelopes. Our rats exhibited very high sensitivity to ITDs at pulse rates up to 900 pps for both envelope shapes, similar to those in common clinical use. However, ITD sensitivity declined to near zero at 1800 pps, for both Hanning and rectangular windowed pulse trains. Current clinical cochlear implant (CI) processors are often set to pulse rates ≥ 900 pps, but ITD sensitivity in human CI listeners has been reported to decline sharply above ~ 300 pps. Our results suggest that the relatively poor ITD sensitivity seen at > 300 pps in human CI users may not reflect the hard upper limit of biCI ITD performance in the mammalian auditory pathway. Perhaps with training or better CI strategies good binaural hearing may be achievable at pulse rates high enough to allow good sampling of speech envelopes while delivering usable ITDs.

Doppler Frequency-Shift Information Processing in WOx -Based Memristive Synapse for Auditory Motion Perception.

Auditory motion perception is one crucial capability to decode and discriminate the spatiotemporal information for neuromorphic auditory systems. Doppler frequency-shift feature and interaural time difference (ITD) are two fundamental cues of auditory information processing. In this work, the functions of azimuth detection and velocity detection, as the typical auditory motion perception, are demonstrated in a WOx -based memristive synapse. The WOx memristor presents both the volatile mode (M1) and semi-nonvolatile mode (M2), which are capable of implementing the high-pass filtering and processing the spike trains with a relative timing and frequency shift. In particular, the Doppler frequency-shift information processing for velocity detection is emulated in the WOx memristor based auditory system for the first time, which relies on a scheme of triplet spike-timing-dependent-plasticity in the memristor. These results provide new opportunities for the mimicry of auditory motion perception and enable the auditory sensory system to be applied in future neuromorphic sensing.

Is the rapid decline in interaural time difference sensitivity above 700 Hz explained by downward spread of excitation into the frequency dominant region?

Low-frequency temporal-fine-structure interaural time differences (ITDs) are so important that they almost completely dominate localization of broadband sounds. This “Dominant Region” occurs at ∼700 Hz and also coincides with best ITD discrimination sensitivity. Above this frequency range, ITD sensitivity worsens, until ∼1400 Hz where it rapidly becomes impossible. Explaining how listeners’ ITD sensitivity transitions from best sensitivity to impossible to detect within ∼1 octave challenges models of ITD processing. One possible explanation is that listeners continue to use binaural information from the Dominant Region even for higher-frequency pure tones. Within this explanation, the steep decline in ITD sensitivity reflects the high-frequency limb of peripheral frequency tuning for neurons within the Dominant Region (i.e., downward spread of excitation). To test this hypothesis, we measured the upper frequency limit of ITD processing as a function of stimulus level. Normal-hearing listeners were tested on a left-right ITD discrimination task. Stimuli were pure tones with 500–1500Hz frequencies and 10–50 dB sensation levels. It was hypothesized that the upper frequency limit of ITD processing would decrease in frequency with decreasing level, which would indicate that the downward spread of excitation into the Dominant Region at least partially explains the rapid decline in ITD sensitivity.

Lateralization of interaural time differences with mixed rates of stimulation in bilateral cochlear implant listeners.

While listeners with bilateral cochlear implants (BiCIs) are able to access information in both ears, they still struggle to perform well on spatial hearing tasks when compared to normal hearing listeners. This performance gap could be attributed to the high stimulation rates used for speech representation in clinical processors. Prior work has shown that spatial cues, such as interaural time differences (ITDs), are best conveyed at low rates. Further, BiCI listeners are sensitive to ITDs with a mixture of high and low rates. However, it remains unclear whether mixed-rate stimuli are perceived as unitary percepts and spatially mapped to intracranial locations. Here, electrical pulse trains were presented on five, interaurally pitch-matched electrode pairs using research processors, at either uniformly high rates, low rates, or mixed rates. Eight post-lingually deafened adults were tested on perceived intracranial lateralization of ITDs ranging from 50 to 1600 μs. Extent of lateralization depended on the location of low-rate stimulation along the electrode array: greatest in the low- and mixed-rate configurations, and smallest in the high-rate configuration. All but one listener perceived a unitary auditory object. These findings suggest that a mixed-rate processing strategy can result in good lateralization and convey a unitary auditory object with ITDs.

Relating spatial release from masking to lateralization using interaural differences

Spatial acoustic cues such as interaural differences in time (ITD) and level (ILD) contribute to the benefit of reduced interference (masking) that occurs when competing sounds (maskers) differ in spatial location from the target talker. This benefit (spatial release from masking or SRM) is well studied but the contributions of ITD and ILD to SRM during speech-on-speech masking remain poorly understood. One hypothesis regarding mixed results in the literature is that SRM primarily reflects differences in overall perceived spatial location rather than independent contributions of ITD and ILD. We evaluated this hypothesis by relating SRM directly to the perceived spatial locations of speech stimuli that were processed in five conditions: ITD only, ILD only, consistent (ITD + ILD), and opposing (ITD – ILD or ILD – ITD). We measured lateralization for speech stimuli in each condition. Lateralization was consistently strongest for consistent ITD + ILD cues, but weaker and more variable across other conditions. Finally, we evaluate a model relating individual differences in SRM to ITD/ILD-based lateralization of target and masker sounds. [Work supported by NIH R01-DC016643]

Wave interference at the contralateral ear helps explain non-monotonic envelope interaural time differences as a function of azimuth.

Interaural time differences (ITDs), an important acoustic cue for perceptual sound-source localization, are conventionally modeled as monotonic functions of azimuth. However, recent literature and publicly available databases from binaural manikins demonstrated ITDs conveyed by the envelopes (ENV-ITDs) of high-frequency (≥2 kHz) signals that were non-monotonic functions of azimuth. This study demonstrates using a simple, time-dependent geometric model of an elliptic head that the back-traveling (longer) sound path around the head, delayed and added to the conventionally treated front-traveling path, can account for non-monotonic ENV-ITDs. These findings have implications for spatial-hearing models in acoustic and electric (cochlear-implant) hearing.

Heptuna: The most outstanding subject

One of the best experimental collaborators I have ever worked with was HEP, a bottlenose dolphin (Tursiops truncatus). HEP was instrumental in leading the way for subsequent dolphin investigations. In the first study on dolphin sound localization, HEP was the willing subject [D. L. Renaud and A. N. Popper, “Sound localization by the bottlenose porpoise Tursiops truncatus,” J. Exp. Biol. 63, 569–585(1975)]. Thus began a series of critical experiments that set the foundations of cetacean hearing and echolocation. I had the pleasure to partner with Whit Au and HEP to explore the many aspects of dolphin hearing and echolocation. HEP carried the experience of his training and testing to new experiments. If he saw just one response manipulandum, he seemed to know it was a go/no go response requirement, or if two, a forced choice response. HEP was the first animal to use “jaw phones” to test hearing, and interaural time and intensity differences. He demonstrated he could control his echolocation clicks, he provided measures of his transmitting and receiving beam patterns, and much more. Unfortunately, we lost HEP in 2010 after a 40-year history with the U.S. Navy Marine Mammal Program.

Quantifying the impact of hearing-aid microphone placement and head-mounted devices on binaural cues from head-related transfer functions

Emerging techniques for assessment and training of spatial hearing use virtual reality (VR) devices to present auditory-visual cues. Sounds may be processed using head-related transfer functions (HRTFs), which capture directional acoustic cues, and visuals may be presented using a head-mounted display (HMD). For users of hearing devices such as hearing aids or cochlear implants, microphone placement may alter the binaural cues carried by the HRTF. Cues may also be distorted in presentations that combine HMD visuals with loudspeaker audio. This work seeks to understand these impacts by measuring and creating a database of HRTFs in the presence of hearing aids and HMDs. HRTFs were recorded within an anechoic chamber, using a standard manikin in a loudspeaker array located on the horizontal plane. Recordings were made with real sound sources of 5.625° resolution and virtual sources of 1° resolution. We will compare interaural time and level differences (ITDs and ILDs) from HRTFs measured through hearing-aid microphones with varied microphone placement (i.e., behind-the-ear and in-the-ear) and the presence versus absence of an HMD during the HRTF recording. Results will provide insights regarding HRTFs for individuals with hearing devices, as well as characterization of the HMD impact on VR-guided individual HRTF measurement.

The Prevalence of Hearing Aid Use in Children With Single-Sided Sensorineural Deafness in Oxfordshire

The Prevalence of Hearing Aid Use in Children With Single-Sided Sensorineural Deafness in Oxfordshire

Aging alters across-hemisphere cortical dynamics during binaural temporal processing.

Differences in the timing and intensity of sounds arriving at the two ears provide fundamental binaural cues that help us localize and segregate sounds in the environment. Neural encoding of these cues is commonly represented asymmetrically in the cortex with stronger activation in the hemisphere contralateral to the perceived spatial location. Although advancing age is known to degrade the perception of binaural cues, less is known about how the neural representation of such cues is impacted by age. Here, we use electroencephalography (EEG) to investigate age-related changes in the hemispheric distribution of interaural time difference (ITD) encoding based on cortical auditory evoked potentials (CAEPs) and derived binaural interaction component (BIC) measures in ten younger and ten older normal-hearing adults. Sensor-level analyses of the CAEP and BIC showed age-related differences in global field power, where older listeners had significantly larger responses than younger for both binaural metrics. Source-level analyses showed hemispheric differences in auditory cortex activity for left and right lateralized stimuli in younger adults, consistent with a contralateral activation model for processing ITDs. Older adults, however, showed reduced hemispheric asymmetry across ITDs, despite having overall larger responses than younger adults. Further, when averaged across ITD condition to evaluate changes in cortical asymmetry over time, there was a significant shift in laterality corresponding to the peak components (P1, N1, P2) in the source waveform that also was affected by age. These novel results demonstrate across-hemisphere cortical dynamics during binaural temporal processing that are altered with advancing age.

Temporal hyper-precision of brainstem neurons alters spatial sensitivity of binaural auditory processing with cochlear implants.

The ability to localize a sound source in complex environments is essential for communication and navigation. Spatial hearing relies predominantly on the comparison of differences in the arrival time of sound between the two ears, the interaural time differences (ITDs). Hearing impairments are highly detrimental to sound localization. While cochlear implants (CIs) have been successful in restoring many crucial hearing capabilities, sound localization via ITD detection with bilateral CIs remains poor. The underlying reasons are not well understood. Neuronally, ITD sensitivity is generated by coincidence detection between excitatory and inhibitory inputs from the two ears performed by specialized brainstem neurons. Due to the lack of electrophysiological brainstem recordings during CI stimulation, it is unclear to what extent the apparent deficits are caused by the binaural comparator neurons or arise already on the input level. Here, we use a bottom-up approach to compare response features between electric and acoustic stimulation in an animal model of CI hearing. Conducting extracellular single neuron recordings in gerbils, we find severe hyper-precision and moderate hyper-entrainment of both the excitatory and inhibitory brainstem inputs to the binaural comparator neurons during electrical pulse-train stimulation. This finding establishes conclusively that the binaural processing stage must cope with highly altered input statistics during CI stimulation. To estimate the consequences of these effects on ITD sensitivity, we used a computational model of the auditory brainstem. After tuning the model parameters to match its response properties to our physiological data during either stimulation type, the model predicted that ITD sensitivity to electrical pulses is maintained even for the hyper-precise inputs. However, the model exhibits severely altered spatial sensitivity during electrical stimulation compared to acoustic: while resolution of ITDs near midline was increased, more lateralized adjacent source locations became inseparable. These results directly resemble recent findings in rodent and human CI listeners. Notably, decreasing the phase-locking precision of inputs during electrical stimulation recovered a wider range of separable ITDs. Together, our findings suggest that a central problem underlying the diminished ITD sensitivity in CI users might be the temporal hyper-precision of inputs to the binaural comparator stage induced by electrical stimulation.

A database of near-field head-related transfer functions based on measurements with a laser spark source

This paper presents a database of near-field head-related transfer functions (HRTFs) of an artificial head, measured at four distances (0.2, 0.3, 0.4 and 0.5 m), with 49 positions recorded at each distance, for a total of 196 measurement points. The HRTFs were recorded using an acoustic pulse created by a laser-induced breakdown of air (LIB), which realizes a close to ideal, massless, monopole sound source. While the LIB produces a high amplitude pressure pulse, the amplitude decays toward low frequencies, which introduced a low frequency limit of about 200Hz for this particular setup. Thus, a spherical head model based on the analytical expression for scattering by a rigid sphere was fitted to the measured data, and used to extend the low frequency range of the measurements. A brief evaluation of the processed dataset was undertaken, considering interaural time and level differences. The measured and processed database, as well as the low frequency extension procedure are made publicly available to support future research into nearby sound localization, and virtual/augmented reality applications.

Neonatal Deafening Selectively Degrades the Sensitivity to Interaural Time Differences of Electrical Stimuli in Low-Frequency Pathways in Rats.

We examined the effect of neonatal deafening on frequency-specific pathways for processing of interaural time differences (ITDs) in cochlear-implant stimuli. Animal studies have demonstrated differences in neural ITD sensitivity in the inferior colliculus (IC) depending on the intracochlear location of intracochlear stimulating electrodes. We used neonatally deafened (ND) rats of both sexes and recorded the responses of single neurons in the IC to electrical stimuli with ITDs delivered to the apical or basal cochlea and compared them with acutely deafened (AD) rats of both sexes with normal hearing (NH) during development. We found that neonatal deafness significantly impacted the ITD sensitivity and the ITD tuning patterns restricted to apically driven IC neurons. In ND rats, the ITD sensitivity of apically driven neurons is reduced to values similar to basally driven neurons. The prevalence of ITD-sensitive apical neurons with a peak-shaped ITD tuning curve, which may reflect predominant input from the medial superior olivary (MSO) complex, in ND rats was diminished compared with that in AD rats (67%, AD vs 40%, ND). Conversely, monotonic-type responses rarely occurred in AD rats (14%) but were approximately equally as prevalent as peak-type tuning curves in ND rats (42%). Nevertheless, in ND rats, the ITD at the maximum slope of the ITD tuning curve was still more concentrated within the physiological ITD range in apically driven than in basally driven neurons. These results indicate that the development of high ITD sensitivity processed by low-frequency pathways depends on normal auditory experience and associated biases in ITD tuning strategies.