Interaural Phase Research Articles

Monaural and binaural masking release with speech-like stimuli.

The relevance of comodulation and interaural phase difference for speech perception is still unclear. We used speech-like stimuli to link spectro-temporal properties of formants with masking release. The stimuli comprised a tone and three masker bands centered at formant frequencies F1, F2, and F3 derived from a consonant-vowel. The target was a diotic or dichotic frequency-modulated tone following F2 trajectories. Results showed a small comodulation masking release, while the binaural masking level difference was comparable to previous findings. The data suggest that factors other than comodulation may play a dominant role in grouping frequency components in speech.

Leak detection and localization in underground water supply system using thermal imaging and geophone signals through machine learning

The underground water pipeline system is a crucial infrastructure that largely remains out of sight. However, it is the source of a clean and uninterrupted flow of water for our everyday lives. Various factors, including corrosion, material degradation, ground movement, and improper maintenance, cause pipe leaks, a silent crisis that causes an estimated 39 billion dollars of loss every year. Prompt leakage detection and localization can help reduce the loss. This research investigates the potential of two machine learning models as supporting tools for surveying extensive areas to identify and pinpoint the location of underground leaks. The presented combined approach ensures the speed and accuracy of the leakage survey. The first machine learning model is a hybrid ML model that employs thermal imaging to identify subterranean water leakage. It relies on detecting thermal anomalies and distinctive signatures associated with water leakage to identify and locate underground water leakage. The developed model can detect up to 750 mm underground leakage with 95.20 % accuracy. The second model uses binaural audio from geophones to localize the leakage position. The model utilizes interaural time difference and interaural phase difference for localization purposes, and the 1D-CNN network delivers an angle in twenty-degree increments with an accuracy of 88.19 %. Large-scale implementation of the proposed model could be a powerful catalyst to reduce water loss in the water supply system.

Midbrain sensitivity to auditory motion studied with dichotic sweeps of broadband noise

Many neurons in the central nucleus of the inferior colliculus (IC) show sensitivity to interaural time differences (ITDs), which is thought to be relayed from the brainstem. However, studies with interaural phase modulation of pure tones showed that IC neurons have a sensitivity to changes in ITD that is not present at the level of the brainstem. This sensitivity has been interpreted as a form of sensitivity to motion.A new type of stimulus is used here to study the sensitivity of IC neurons to dynamic changes in ITD, in which broad- or narrowband stimuli are swept through a range of ITDs with arbitrary start-ITD, end-ITD, speed, and direction. Extracellular recordings were obtained under barbiturate anesthesia in the cat. We applied the same analyses as previously introduced for the study of responses to tones.We find effects of motion which are similar to those described in response to interaural phase modulation of tones. The size of the effects strongly depended on the motion parameters but was overall smaller than reported for tones. We found that the effects of motion could largely be explained by the temporal response pattern of the neuron such as adaptation and build-up. Our data add to previous evidence questioning true coding of motion at the level of the IC.

Objective measure of binaural processing: Acoustic change complex in response to interaural phase differences

Combining cochlear implants with binaural acoustic hearing via preserved hearing in the implanted ear(s) is commonly referred to as combined electric and acoustic stimulation (EAS). EAS fittings can provide patients with significant benefit for speech recognition in complex noise, perceived listening difficulty, and horizontal-plane localization as compared to traditional bimodal hearing conditions with contralateral and monaural acoustic hearing. However, EAS benefit varies across patients and the degree of benefit is not reliably related to the underlying audiogram. Previous research has indicated that EAS benefit for speech recognition in complex listening scenarios and localization is significantly correlated with the patients’ binaural cue sensitivity, namely interaural time differences (ITD). In the context of pure tones, interaural phase differences (IPD) and ITD can be understood as two perspectives on the same phenomenon. Through simple mathematical conversion, one can be transformed into the other, illustrating their inherent interrelation for spatial hearing abilities. However, assessing binaural cue sensitivity is not part of a clinical assessment battery as psychophysical tasks are time consuming, require training to achieve performance asymptote, and specialized programming and software all of which render this clinically unfeasible. In this study, we investigated the possibility of using an objective measure of binaural cue sensitivity by the acoustic change complex (ACC) via imposition of an IPD of varying degrees at stimulus midpoint. Ten adult listeners with normal hearing were assessed on tasks of behavioral and objective binaural cue sensitivity for carrier frequencies of 250 and 1000 Hz. Results suggest that 1) ACC amplitude increases with IPD; 2) ACC-based IPD sensitivity for 250 Hz is significantly correlated with behavioral ITD sensitivity; 3) Participants were more sensitive to IPDs at 250 Hz as compared to 1000 Hz. Thus, this objective measure of IPD sensitivity may hold clinical application for pre- and post-operative assessment for individuals meeting candidacy indications for cochlear implantation with low-frequency acoustic hearing preservation as this relatively quick and objective measure may provide clinicians with information identifying patients most likely to derive benefit from EAS technology.

Estimating the high-frequency extent of binaural sensitivity to temporal fine structure across listeners

Füllgrabe et al. [2017, Int. J. Audiol. 56:926-35] developed a novel procedure to estimate the frequency dependence of listeners’ sensitivity to temporal fine structure in the form of interaural phase differences (IPD). Whereas traditional approaches fixed tone frequency and varied IPD, the new “TFS-AF” test fixed the IPD and varied frequencies adaptively so as to estimate the upper frequency limit (UFL) of IPD sensitivity, even in listeners who struggle with traditional tests. UFL estimates for young normal hearing listeners were consistently 1200–1400 Hz, with notable exceptions of lower UFL (e.g. 800–1000 Hz) in a small subset. The consistency of UFL and abrupt loss of IPD sensitivity above it remain among the most puzzling aspects of binaural hearing, unexplained by the (rather shallow) frequency dependence of audibility, neuronal phase locking, or internal noise in this region. Recently, our group adapted a version of the TFS-AF test to measure UFL as a function of sound level, comparing the resulting slope to predictions of candidate mechanisms. The limitation closely matches the upper edge of an auditory filter centered at ∼700 Hz, suggesting that IPD sensitivity is mediated exclusively by neurons tuned to that frequency or below. [Work supported by NIH R01DC014948, R01DC016643, and R01DC017924.]

Is Recognition of Speech in Noise Related to Memory Disruption Caused by Irrelevant Sound?

Listeners with normal audiometric thresholds show substantial variability in their ability to understand speech in noise (SiN). These individual differences have been reported to be associated with a range of auditory and cognitive abilities. The present study addresses the association between SiN processing and the individual susceptibility of short-term memory to auditory distraction (i.e., the irrelevant sound effect [ISE]). In a sample of 67 young adult participants with normal audiometric thresholds, we measured speech recognition performance in a spatial listening task with two interfering talkers (speech-in-speech identification), audiometric thresholds, binaural sensitivity to the temporal fine structure (interaural phase differences [IPD]), serial memory with and without interfering talkers, and self-reported noise sensitivity. Speech-in-speech processing was not significantly associated with the ISE. The most important predictors of high speech-in-speech recognition performance were a large short-term memory span, low IPD thresholds, bilaterally symmetrical audiometric thresholds, and low individual noise sensitivity. Surprisingly, the susceptibility of short-term memory to irrelevant sound accounted for a substantially smaller amount of variance in speech-in-speech processing than the nondisrupted short-term memory capacity. The data confirm the role of binaural sensitivity to the temporal fine structure, although its association to SiN recognition was weaker than in some previous studies. The inverse association between self-reported noise sensitivity and SiN processing deserves further investigation.

Degradation of speech reception performance in competing sounds in middle-aged adults and its factor

Speech reception in the presence of competing sounds declines in middle age. Here we examined how energetic and informational masking degrades speech reception in middle-aged adults, and further explored how the performance degradation depends on temporal processing efficiency and working memory capacity. We measured the intelligibility of a target phrase masked by noise or a competing masker phrase, which was spoken by the same talker, same-sex talker, or different-sex talker. Temporal processing was assessed by interaural phase difference thresholds and frequency modulation detection limen. Working memory capacity was assessed by the listening span test. Speech reception performance was lower in middle-aged adults than young adults, regardless of the type of disturbing sounds. The performance differences between the same-talker and different-sex talker conditions, which reflects the effects of informational masking, was larger in middle-aged adults. These results suggest that both energetic and informational masking contributes to the degradation of speech reception in middle-aged adults. The correlation analysis for the middle-aged adults further revealed that the contribution of working memory capacity was larger in the same-talker condition, where informational masking was strong, whereas the contribution of temporal processing efficiency was larger in the noise condition, where energetic masking was dominant.

Association between interaural phase sensitivity and performance on the trail making cognitive test

Sensitivity to interaural differences in time or phase critically supports human abilities to spatially localize sounds in the horizontal plane. The trail making test (TMT) is a non-auditory cognitive test that provides information about visual search speed, visual/spatial attention, and executive function. Surprisingly, we have shown strong positive relationships between interaural phase difference (IPD) sensitivity and TMT performance, both in a sample (n = 20) of older adults (48–85 years) with very similar sensorineural hearing loss profiles [Strelcyk et al., 23, 2331216519864499 (2019)], and in a sample (n = 41) of young to middle-aged adults (21–54 years) with normal audiograms [Shehorn et al., Hear. Res. 392, 107982 (2020)]. The fact that associations between IPD sensitivity and TMT performance were observed in both samples suggests that the associations were not related to audiometric sensitivity. Alternative explanations for the associations will be discussed, including potential higher-order relationships with spatial processing abilities and/or cognitive processing speed.

Potential objective measures to assess the presence of hidden hearing loss in young adults with long-term exposure to loud music

Several objective measures for identifying individuals with “hidden hearing loss” (HHL), have been proposed based on cochlear synaptopathy and the resulting central changes in neural gain. While the loss of high-threshold auditory nerve fibres may result in weaker middle-ear muscle reflexes (MEMR) in HHL sufferers, binaural processing is likely disproportionately and deleteriously impacted by central gain changes in the same group of individuals. In this study, two groups of young adults (<35 years old) with normal audiometric thresholds were recruited, one of which consisted of students from a music conservatory who co-reported tinnitus and listening-in-noise difficulties, likely as a result of HHL. Participants completed a set of questionnaires as well as undergoing both multi-frequency MEMR testing (four ipsilateral elicitors), and an electrophysiological measure of IPD processing known as the ‘Interaural Phase Modulation-Following-Response(“IPM-FR”) (evoked using two IPD transitions using a clinical setup). Results show that the ‘HHL/Tinnitus’ group presents lower scores on the Speech, Spatial and Qualities of Hearing Scale questionnaire, less steep MEMR growth functions, and smaller IPM-FRs compared to the ‘control’ group. This suggests that both objective measures could be clinically useful to identify populations with HHL.

Analysis of binaural sound pressure control and virtual source characteristics in stereophonic reproduction

In stereophonic reproduction, the manipulation of relative magnitude and phase between two loudspeakers enables to create virtual sources within an azimuth region in frontal–horizontal plane. The law of sine, which is based on evaluating the interaural phase difference and a conventional method for stereophonic analysis at low frequencies, fails to explain some characteristic in stereophonic reproduction, including the influence of span angle of loudspeakers on virtual source localization and the stability of virtual source created by out-of-phase loudspeaker signals. Analogy to the method in active sound control, an alternative method for analyzing the characteristics of virtual source in stereophonic reproduction is presented in this work. On the basis of shadowless head model at low frequency, the power effort of loudspeaker signals for creating virtual source in stereophonic reproduction is evaluated. The results indicate that for frontal virtual source within the span of loudspeakers, the power effort of in-phase loudspeaker signals increases with the increase in the span angle of loudspeakers. For lateral virtual source outside the span of loudspeakers, the power effort of out-of-phase loudspeaker signals increases with the target azimuth of virtual source. Increase in power effort makes the control of binaural pressures difficulty and, thus, virtual source instable.

Towards a yardstick of spatial impression: Psychophysical experiments using virtual piano stimuli

A rich “spatial impression” is an important factor in the product value of musical instruments. However, it also represents a development challenge since there is no established method to measure and control it. We introduce a novel yardstick for spatial impression, changing tacit knowledge into a measurable property. We focused on the difference in spatial impression between digital pianos and grand pianos, and conducted psychophysical experiments using piano sounds to identify the features that contribute to grand pianos’ spatial impression. We used multiple virtual stimuli which are created by varying physical parameters and conducted the experiment in a virtual environment including head-tracking from a VR headset. This allowed us to find a trend in the spatial impression and identify a physical parameter that correlates to the perceptual results of the experiment. This parameter, “phase entropy,” denotes the time complexity of the interaural phase difference. The correlation coefficient between the psychophysical experiment results based on virtual stimuli and the phase entropy was 0.8752. In addition to the virtual stimuli, we confirmed the correlation between perceptual score and phase entropy in an experiment using actual piano recordings; the correlation coefficient in this case was 0.6500.

Method for increasing lateral perceived azimuth of binaural reproduction over two loudspeakers

In binaural reproduction over loudspeakers (BROL), at the target azimuth beyond the span angle of loudspeakers, the lateral perceived azimuth is usually smaller than the corresponding target azimuth, which is a major intrinsic limitation of BROL. To increase the perceived azimuth of binaural reproduction over two loudspeakers, a perceptually motivated method with low-computational complexity is proposed in this study. Based on the out-of-phase characteristics between the low-frequency amplitude of the two reproduction loudspeakers when the target azimuth is outside the span angle of loudspeakers, the proposed method widens the lateral perceived azimuth by increasing the low-frequency amplitude of the contralateral loudspeaker, which is the loudspeaker on the contralateral side of the target azimuth. Moreover, the amplitude required to be increased for the contralateral loudspeaker can be obtained through velocity vector synthesis. Objective analysis indicates that the proposed method affords a greater interaural phase delay difference than the conventional BROL. Additionally, this method alleviates the lack of dynamic cues to some extent. The results of the subjective listening experiment show that the proposed method can significantly increase the lateral perceived azimuth for different stimuli without introducing additional unnatural perception.

Space-and-speaker-aware acoustic modeling with effective data augmentation for recognition of multi-array conversational speech

We propose a space-and-speaker-aware (SSA) approach to acoustic modeling (AM), denoted as SSA-AM, to improve system performances of automatic speech recognition (ASR) in distant multi-array conversational scenarios. In contrast to conventional AM which only uses spectral features from a target speaker as inputs, the inputs to SSA-AM consists of speech features from both the target and interfering speakers, which contain discriminative information from different speakers, including spatial information embedded in interaural phase differences (IPDs) between individual interfering speakers and the target speaker. In the proposed SSA-AM framework, we explore four acoustic model architectures consisting of different combinations of four neural networks, namely deep residual network, factorized time delay neural network, self-attention and residual bidirectional long short-term memory neural network. Various data augmentation techniques are adopted to expand the training data to include different options of beamformed speech obtained from multi-channel speech enhancement. Evaluated on the recent CHiME-6 Challenge Track 1, our proposed SSA-AM framework achieves consistent recognition performance improvements when compared with the official baseline acoustic models. Furthermore, SSA-AM outperforms acoustic models without explicitly using the space and speaker information. Finally, our data augmentation schemes are shown to be especially effective for compact model designs. Code is released at https://github.com/coalboss/SSA_AM.

Inter-aural separation during hearing by bilateral bone conduction stimulation

Cross-head transmission inherent in bone conduction (BC) hearing is one of the most important factors that limit the performance of BC binaural hearing compared to air conduction (AC) binaural hearing. In AC, cross-head transmission is imperceptible leading to a clear understanding of the nature and position of the sound source(s). In this study, the prominence of cross-head transmission in BC hearing is addressed using the fact that ipsilateral cochlear excitation can be canceled by controlled bilateral BC stimulation.A cancellation experiment was conducted on twenty participants with normal hearing at thirteen third-octave frequencies between 250 and 4000 Hz. Both stationary and transient BC stimulation at the mastoid was used. The technique employed multiple stages of masking enabling adjustments of the stimulation level and phase until the tones got canceled in the ipsilateral ear. In addition, the ear canal sound pressure was obtained for ipsilateral and contralateral BC stimulation in isolation, and with bilateral BC stimulation at perceptual cancellation.The inter-aural level differences of both the types of stimulations were found to be the same. Crosstalk was found to be the lowest around 2 kHz and the highest around 1 kHz. The unwrapped inter-aural phase difference from stationary signal cancellation showed an overall increase with frequency starting at around no difference (35°) at 250 Hz to reach 607° at 4 kHz. Cycle-adjusted inter-aural time difference was very low (61 µs) at 250 Hz and increased to 1.1 ms at 800 Hz before falling to 0.6 ms at 4 kHz. It was also found that the ear canal sound pressure was not cancelled at the same phase as the sound in the cochlea.

Antlion larvae localize long distant preys by a mechanism based on time difference.

Pit building antlions Euroleon nostras have been submitted to artificial cues in order to delineate their faculty to localize a prey. Series of propagating pulses in sand have been created from an extended source made of 10 piezoelectric transducers equally spaced on a line and located at a large distance from the pit. The envelope of each pulse encompasses six oscillations at a carrier frequency of 1250 Hz and up to eight oscillations at 1666 Hz. In one set of experiments, the first wave front is followed by similar wave fronts and the antlions respond to the cue by throwing sand in the opposite direction of the wave front propagation direction. In another set of experiments, the first wave front is randomly spatially structured while the propagation of the wave fronts inside the envelope of the pulse are not. In that case, the antlions respond less to the cue by throwing sand, and when they do, their sand throwing is more randomly distributed in direction. The finding shows that the localization of vibration signal by antlions are based on the equivalent for hearing animals of interaural time difference in which the onset has more significance than the interaural phase difference.

Particle-Velocity-Based Mixed-Source Sound Field Translation for Binaural Reproduction

Following the rise of virtual reality is a demand for sound field reproduction techniques that allow the user to interact and move within acoustic reproductions with six-degrees-of-freedom. To this end, a mixed-source model of near-field and far-field virtual sources has been introduced to improve the performance of sound field translation in binaural reproductions of spatial audio recordings. The previous works, however, expand the sound field in terms of the mixed sources based on sound pressure. In this paper, we develop a new mixed-source expansion based on particle velocity, which contributes to more precise reconstruction of the interaural phase difference and, therefore, contributes to improved human perception of sound localization. We represent particle velocity over space using velocity coefficients in the spherical harmonic domain, and the driving signals of the virtual mixed-sources are estimated by constructing cost functions to optimize the velocity coefficients. Compared to the state-of-the-art method, sound-pressure-based mixed-source expansion, we show through numerical simulations that the proposed particle-velocity-based mixed-source expansion has better reconstruction performance in sparse solutions, allowing for sound field translation with better perceptual immersion over a larger space. Finally, we perceptually validate the proposed method through a Multiple Stimulus with Hidden Reference and Anchor (MUSHRA) experiment for a single source scenario. The experimental results support the better perceptual immersion of the proposed method.

Neural coding of dichotic pitches in auditory midbrain.

Dichotic pitches such as the Huggins pitch (HP) and the binaural edge pitch (BEP) are perceptual illusions whereby binaural noise that exhibits abrupt changes in interaural phase differences (IPDs) across frequency creates a tonelike pitch percept when presented to both ears, even though it does not produce a pitch when presented monaurally. At the perceptual and cortical levels, dichotic pitches behave as if an actual tone had been presented to the ears, yet investigations of neural correlates of dichotic pitch in single-unit responses at subcortical levels are lacking. We tested for cues to HP and BEP in the responses of binaural neurons in the auditory midbrain of anesthetized cats by varying the expected pitch frequency around each neuron's best frequency (BF). Neuronal firing rates showed specific features (peaks, troughs, or edges) when the pitch frequency crossed the BF, and the type of feature was consistent with a well-established model of binaural processing comprising frequency tuning, internal delays, and firing rates sensitive to interaural correlation. A Jeffress-like neural population model in which the behavior of individual neurons was governed by the cross-correlation model and the neurons were independently distributed along BF and best IPD predicted trends in human psychophysical HP detection but only when the model incorporated physiological BF and best IPD distributions. These results demonstrate the existence of a rate-place code for HP and BEP in the auditory midbrain and provide a firm physiological basis for models of dichotic pitches.NEW & NOTEWORTHY Dichotic pitches are perceptual illusions created centrally through binaural interactions that offer an opportunity to test theories of pitch and binaural hearing. Here we show that binaural neurons in auditory midbrain encode the frequency of two salient types of dichotic pitches via specific features in the pattern of firing rates along the tonotopic axis. This is the first combined single-unit and modeling study of responses of auditory neurons to stimuli evoking a dichotic pitch.

Influence of first-order lateral reflections on the localization of virtual source reproduced by crosstalk cancellation system

This study elucidates the influence of first-order lateral reflections on the localization of virtual sources reproduced by a crosstalk cancellation system. First, an experimental study was conducted to examine how the reflections with a delay within the scope of the summing localization and the precedence effect influence the localization of virtual sources with different frequency bands and target azimuths. The results show that the localization of virtual sources is influenced due to the reflections with a delay below the low limit of the precedence effect (1 ms universally), and the influence degree is related to the frequency band and target azimuth. When the delay is further increased, the effect of reflected sound is almost negligible. Particularly, for a virtual source in the region outside the loudspeaker on the contralateral side of the reflecting surface (at -60° or -90°), the perceived azimuth under reflections with a small delay was more lateral than that without reflections. Then, simulations were conducted under the same conditions as the experiments. Based on the binaural signals modified by the inhibition function of the precedence effect, binaural cues, including interaural phase delay difference (ITDp) and interaural level difference, were calculated to explain the experimental results. Particularly, the average ITDp over frequency can qualitatively predict the localization results below 1.5 kHz from the subjective experiment in most cases.

Frequency dependence of sensitivity to interaural phase differences in pure tones.

It is well established that in normal-hearing humans, the threshold of interaural time differences for pure tones increases dramatically above about 1300 Hz, only to become unmeasurable above 1400 Hz. However, physiological data and auditory models suggest that the actual decline in sensitivity is more gradual and only appears to be abrupt because the maximum of the psychometric function dips below the threshold proportion correct, e.g., 0.794. Published data only report thresholds at certain proportions correct but not the decline of proportions correct or of the sensitivity index d' with increasing frequencies. Here, we present pure-tone behavioral data obtained with a constant stimulus procedure. Seven of nine subjects showed proportions correct above 0.9 at 1300 Hz and virtually no sensitivity at 1500 Hz (proportion correct within 0.07 of chance level). This corresponds to a sensitivity decline of 46-78 dB/oct, much steeper than predicted by existing models or by the decline of phase locking of the auditory nerve fibers in animal data.

Statistics of the instantaneous interaural parameters for dichotic tones in diotic noise (N0S ψ ).

Stimuli consisting of an interaurally phase-shifted tone in diotic noise-often referred to as N 0 S ψ -are commonly used to study binaural hearing. As a consequence of mixing diotic noise with a dichotic tone, this type of stimulus contains random fluctuations in both interaural phase- and level-difference. We report the joint probability density functions of the two interaural differences as a function of amplitude and interaural phase of the tone. Furthermore, a second joint probability density function for interaural phase differences and the instantaneous cross-power is derived. The closed-form expression can be used in future studies of binaural unmasking first to obtain the interaural statistics and then study more directly the relation between those statistics and binaural tone detection.