Models Of Binaural Processing Research Articles

Application of an extended equalization-cancellation model to speech intelligibility with spatially distributed maskers

An extended version of the equalization-cancellation (EC) model of binaural processing is described and applied to speech intelligibility tasks in the presence of multiple maskers. The model incorporates time-varying jitters, both in time and amplitude, and implements the equalization and cancellation operations in each frequency band independently. The model is consistent with the original EC model in predicting tone-detection performance for a large set of configurations. When the model is applied to speech, the speech intelligibility index is used to predict speech intelligibility performance in a variety of conditions. Specific conditions addressed include different types of maskers, different numbers of maskers, and different spatial locations of maskers. Model predictions are compared with empirical measurements reported by Hawley et al. [J. Acoust. Soc. Am. 115, 833-843 (2004)] and by Marrone et al. [J. Acoust. Soc. Am. 124, 1146-1158 (2008)]. The model succeeds in predicting speech intelligibility performance when maskers are speech-shaped noise or broadband-modulated speech-shaped noise but fails when the maskers are speech or reversed speech.

Oscillatory Dipoles As a Source of Phase Shifts in Field Potentials in the Mammalian Auditory Brainstem

A popular model of binaural processing, proposed by Jeffress (1948), states that external interaural time delays (ITDs) are compensated by internal axonal delays allowing ITD to be spatially represented by a population of coincidence detectors in the medial superior olive (MSO). Isolating single-neuron responses in MSO is difficult because of the presence of a strong extracellular field potential known as the neurophonic, so that few studies have tested Jeffress's key prediction. Phase delays in the nucleus laminaris neurophonic in owls have been observed and are consistent with a Jeffress-like model. Here, we recorded neurophonic responses in cat MSO to monaural tones at locations along its dendritic axis. Fourier analysis of the neurophonic was used to extract amplitude and phase at the stimulus frequency. Amplitude, as a function of depth, showed two peaks separated by a dip. A half-cycle phase shift was observed at depths close to the dip, over a wide frequency range. Current source density analysis for contralateral (ipsilateral) stimulation shows a current source close to the neurophonic amplitude peak and a sink a few hundred micrometers ventromedially (dorsolaterally). These results are consistent with a dipole configuration: contralateral (ipsilateral) excitation causes a current sink at the ventromedial (dorsolateral) dendrites and a source at the soma and dorsolateral (ventromedial) dendrites. Incorporating these results in a dipole model explains the phase and amplitude patterns observed. We conclude that the half-cycle phase shift is consistent with a current dipole, making it difficult to derive measurements of axonal delays from the neurophonic.

An introduction to the physiological, psychological, and modeling aspects of the binaural display.

Binaural hearing is a model system to understand behavioral outcomes through models based on physiological measurements. Internal or compensating delays are one component that is generally present in binaural models. These delays are typically modeled as a continuum of pure time delays. We will review the physiological evidence for the distribution of these delays in different mammals, their (in)dependence on stimulus parameters, and we will briefly address their possible origin. Besides in physiology, the binaural display also plays a central role in describing psychoacoustic data. In a typical model of binaural processing, the binaural display is an intermediate stage in which activity is represented as a function of the internal interaural delay, computed by a cross‐correlation process, and frequency. Such models have been applied successfully to a variety of lateralization experiments and to a great variety of binaural signal detection conditions. Besides introducing these elements of binaural models, we will also shortly address the question of whether there is a strict need for behavioral models to contain a continuous internal delay axis.

Accurate Sound Localization in Reverberant Environments Is Mediated by Robust Encoding of Spatial Cues in the Auditory Midbrain

In reverberant environments, acoustic reflections interfere with the direct sound arriving at a listener's ears, distorting the spatial cues for sound localization. Yet, human listeners have little difficulty localizing sounds in most settings. Because reverberant energy builds up over time, the source location is represented relatively faithfully during the early portion of a sound, but this representation becomes increasingly degraded later in the stimulus. We show that the directional sensitivity of single neurons in the auditory midbrain of anesthetized cats follows a similar time course, although onset dominance in temporal response patterns results in more robust directional sensitivity than expected, suggesting a simple mechanism for improving directional sensitivity in reverberation. In parallel behavioral experiments, we demonstrate that human lateralization judgments are consistent with predictions from a population rate model decoding the observed midbrain responses, suggesting a subcortical origin for robust sound localization in reverberant environments.

Spatial aspects of acoustic processing in complex environments.

From the perspective of acoustical signal processing, it is a challenge to analyze a complex sound environment and segregate the acoustic mixture into individual sources, each with its own spatial characteristics and reverberation pattern. However, auditory systems perform this perceptually critical task with impressive ease. This talk will review how acoustical array processors work to separate sound sources. We then consider what most vertebrates, who have two point receivers separated in space, can accomplish by combining the signals that reach their two ears. The nature of the information available to bird and mammalian auditory systems is briefly reviewed, and the computational approaches these systems take to separating sound sources is assessed, including the logic of processing sound within narrow frequency channels over brief time epochs (rather than, e.g., undertaking broadband analysis). Some observations about the ability of classical binaural processing models to implement earlier stages of processing for these complex environments will be discussed. Finally, performance of human and zebra‐finch subjects in tasks requiring judgments about individual sources embedded in complex sound mixtures will be considered in the context of physiologically motivated processing schemes. [Work supported by NIDCD: DC00100 (Colburn), DC05778 (Shinn‐Cunningham), and DC07610 (Sen).]

Coding of temporally fluctuating interaural timing disparities in a binaural processing model based on phase differences

A model of the effective processing of interaural timing disparities in the human auditory system is presented which provides modifications and extensions to existing models motivated by recent physiological findings. In particular, an established model of excitatory–inhibitory (EI) neuronal connectivity is complemented by a model that is based on a rate code derived from the interaural phase difference (IPD). The IPD model is shown to successfully simulate literature data on fine structure and envelope-based binaural detection and lateralization experiments. In order to investigate the processing of temporal fluctuations of interaural timing disparities, detection thresholds of broadband binaural-beat stimuli were measured in six normal-hearing listeners and were compared with model simulations. In a first experiment, the highest detectable beat frequency was found to be 96 Hz for a noise bandwidth of 550 Hz and 219 Hz for a bandwidth of 1100 Hz. Both models predicted lower thresholds, but performed increasingly better when the integration time constants of the binaural processors were reduced. In a second experiment, the signal-to-noise ratio at the detection threshold of binaural-beat stimuli mixed with interaurally uncorrelated noise was measured as a function of the beat frequency. The threshold increased about 1.7 dB per octave which was simulated similarly by both models. The results indicate that the primary temporal resolution of the binaural system for detecting interaural timing disparities is much higher than the temporal resolution found in higher auditory processes as supposedly involved in, e.g., masking.

Binaural and spatial factors affecting speech intelligibility in rooms

Prominent complaints of hearing-impaired listeners relate to the difficulties they experience in complex acoustic environments, particularly environments with multiple sources and reverberation. Available data and some theoretical ideas about the nature of the difficulties with speech intelligibility in these environments will be reviewed with impaired listeners in mind. The effects of the spatial characteristics of the sound sources and the room on the binaural signals and their processing is the primary focus of this presentation. Listeners with normal hearing as well as listeners with hearing impairments, with and without hearing aids, will be considered. Theoretical topics to be discussed include the effects of reverberation on interaural differences and on temporal modulation patterns, including the consequences for performance as predicted by standard models of binaural processing. [Work supported by NIH R01 DC00100.]

The case of the missing delay lines: Synthetic delays obtained by cross-channel phase interaction

Temporal models of pitch and harmonic segregation call for delays of up to 30 ms to cover the full range of existence of musical pitch. To date there is little anatomical or physiological evidence for delays that long. We propose a mechanism by which delays may be synthesized from cross-channel phase interaction. Phases of adjacent cochlear filter channels are shifted by an amount proportional to frequency and then combined as a weighted sum to approximate a delay. Synthetic delays may be used by pitch perception models such as autocorrelation, segregation models such as harmonic cancellation, and binaural processing models to explain sensitivity to large interaural delays. The maximum duration of synthetic delays is limited by the duration of the impulse responses of cochlear filters, itself inversely proportional to cochlear filter bandwidth. Maximum delay is thus frequency dependent. This may explain the fact, puzzling for temporal pitch models such as autocorrelation, that pitch is more salient and easy to discriminate for complex tones that contain resolved partials.

The inclusion of time-varyng jitter in the equalization-cancellation model of binaural processing

A more realistic version of the analytically defined equalization-cancellation(EC) model of binaural processing can be simulated in the time domain by introducing time-varying jitters, both in time and amplitude. Although the classic EC model can predict a wide range of binaural-masking-level-difference (BMLD) data in binaural detection, it is more reasonable physiologically to assume that internal noise is time varying, as opposed to the original assumption that the internal time and amplitude errors are different from waveform to waveform but stay constant within a waveform. In the current study, the BMLD predictions for both the analytical model and the simulation are compared for the representative conditions(NoSπ, NπSo, NoSm). It is also proved theoretically that the simulation results converge to the analytical prediction in the NτnSτs case. Differences between the EC model and the simulation are discussed. Applying this simulation to speech perception based on the speech intelligibility index(SII) in anechoic environments predicts binaural advantages consistent with results described in the literature (Zurek,1993). The simulation is also being applied to the speech-transmission-index (STI) prediction in complex environments. [Work supported by NIDCD R01 DC00100.]

Neural Correlates and Mechanisms of Spatial Release From Masking: Single-Unit and Population Responses in the Inferior Colliculus

Spatial release from masking (SRM), a factor in listening in noisy environments, is the improvement in auditory signal detection obtained when a signal is separated in space from a masker. To study the neural mechanisms of SRM, we recorded from single units in the inferior colliculus (IC) of barbiturate-anesthetized cats, focusing on low-frequency neurons sensitive to interaural time differences. The stimulus was a broadband chirp train with a 40-Hz repetition rate in continuous broadband noise, and the unit responses were measured for several signal and masker (virtual) locations. Masked thresholds (the lowest signal-to-noise ratio, SNR, for which the signal could be detected for 75% of the stimulus presentations) changed systematically with signal and masker location. Single-unit thresholds did not necessarily improve with signal and masker separation; instead, they tended to reflect the units' azimuth preference. Both how the signal was detected (through a rate increase or decrease) and how the noise masked the signal response (suppressive or excitatory masking) changed with signal and masker azimuth, consistent with a cross-correlator model of binaural processing. However, additional processing, perhaps related to the signal's amplitude modulation rate, appeared to influence the units' responses. The population masked thresholds (the most sensitive unit's threshold at each signal and masker location) did improve with signal and masker separation as a result of the variety of azimuth preferences in our unit sample. The population thresholds were similar to human behavioral thresholds in both SNR value and shape, indicating that these units may provide a neural substrate for low-frequency SRM.

Microsecond temporal resolution in monaural hearing without spectral cues?

The auditory system encodes the timing of peaks in basilar-membrane motion with exquisite precision, and perceptual models of binaural processing indicate that the limit of temporal resolution in humans is as little as 10-20 microseconds. In these binaural studies, pairs of continuous sounds with microsecond differences are presented simultaneously, one sound to each ear. In this paper, a monaural masking experiment is described in which pairs of continuous sounds with microsecond time differences were combined and presented to both ears. The stimuli were matched in terms of the excitation patterns they produced, and a perceptual model of monaural processing indicates that the limit of temporal resolution in this case is similar to that in the binaural system.

Binaural processing model based on contralateral inhibition. III. Dependence on temporal parameters.

This paper and two accompanying papers [Breebaart et al., J. Acoust. Soc. Am. 110, 1074-1088 (2001); 110, 1089-1104 (2001)] describe a computational model for the signal processing of the binaural auditory system. The model consists of several stages of monaural and binaural preprocessing combined with an optimal detector. Simulations of binaural masking experiments were performed as a function of temporal stimulus parameters and compared to psychophysical data adapted from literature. For this purpose, the model was used as an artificial observer in a three-interval, forced-choice procedure. All model parameters were kept constant for all simulations. Model predictions were obtained as a function of the interaural correlation of a masking noise and as a function of both masker and signal duration. Furthermore, maskers with a time-varying interaural correlation were used. Predictions were also obtained for stimuli with time-varying interaural time or intensity differences. Finally, binaural forward-masking conditions were simulated. The results show that the combination of a temporal integrator followed by an optimal detector in the time domain can account for all conditions that were tested, except for those using periodically varying interaural time differences (ITDs) and those measuring interaural correlation just-noticeable differences (jnd's) as a function of bandwidth.

Binaural processing model based on contralateral inhibition. I. Model structure.

This article presents a quantitative binaural signal detection model which extends the monaural model described by Dau et al. [J. Acoust. Soc. Am. 99, 3615-3622 (1996)]. The model is divided into three stages. The first stage comprises peripheral preprocessing in the right and left monaural channels. The second stage is a binaural processor which produces a time-dependent internal representation of the binaurally presented stimuli. This stage is based on the Jeffress delay line extended with tapped attenuator lines. Through this extension, the internal representation codes both interaural time and intensity differences. In contrast to most present-day models, which are based on excitatory-excitatory interaction, the binaural interaction in the present model is based on contralateral inhibition of ipsilateral signals. The last stage, a central processor, extracts a decision variable that can be used to detect the presence of a signal in a detection task, but could also derive information about the position and the compactness of a sound source. In two accompanying articles, the model predictions are compared with data obtained with human observers in a great variety of experimental conditions.

Spatial unmasking of nearby pure-tone sources in a simulated anechoic environment

Spatial unmasking of nearby pure-tone sources masked by broadband noise was studied in a simulated anechoic environment. The predicted amount of binaural unmasking was calculated using interaural differences from a spherical-head model [B. G. Shinn-Cunningham, S. G. Santarelli, and N. Kopčo, J. Acoust. Soc. Am. (in press)] and the Colburn model of binaural processing [H. S. Colburn, J. Acoust. Soc. Am. 61, 525–533 (1977)]. Predictions were generated for several spatial configurations of signal and masker in the horizontal plane (varying target and masker azimuthal positions and distances) and for multiple tone frequencies. Predictions were compared with behaviorally measured detection thresholds obtained with headphone simulations using individually measured HRTFs. Results show that changes in detection threshold for free-field signals are dominated by monaural effects due to large interaural level differences. Overall, spatial unmasking effects are very large for nearby sources due to simple energy effects at the better ear. Binaural unmasking is observable only for a subset of spatial configurations where the energy of signal and/or noise at both ears is sufficient to allow binaural processing, and even for these configurations the effect is generally weaker than that observed with distant sources. [Work supported in part by AFOSR Grant F49620-98-1-0108.]

A binaural analog of gap detection.

The temporal resolution of the binaural auditory system was measured using a binaural analog of gap detection. A binaural "gap" was defined as a burst of interaurally uncorrelated noise (Nu) placed between two bursts of interaurally correlated noise (N0). The Nu burst creates a dip in the output of a binaural temporal window integrating interaural correlation, analogous to the dip created by a silent gap in the output of a monaural temporal window integrating intensity. The equivalent rectangular duration (ERD) of the binaural window was used as an index of binaural temporal resolution. In order to derive the ERD, both the shortest-detectable binaural gap and the jnd for a reduction in interaural correlation from unity were measured. In experiment 1, binaural-gap thresholds were measured using narrow-band noise carriers as a function of center frequency from 250 to 2000 Hz (fixed 100-Hz bandwidth) and a function of lower-cutoff frequency from 100 to 400 Hz (fixed 500-Hz upper-cutoff frequency). Binaural-gap thresholds (1) increased significantly with increasing frequency in both tasks, and (2) at frequencies below 500 Hz, were shorter than corresponding silent-gap thresholds measured with the same N0 noises. In experiment 2, interaural-correlation jnd's were measured for the same conditions. The jnd's also increased significantly with increasing frequency. The results were analyzed using a temporal window integrating the output of a computational model of binaural processing. The ERD of the window varied widely across listeners, with a mean value of 140 ms, and did not significantly depend on frequency. This duration is about an order of magnitude longer than the ERD of the monaural temporal window and is, therefore, consistent with "binaural sluggishness."

Should normalization be included in cross-correlation-based models of binaural processing?

It is generally accepted that cross-correlation-based models of binaural information processing provide excellent qualitative and quantitative accounts of data obtained in a wide variety of experimental contexts. These contexts include binaural detection, lateralization, localization, and the perception of pitch mediated by strictly binaural cues. The purpose of this research was to investigate the application of ‘‘normalization’’ as part of the computation of indices of interaural correlation. Such indices have often been utilized to account for binaural detection data but the normalization, to our knowledge, has neither been explicitly studied nor evaluated on its own merits. Normalization ensures that the value of correlation obtained with identical inputs will be +1 (or for polarity-opposite inputs, −1). Nevertheless, normalization, as a stage of processing, is not overtly included in modern cross-correlation-function-based models. In this presentation logical arguments and new data regarding the need to include normalization within cross-correlation-based models of binaural detection/discrimination are provided. The new data were obtained while roving the overall levels of the stimuli in a binaural detection task. The data indicate that, to account for binaural detection, an extremely rapid and accurate stage of normalization must be included in cross-correlation-based models. [Work supported by NIH-DC-02103 and NIH-DC-00234.]

Binaural detection with spectrally nonoverlapping signals and maskers: evidence for masking by aural distortion products.

Thresholds were measured for diotic tonal signals in the presence of interaurally delayed bands of Gaussian noise. When the signal frequency was 525 Hz., the spectrum of the noise was either below (highest frequency, 450 Hz) or above (lowest frequency, 600 Hz) the frequency of the signal. When the signal frequency was 450 Hz, the spectrum of the noise was always above the signal frequency (lowest frequency, 600 Hz). Signals had a 250-ms duration and were temporally centered within the 300-ms long bursts of noise. The spectrum level of the noise was 60 dB. Thresholds obtained in all three conditions varied essentially sinusoidally with the interaural delay of the noise. For signals below the spectrum of the noise, the periodicities within the data were close to, but not identical with, the periodicities of the signals. This outcome is discussed in terms of masking produced by aural distortion products stemming from interactions within the bands of noise [cf. van der Heijden and Kohlrausch, J. Acoust. Soc. Am. 98, 3125-3134 (1995)]. For signals above the spectrum of the noise, the periodicities in the data suggested that masking was produced by components within the band of noise. Patterns within the data are also discussed in terms of limitations concerning the magnitude of external delays that can be matched by internal delays that are incorporated in modern models of binaural processing.

The effects of randomizing values of interaural disparities on binaural detection and on discrimination of interaural correlation.

The purpose of this study was to assess whether randomizing (roving) baseline interaural parameters of binaural stimuli would adversely affect performance in masking-level differences and in interaural correlation-discrimination paradigms. Listeners' detection thresholds were measured in NoSo and NoS pi configurations for both broadband (100-3000 Hz) and narrow-band (450-550 Hz) maskers centered at 500 Hz. In addition, thresholds of interaural decorrelation (from a reference correlation of 1.0) were measured for 100-Hz-wide bands of noise centered at 500 Hz. Data were obtained under conditions in which either values of ITD or values of IID were roved both within and across trials. Data were also collected in the absence of a rove. The range of the rove was +/- 500 microseconds for ITDs and was +/- 6 dB for IIDs. The duration of the stimuli was varied between 17 and 310 ms. Overall, the results indicate that, independent of duration, roving the interaural cues produced small degradations of performance as compared to data obtained in the absence of a rove. The data are inconsistent with the notion that binaural detection depends upon reliable changes in laterality produced by adding S pi signals to diotic maskers. On the other hand, the data are consistent with modern models of binaural processing.

Dichotic pitch discrimination

Dichotic Huggins pitch was produced by generating broadband noise stimuli with narrow sections of the noise (bandwidths of 2, 4, 8, 16, 32, 64, and 128 Hz) interaurally phase shifted (by 90°, 135°, or 180°). The center frequencies of the narrow dichotic bands to which the interaural phase shifts were added were varied to change the value of the Huggins pitches. The just discriminable differences (obtained in a two-alternative, forced-choice adaptive procedure) in center frequency were determined for six listeners and at four (250, 400, 500, and 750 Hz) base center frequencies in an attempt to describe the pitch discrimination of dichotic Huggins pitches. For some listeners, at base center frequencies of 250 and 400 Hz and when the interaural phase shift used to produce the pitch was 180°, discrimination was almost as acute for these dichotic pitches (Δf/f=0.05%) as it was for diotically produced pitches. Pitch discrimination became more difficult as the center frequency increased to 500 and 750 Hz, the bandwidth of the interaurally altered band decreased from 128 to 2 Hz, or the interaural phase shift was reduced from 180° to 135° and 90°. The results will be discussed in terms of models of binaural processing and the use of interaural variables to segregate one sound image from other sound images. [Work supported by the NIDCD and the AFOSR.]

Effects of temporal position on the detectability of interaurally uncorrelated noise

Listeners' ability to detect an interaurally uncorrelated segment of noise within binaural broadband noise bursts was measured as a function of the temporal position of the uncorrelated noise. Psychometric functions were obtained by varying the duration of uncorrelated noise. The results indicate that detectability improves as the duration of uncorrelated noise is increased. Temporal position affected detectability only at the shortest durations of uncorrelated noise tested; a segment positioned either at the end or at the beginning of the burst is more detectable than one in the middle. This result is not easily accounted for by models of binaural processing. For example, the EC model [N. I. Durlach, J. Acoust. Soc. Am. 35, 1206–1218 (1963)] assumes that performance is determined by the power of the difference in the waveforms in the two auditory channels. Since power is computed over the total stimulus duration, it is independent of the temporal position of the uncorrelated noise. The relationship of the current findings to those for the discriminability of diotic noise stimuli will be discussed. [Work supported by AFOSR.]