Binaural Processor Research Articles

Combination of binaural and harmonic masking release effects in the detection of a single component in complex tones

Both harmonic and binaural signal properties are relevant for auditory processing. To investigate how these cues combine in the auditory system, detection thresholds for an 800-Hz tone masked by a diotic (i.e., identical between the ears) harmonic complex tone were measured in six normal-hearing subjects. The target tone was presented either diotically or with an interaural phase difference (IPD) of 180° and in either harmonic or “mistuned” relationship to the diotic masker. Three different maskers were used, a resolved and an unresolved complex tone (fundamental frequency: 160 and 40 Hz) with four components below and above the target frequency and a broadband unresolved complex tone with 12 additional components. The target IPD provided release from masking in most masker conditions, whereas mistuning led to a significant release from masking only in the diotic conditions with the resolved and the narrowband unresolved maskers. A significant effect of mistuning was neither found in the diotic condition with the wideband unresolved masker nor in any of the dichotic conditions. An auditory model with a single analysis frequency band and different binaural processing schemes was employed to predict the data of the unresolved masker conditions. Sensitivity to modulation cues was achieved by including an auditory-motivated modulation filter in the processing pathway. The predictions of the diotic data were in line with the experimental results and literature data in the narrowband condition, but not in the broadband condition, suggesting that across-frequency processing is involved in processing modulation information. The experimental and model results in the dichotic conditions show that the binaural processor cannot exploit modulation information in binaurally unmasked conditions.

Sound Localization in Noise by Gerbils and Humans

Detection and localization of a target sound in the presence of concurrent, spatially distributed masking sounds is one of the most challenging tasks for the mammalian auditory system. Previous studies demonstrated that the ability to localize signals is decreased by interfering noise. In order to directly compare the behavioral performance in a signal processing task in noise between gerbils and humans in the free sound field, we quantified their localization ability for a low-frequency signal in the presence of six masking noise sources surrounding the subject. Thresholds were measured both for masking noises that were correlated or uncorrelated across the masking sources. Overall, the gerbils required a higher signal/noise ratio to detect the low-frequency signal than the humans; that is, the behavioral performance of the gerbils was considerably worse than that of the humans. Moreover, switching from maskers that were uncorrelated across the masking sources to correlated maskers resulted in more masking in gerbils but in a release from masking in humans. These results would suggest that the gerbil may not be a good animal model for binaural processing. However, simulations of the localization thresholds in a numerical model of binaural processing in gerbils and humans reveal that both the inferior overall performance in gerbils and the opposite effect of masker correlation on the detection thresholds can be attributed to the smaller head size and the wider peripheral auditory filters in gerbils. Thus, the current data indicate that the binaural processor itself (i.e., the evaluation of signals coming from the two ears) is equally sensitive in gerbils and humans. However, the physical limitations imposed by the small head prevent the gerbil from performing equally well in the current paradigm.

A computational model of binaural speech intelligibility level difference

This study addresses two questions relating to the binaural intelligibility level difference (BILD). First, we ask whether the BILD is underlain by an equalization‐cancellation (EC) mechanism, in which a disparity between the interaural time difference of the target and masker is exploited within each frequency channel, rather than across channels. Second, we consider the effects of three sources of internal noise on the EC mechanism: jitter in neural delays, noise in the equalization process and nonlinearities in the auditory pathway. These issues are investigated using a computational model consisting of peripheral auditory model, binaural processor, auditory scene processor and automatic speech recognition system. The binaural model is based on EC processing, with performance limited by internal noise. The auditory scene processor groups speech harmonics by common F0 and identifies 'glimpses' in which the signal‐to‐noise ratio is favorable for speech. The performance of human listeners and the computational model are compared on the same speech intelligibility test (Edmonds & Culling, 2005, JASA 117(5), 3069‐3078). The BILD of human listeners can be replicated by adjusting parameters that determine the internal noise in the EC model; however, the speech reception threshold of the model is lower than that of human listeners.

The salience of interaural temporal disparities (ITDs) conveyed by high-frequency signals is determined by the temporal features of their ongoing envelopes

Many studies of binaural hearing reveal that envelopes of complex, high-frequency stimuli convey useful interaural time differences (ITDs). Envelope-based ITDs have, typically, been found to be less "potent" than those conveyed by low-frequency waveforms. When considering such findings, Blauert, in his classic textbook, noted that monaural, peripheral processing would result in 1) low-frequency inputs to the binaural processor that are essentially half-wave rectified versions of the input waveforms and 2) high-frequency inputs that are essentially full-wave rectified versions of only the envelopes of the input waveforms. Taking this into account, Blauert, echoing Colburn and Esquissaud (1976), surmised that low-frequency signals would ultimately convey "more distinct time cues" having greater "transient features" than would high-frequency signals and, therefore, be expected to provide more precise ITD information. By using high-frequency "transposed" stimuli having envelopes constructed to provide "more distinct time cues" we have shown in a variety of experimental contexts that the potency of envelope-based ITDs, can rival that measured with low-frequency stimuli. Our most recent experiments use "raised sine stimuli" in order to vary directly the temporal features of the envelope waveform including its "sharpness," "off-time," and interaural cross-correlation function. The new data will be discussed in terms of Blauert's insight.

Coding of temporally fluctuating interaural timing disparities in a binaural processing model based on phase differences

A model of the effective processing of interaural timing disparities in the human auditory system is presented which provides modifications and extensions to existing models motivated by recent physiological findings. In particular, an established model of excitatory–inhibitory (EI) neuronal connectivity is complemented by a model that is based on a rate code derived from the interaural phase difference (IPD). The IPD model is shown to successfully simulate literature data on fine structure and envelope-based binaural detection and lateralization experiments. In order to investigate the processing of temporal fluctuations of interaural timing disparities, detection thresholds of broadband binaural-beat stimuli were measured in six normal-hearing listeners and were compared with model simulations. In a first experiment, the highest detectable beat frequency was found to be 96 Hz for a noise bandwidth of 550 Hz and 219 Hz for a bandwidth of 1100 Hz. Both models predicted lower thresholds, but performed increasingly better when the integration time constants of the binaural processors were reduced. In a second experiment, the signal-to-noise ratio at the detection threshold of binaural-beat stimuli mixed with interaurally uncorrelated noise was measured as a function of the beat frequency. The threshold increased about 1.7 dB per octave which was simulated similarly by both models. The results indicate that the primary temporal resolution of the binaural system for detecting interaural timing disparities is much higher than the temporal resolution found in higher auditory processes as supposedly involved in, e.g., masking.

Binaural segregation in multisource reverberant environments

In a natural environment, speech signals are degraded by both reverberation and concurrent noise sources. While human listening is robust under these conditions using only two ears, current two-microphone algorithms perform poorly. The psychological process of figure-ground segregation suggests that the target signal is perceived as a foreground while the remaining stimuli are perceived as a background. Accordingly, the goal is to estimate an ideal time-frequency (T-F) binary mask, which selects the target if it is stronger than the interference in a local T-F unit. In this paper, a binaural segregation system that extracts the reverberant target signal from multisource reverberant mixtures by utilizing only the location information of target source is proposed. The proposed system combines target cancellation through adaptive filtering and a binary decision rule to estimate the ideal T-F binary mask. The main observation in this work is that the target attenuation in a T-F unit resulting from adaptive filtering is correlated with the relative strength of target to mixture. A comprehensive evaluation shows that the proposed system results in large SNR gains. In addition, comparisons using SNR as well as automatic speech recognition measures show that this system outperforms standard two-microphone beamforming approaches and a recent binaural processor.

Binary and ratio time-frequency masks for robust speech recognition

A time-varying Wiener filter specifies the ratio of a target signal and a noisy mixture in a local time-frequency unit. We estimate this ratio using a binaural processor and derive a ratio time-frequency mask. This mask is used to extract the speech signal, which is then fed to a conventional speech recognizer operating in the cepstral domain. We compare the performance of this system with a missing-data recognizer that operates in the spectral domain using the time-frequency units that are dominated by speech. To apply the missing-data recognizer, the same binaural processor is used to estimate an ideal binary time-frequency mask, which selects a local time-frequency unit if the speech signal within the unit is stronger than the interference. We find that the performance of the missing data recognizer is better on a small vocabulary recognition task but the performance of the conventional recognizer is substantially better when the vocabulary size is increased.

Self-reported benefits from successive bilateral cochlear implantation in post-lingually deafened adults: randomised controlled trial

Adult users of unilateral Nucleus CI24 cochlear implants with the SPEAK processing strategy were randomised either to receive a second identical implant in the contralateral ear immediately, or to wait 12 months while they acted as controls for late-emerging benefits of the first implant. Twenty four subjects, twelve from each group, completed the study. Receipt of a second implant led to improvements in self-reported abilities in spatial hearing, quality of hearing, and hearing for speech, but to generally non-significant changes in measures of quality of life. Multivariate analyses showed that positive changes in quality of life were associated with improvements in hearing, but were offset by negative changes associated with worsening tinnitus. Even in a best-case scenario, in which no worsening of tinnitus was assumed to occur, the gain in quality of life was too small to achieve an acceptable cost-effectiveness ratio. The most promising strategies for improving the cost-effectiveness of bilateral implantation are to increase effectiveness through enhanced signal processing in binaural processors, and to reduce the cost of implant hardware.

A binaural processor for missing data speech recognition in the presence of noise and small-room reverberation

In this study we describe a binaural auditory model for recognition of speech in the presence of spatially separated noise intrusions, under small-room reverberation conditions. The principle underlying the model is to identify time–frequency regions which constitute reliable evidence of the speech signal. This is achieved both by determining the spatial location of the speech source, and by grouping the reliable regions according to common azimuth. Reliable time–frequency regions are passed to a ‘missing data’ speech recogniser, which performs decoding based on this partial description of the speech signal. In order to obtain robust estimates of spatial location in reverberant conditions, we incorporate some aspects of precedence effect processing into the auditory model. We show that the binaural auditory model improves speech recognition performance in small room reverberation conditions in the presence of spatially separated noise, particularly for conditions in which the spatial separation is 20° or larger. We also demonstrate that the binaural system outperforms a single channel approach, notably in cases where the target speech and noise intrusion have substantial spectral overlap.

Auditory processing of interaural timing information: new insights.

Differences in the time-of-arrival of sounds at the two ears, or interaural temporal disparities (ITDs), constitute one of the major binaural cues that underlie our ability to localize sounds in space. In addition, ITDs contribute to our ability to detect and to discriminate sounds, such as speech, in noisy environments. For low-frequency signals, ITDs are conveyed primarily by "cycle-by-cycle" disparities present in the fine-structure of the waveform. For high-frequency signals, ITDs are conveyed by disparities within the time-varying amplitude, or envelope, of the waveform. The results of laboratory studies conducted over the past few decades indicate that ITDs within the envelopes of high-frequency are less potent than those within the fine-structure of low-frequency stimuli. This is true for both measures of sensitivity to changes in ITD and for measures of the extent of the perceived lateral displacement of sounds containing ITDs. Colburn and Esquissaud (1976) hypothesized that it is differences in the specific aspects of the waveform that are coded neurally within each monaural (single ear) channel that account for the greater potency of ITDs at low frequencies rather than any differences in the more central binaural mechanisms that serve these different frequency regions. In this review, the results of new studies are reported that employed special high-frequency "transposed" stimuli that were designed to provide the high-frequency channels of the binaural processor with envelope-based information that mimics waveform-based information normally available only in low-frequency channels. The results demonstrate that these high-frequency transposed stimuli (1) yield sensitivity to ITDs that approaches, or is equivalent to, that obtained with "conventional" low-frequency stimuli and (2) yield large extents of laterality that are similar to those measured with conventional low-frequency stimuli. These findings suggest that by providing the high-frequency channels of the binaural processor with information that mimics that normally available only at low frequencies, the potency of ITDs in the two frequency regions can be made to be similar, if not identical. These outcomes provide strong support for Colburn and Esquissaud's (1976) hypothesis. The use of high-frequency transposed stimuli, in both behavioral and physiological investigations offers the promise of new and important insights into the nature of binaural processing.

Binaural processing model based on contralateral inhibition. I. Model structure.

This article presents a quantitative binaural signal detection model which extends the monaural model described by Dau et al. [J. Acoust. Soc. Am. 99, 3615-3622 (1996)]. The model is divided into three stages. The first stage comprises peripheral preprocessing in the right and left monaural channels. The second stage is a binaural processor which produces a time-dependent internal representation of the binaurally presented stimuli. This stage is based on the Jeffress delay line extended with tapped attenuator lines. Through this extension, the internal representation codes both interaural time and intensity differences. In contrast to most present-day models, which are based on excitatory-excitatory interaction, the binaural interaction in the present model is based on contralateral inhibition of ipsilateral signals. The last stage, a central processor, extracts a decision variable that can be used to detect the presence of a signal in a detection task, but could also derive information about the position and the compactness of a sound source. In two accompanying articles, the model predictions are compared with data obtained with human observers in a great variety of experimental conditions.

Transposed stimuli reveal similar underlying sensitivity to interaural timing information at high and low frequencies

Threshold interaural temporal disparities (ITDs) obtained at high frequencies are typically much larger than those obtained at low frequencies. Colburn and Esquissaud [J. Acoust. Soc. Am. Suppl. 1 59, S23 (1976)] attributed this to differences in the peripheral, monaural processing of the stimuli. They hypothesized that neural impulses synchronized to the waveform at low frequencies and to only the envelope at high frequencies serve as inputs to a binaural processor that functions similarly at low- and high-signal frequencies. This hypothesis was supported in ITD-discrimination experiments employing novel, high-frequency ‘‘transposed stimuli’’ designed to provide high-frequency channels with envelope-based information that mimics information that is normally available only in low-frequency channels. That is, when the inputs to the high-frequency binaural channels were provided with information like that normally available at low frequencies, sensitivity to ITD in the two spectral regions was found to be similar. The data at both low and high frequencies were well accounted for, quantitatively, in terms of a constant-criterion change in normalized interaural correlation computed subsequent to bandpass filtering. compression, rectification, and low-pass filtering. [Work supported by NIH.]

A model of auditory localization in noise using neural networks

Janko et al. [Binaural and Spatial Hearing in Real and Virtual Environments, edited by Gilkey and Anderson (Erlbaum, Hillsdale, NJ, 1996), pp. 557–570] used neural networks to describe auditory localization and found that data in quiet were insufficient to discriminate among models. To further constrain the models, localization in the presence of a spatially fixed masker is considered here. The wideband targets and maskers were filtered by head-related transfer functions, then by a gammatone filter bank. A model for binaural interaction similar to that of Lindemann [J. Acoust. Soc. Am. 80, 1608–1622 (1986)] was used to process the filter-bank outputs. The inhibited cross-correlation output of the binaural processor was sampled and used as input to a neural network (e.g., 24 correlation lags at each of 13 frequency channels for 312 input nodes). The network was trained using back propagation to estimate the target location in quiet, then tested on targets in noise at several values of signal-to-noise ratio (SNR). The network was also trained across several values of SNR. Results are compared to localization in noise by humans [Good and Gilkey, J. Acoust. Soc. Am. 99, 1108–1117 (1996)]. [Work supported by AFOSR F49620-95-1-0106 and the Ohio Board of Regents.]

Performance of an electrical-analog model of the auditory system in several masking conditions

The present paper includes a description of an electrical-analog model of the monaural and binaural system and an evaluation of its performance in a variety of masking conditions. Each monaural channel of the model is similar to that described by Jeffress [J. Acoust. Soc. Am. 41, 2 (1967); 44, 1 (1968)] and consists of a narrow filter, half-wave rectifier, and leaky-integrator network prior to a decision device. The binaural channel receives inputs from each monaural channel and consists of a subtractor network and full-wave rectifier prior to a decision device. Although the binaural processor is borrowed conceptually from Durlach's equalization-cancellation model [J. Acoust. Soc. Am. 32 (1960); 35 (1963)], the binaural mechanism operates on the outputs of the monaural channels rather than the input waveforms. Data gathered in masking experiments which have manipulated such signal parameters as intensity, duration, interaural phase, and interaural intensity are compared to the model's performance in identical situations.

Reproducing the Cocktail Party Effect

The Cocktail Party Effect allows the binaural listener to concentrate on speech from one talker, suppressing others in his environment. As contrasted to the monaural listener, the binaural auditor gains between 5 and 15 db signal-to-noise ratio necessary for 50% intelligibility. A gain of this magnitude cannot be reconciled with linear array theory which prescribes addition of the two available aural inputs. An alternative accounting can be made by postulating signal processing based upon properties of binaural hearing. In this modal the binaural processor derives a temporary signal which is used to gate the aural input. In effect, the gating signal leaves the major portions of the preferred talker's speech envelope intact while suppressing sound from other talkers or background noise when these do not overlap with the preferred speech. A preliminary circuit version of such a processor which derives the gating signal by cross correlation has been built and tested. Subjective measurements in two- and three-speaker environments yielded increases of 9 and 5 db, respectively, in signal-to-noise ratio.