Head-related Impulse Responses Research Articles

Robust Speech Dereverberation With a Neural Network-Based Post-Filter That Exploits Multi-Conditional Training of Binaural Cues

This study presents an algorithm for binaural speech dereverberation based on the supervised learning of short-term binaural cues. The proposed system combined a delay-and-sum beamformer with a neural network-based post-filter that attenuated reverberant components in individual time-frequency units. A multi-conditional training procedure was used to simulate the uncertainties of short-term binaural cues in response to room reverberation by mixing the direct part of head related impulse responses (HRIRs) with diffuse noise. Despite being trained with only anechoic HRIRs, the proposed dereverberation algorithm was tested in a variety of reverberant environments and achieved considerable improvements relative to a coherence-based approach in terms of three objective metrics reflecting speech quality and speech intelligibility. Moreover, a systematic evaluation showed that the proposed system generalized very well to a wide range of acoustic conditions, including various measured binaural room impulse responses reflecting different reverberation times, azimuth positions spanning the entire frontal hemifield, various source-receiver distances as well as different artificial heads.

Auralization generated by modeling HRIRs with artificial neural networks and its validation using articulation tests

The usual method to generate an auralization is to simulate the acoustic field with a computer code and to convolve each wavefront that arrives to the receiver with the corresponding measured head-related impulse response (HRIR) direction, to obtain the binaural room impulse responses (BRIRs) for selected positions. In this work, this technique is replaced by a procedure where the HRIRs are modeled by a set of artificial neural networks, with significant computational gain. The type of network and its architecture is briefly described and a comparison between measured and modeled responses — both in time and frequency domains — are presented for several directions covering the sphere around the head. Furthermore, since graphs do not tell all when auralizations are the concern, a virtual auralization assessment is performed. In this work, computer modeling auralizations are validated through articulation tests. Three articulation scores are computed. The first one is obtained in the actual room itself. The second one is obtained from measuring the BRIRs and convolving them with anechoic speech signals. The third one result from the room’s numerical simulation, generating as output the virtual BRIRs. These are then convolved with the same anechoic speech signals. The three articulation scores are then compared, and this is done in four different conditions. To investigate the influence of background noise level in the error among the compared articulation scores, pink noise with two distinct levels is introduced in one of the rooms, both in the actual and virtual tests. Finally, one strongly reverberant room is considered in other to evaluate if, in this condition of low intelligibility, the auralization is still trustworthy. In all cases, the results show that differences lower than 3.5% among actual and virtual articulation scores were found. The main conclusions are that the virtual auralizations obtained with the room simulator using artificial neural networks to model the head-related impulse responses are successful and that articulation scores are a reliable metric to validate auralizations.

Identification of perceptually relevant methods of inter-aural time difference estimation.

The inter-aural time difference (ITD) is a fundamental cue for human sound localization. Over the past decades several methods have been proposed for its estimation from measured head-related impulse response (HRIR) data. Nevertheless, inter-method variations in ITD calculation have been found to exceed the known just noticeable differences (JNDs), leading to possible perceptible artifacts in virtual binaural auditory scenes, when personalized HRIRs are being used. In the absence of an objective means for validating ITD estimations, this paper examines which methods lead to the most perceptually relevant results. A subjective lateralization study compared objective ITDs to perceptually evaluated inter-aural pure delay offsets. Results clearly indicate the first-onset threshold detection method, using a low relative threshold of -30 dB, applied on 3 kHz low-pass filtered HRIRs as consistently the most perceptually relevant procedure across various metrics. Several alternative threshold values and methods based on the maximum or centroid of the inter-aural cross correlation of similarly filtered HRIR or HRIR envelopes also provided reasonable results. On the contrary, phase-based methods employing the integrated relative group delay or auditory model were not found to perform as well.

Identifying a perceptually relevant estimation method of the inter-aural time delay

The Inter-aural Time Difference (ITD) is a fundamental cue for human sound localization. Over the past decades, several methods have been proposed for its estimation from measured Head-Related Impulse Response (HRIR) data. Nevertheless, inter-method variations in ITD calculation have been found to exceed the known Just Noticeable Differences (JNDs), hence leading to possible perceptible artifacts in virtual binaural auditory scenes, even for cases when personalized HRIRs are being used. In the absence of an objective means for validating ITD estimations, this paper evaluates which methods lead to the most perceptually relevant results. A subjective lateralization study compared objective ITDs to perceptually driven inter-aural pure delay offsets. Results clearly indicate the first-onset Threshold detection method, using a low relative threshold of -30 dB, applied on 3 kHz low-pass filtered HRIRs as the most perceptually relevant procedure across various metrics. Alternative threshold values and methods based on the maximum or centroid of the Inter-Aural Cross Correlation of similarly filtered HRIRs or HRIR envelopes also provided reasonable results. On the contrary, phase-based methods employing the Integrated Relative Group Delay were not found to perform as well.

Computing head related impulse responses and transfer functions using time domain equivalent sources

In the past, head-related impulse responses (HRIR) and head-related transfer functions (HRTF) have primarily been computed using frequency domain boundary element methods or finite-difference time domain methods. The possibility of computing HRIRs and HRTFs using transient equivalent sources is examined using a lumped parameter technique for enforcing the specified boundary condition. It is demonstrated that performing the computations in the time domain is advantageous because only a few thousand time steps are need to fully define the HRIRs and nonuniform meshes can be used to reduce the number of acoustic variables drastically without significantly degrading the solution accuracy. It is also shown that the computations adapt well to parallel processing environments and the times associated with the equivalent source calculations are proportional to the number of processors.

A comparison of modal and spatially matched-filter beamforming for rigid spherical microphone arrays in the context of data-based binaural synthesis

Several approaches to data-based binaural synthesis have been published that capture a sound field by means of a spherical microphone array. The captured sound field is decomposed into plane waves, which are auralized using (far-field) head-related impulse responses (HRIRs). In practice, the decomposition into plane waves is performed by beamforming. A well-known technique for spherical microphone arrays is modal beamforming where the sound field is represented with respect to surface spherical harmonics. A numerically stable approximation is the spatio-temporal matched-filter technique. Here the acoustic reciprocity theorem is used in conjunction with the matched filter technique for beamforming. This paper reviews the matched filter beamformer and compares its properties to the modal beamformer in a realistic scenario. The performance of the two beamforming techniques is compared in the context of data-based binaural synthesis. Time-domain properties of the resulting ear-signals, as well as suitable binaural measures, are investigated for simulated and captured scenarios.

Analytical Ellipsoidal Model of Interaural Time Differences for the Individualization of Head-Related Impulse Responses

Analytical Ellipsoidal Model of Interaural Time Differences for the Individualization of Head-Related Impulse Responses

Discrimination of virtual sound fields different in spatial aliasing

There are theoretical techniques for sound field reproduction with a loudspeaker array. However, physical precision of synthesized sound field is limited in frequency domain. When the distance among loudspeakers is below a half of wavelength, spatial aliasing will occur. The ideal distance among loudspeakers was less than 8.5 mm@20 kHz, and it is impossible to arrange loudspeakers. Therefore, we carried out experiments to know an upper-limit frequency condition, which can give subjective experience identical to an ideal sound field. We assumed a virtual spherical boundary around a listener. Numerous virtual loudspeakers were set on the boundary to simulate a WFS system. Room impulse responses and head-related impulse responses were calculated by computer simulation. Then, binaural impulse responses were synthesized as combinations of RIRs and HRIRs. We picked up the boundary points so that upper-limit frequency without the spatial aliasing of BRIRs was systematically controlled. Listening test was conduct...

Psychophysical investigation of binaural spatial-audio music perception with varying degrees of realism

A psychoacoustic listening test is conducted to explore the influence of varying degrees of realism on binaural spatial-audio music perception. The specific factors that are varied in combination are: (1) whether or not head-tracking is used; (2) whether or not individualized Head-Related Impulse Response filters are used; (3) the number of sound-track channels: mono, stereo, and 5.2 surround; and (4) whether or not reverberation is used. Experiments are repeated a number of times with listeners requested to make judgement comparisons regarding: externalization, quality of the frontal image, timbre, and preference. Open headphones (AKG 1000) are used so that listeners can compare the various binaural spatial-audio rendering conditions with standard non-binaural headphone presentation and also loudspeaker presentation. The results indicate the impact of varying degreesof realism on various aspects of binaural spatial-audio music perception.

Discrimination and streaming of speech sounds based on differences in lateralization in the horizontal and median planes

Understanding speech in complex backgrounds relies on our ability to perceptually organize competing voices into streams. Differences in fundamental frequency (F0) between voiced sounds are known to enhance stream segregation; less is known about the perceptual organization of unvoiced sounds such as fricative consonants. We showed previously that natural consonant-vowel (CV) pairs can be segregated based on F0 differences, despite lacking F0 cues in the fricative part. This study also used CVs, filtered by head-related impulse responses (HRIR) to simulate different positions in the horizontal and median planes. In the median plane, cues are limited to high frequencies, and so should affect fricatives more than vowels. Both discrimination, using a three-interval forced-choice task, and streaming, using a within- or across-stream repetition-detection task, were tested. The CV pairs were either held constant during a trial, or varied randomly. In the constant condition, any difference in spectrum would indicate a change in location along the median plane; in the variable condition, such differences would require listeners to extract spectral regularities from across widely varying spectra of the speech stimuli. Preliminary results suggest that discrimination and streaming are more challenging in the median than in the horizontal plane. [Work supported by NIH grant R01DC012262.]

3D Sound applied to the design of Assisted Navigation Devices for the Visually Impaired

This work presents an approach to generate 3D sound by using a set of artificial neural networks (ANNs). The proposed method is capable to reconstruct the Head Related Impulse Responses (HRIRs) by means of spatial interpolation. In order to cover the whole reception auditory space, without increasing the network complexity, a structure of multiple networks (set), each one modeling a specific area was adopted. The three main factors that influence the model accuracy --- the network's architecture, the reception area's aperture angles and the HRIR's time shifts --- are investigated and an optimal setup is presented. The computational effort to process the ANN is shown to be slightly smaller than traditional interpolation methods and all error calculation reached very low levels, validating the method to be used in the design of a 3D sound emitter capable of provide navigation aid for the visually impaired. Two approaches are presented in order to detect obstacles, one which makes use of computational vision techniques and other with laser proximity sensors.

Locating virtual sound sources at arbitrary distances in real-time binaural reproduction

A real-time system for sound spatialization via headphones is presented. Conventional headphone spatialization techniques effectively place sources on the surface of a virtual sphere around the listener. In the new system, sources can be spatialized at different distances from a listener by interpolating head-related impulse responses (HRIRs) measured between 20 and 160 cm. These HRIRs are stored in different databases depending on the audio sampling rate. To ease the real-time constraints, users can choose the number of HRIR taps used in the convolution, and an alternative interpolation technique (simplex interpolation) was implemented instead of trilinear interpolation. Subjective tests showed that such simplifications yield satisfactory spatialization for some angles and distances.

A New Selection Method of Anthropometric Parameters in Individualizing HRIR

A trend issue in modeling head-related impulse responses (HRIRs) is how to individualize HRIRs models that are convenient for a particular listener. The objective of this research is to show a robust selection method of eight anthropometric parameters out of all 27 parameters defined in CIPIC HRTF Database. The proposed selection method is systematically and scientifically acceptable, compared to ‘trial and error’ method in selecting the parameters. The selected anthropometric parameters of a given listener were applied in establishing multiple linear regression models in order to individualize his / her HRIRs. We modeled the entire minimum phase HRIRs in horizontal plane of 35 subjects using principal components analysis (PCA). The individual minimum phase HRIRs can be estimated adequately by a linear combination of ten orthonormal basis functions.

Effect of domain selection for compact representation of spatial variation of Head-Related Transfer Function in all directions based on Spatial Principal Components Analysis

In this article, compact representation of spatial variation of Head-Related Transfer Function (HRTF) or its corresponding inverse Fourier transform, namely Head-Related Impulse Response (HRIR) based on Principal Components Analysis (PCA), which is called the Spatial Principal Component Analysis (SPCA), is investigated, focusing on effect of domain selection. The SPCA was carried out for a database of HRTFs in all directions by selecting the domain as one of the HRIRs, the complex HRTFs, the frequency amplitudes of HRTFs, and log-amplitudes of HRTFs. In the latter two cases the minimum phase approximation was incorporated. Comparison of the accuracy in both time and frequency domains showed that the most compact representation is obtained by using the frequency amplitudes of HRTFs when the minimum phase approximation is acceptable, and the complex HRTFs bring about the most compact representation when the minimum phase approximation is not acceptable.

Head-related impulse response cues for spatial auditory brain-computer interface.

This study provides a comprehensive test of a head-related impulse response (HRIR) cues for a spatial auditory brain-computer interface (saBCI) speller paradigm. We present a comparison with the conventional virtual sound headphone-based spatial auditory modality. We propose and optimize the three types of sound spatialization settings using a variable elevation in order to evaluate the HRIR efficacy for the saBCI. Three experienced and seven naive BCI users participated in the three experimental setups based on ten presented Japanese syllables. The obtained EEG auditory evoked potentials (AEP) resulted with encouragingly good and stable P300 responses in online BCI experiments. Our case study indicated that users could perceive elevation in the saBCI experiments generated using the HRIR measured from a general head model. The saBCI accuracy and information transfer rate (ITR) scores have been improved comparing to the classical horizontal plane-based virtual spatial sound reproduction modality, as far as the healthy users in the current pilot study are concerned.

Bone-Conduction Hearing Aids

Bone-Conduction Hearing Aids

Continuous Function Modeling of Head-Related Impulse Response

A continuous function model is developed for efficient head-related impulse response (HRIR) representation. In this model, HRIR is modeled as the response of an IIR filter via common factor decomposition analysis, where HRIRs at the same azimuth but different elevations share the same IIR pole part and HRIRs at the same elevation but different azimuthes share the same IIR zero part. The pole and zero coefficients of the proposed common factor IIR (CF-IIR) filters are further modeled using sinusoidal functions of azimuth and elevation, respectively. A fast algorithm for evaluating sinusoidal function for efficient synthesis of virtual 3D sound is also developed.

Effects of virtual speaker density and room reverberation on spatiotemporal thresholds of audio-visual motion coherence.

The present study examined the effects of spatial sound-source density and reverberation on the spatiotemporal window for audio-visual motion coherence. Three different acoustic stimuli were generated in Virtual Auditory Space: two acoustically “dry” stimuli via the measurement of anechoic head-related impulse responses recorded at either 1° or 5° spatial intervals (Experiment 1), and a reverberant stimulus rendered from binaural room impulse responses recorded at 5° intervals in situ in order to capture reverberant acoustics in addition to head-related cues (Experiment 2). A moving visual stimulus with invariant localization cues was generated by sequentially activating LED's along the same radial path as the virtual auditory motion. Stimuli were presented at 25°/s, 50°/s and 100°/s with a random spatial offset between audition and vision. In a 2AFC task, subjects made a judgment of the leading modality (auditory or visual). No significant differences were observed in the spatial threshold based on the point of subjective equivalence (PSE) or the slope of psychometric functions (β) across all three acoustic conditions. Additionally, both the PSE and β did not significantly differ across velocity, suggesting a fixed spatial window of audio-visual separation. Findings suggest that there was no loss in spatial information accompanying the reduction in spatial cues and reverberation levels tested, and establish a perceptual measure for assessing the veracity of motion generated from discrete locations and in echoic environments.

The moving minimum audible angle is smaller during self motion than during source motion.

We are rarely perfectly still: our heads rotate in three axes and move in three dimensions, constantly varying the spectral and binaural cues at the ear drums. In spite of this motion, static sound sources in the world are typically perceived as stable objects. This argues that the auditory system—in a manner not unlike the vestibulo-ocular reflex—works to compensate for self motion and stabilize our sensory representation of the world. We tested a prediction arising from this postulate: that self motion should be processed more accurately than source motion. We used an infrared motion tracking system to measure head angle, and real-time interpolation of head related impulse responses to create “head-stabilized” signals that appeared to remain fixed in space as the head turned. After being presented with pairs of simultaneous signals consisting of a man and a woman speaking a snippet of speech, normal and hearing impaired listeners were asked to report whether the female voice was to the left or the right of the male voice. In this way we measured the moving minimum audible angle (MMAA). This measurement was made while listeners were asked to turn their heads back and forth between ± 15° and the signals were stabilized in space. After this “self-motion” condition we measured MMAA in a second “source-motion” condition when listeners remained still and the virtual locations of the signals were moved using the trajectories from the first condition. For both normal and hearing impaired listeners, we found that the MMAA for signals moving relative to the head was ~1–2° smaller when the movement was the result of self motion than when it was the result of source motion, even though the motion with respect to the head was identical. These results as well as the results of past experiments suggest that spatial processing involves an ongoing and highly accurate comparison of spatial acoustic cues with self-motion cues.

A comparative study of Interaural Time Delay estimation methods.

The Interaural Time Delay (ITD) is an important binaural cue for sound source localization. Calculations of ITD values are obtained either from measured time domain Head-Related Impulse Responses (HRIRs) or from their frequency transform Head-Related Transfer Functions (HRTFs). Numerous methods exist in current literature, based on a variety of definitions and assumptions of the nature of the ITD as an acoustic cue. This work presents a thorough comparative study of the degree of variability between some of the most common methods for calculating the ITD from measured data. Thirty-two different calculations or variations are compared for positions on the horizontal plane for the HRTF measured on both a KEMAR mannequin and a rigid sphere. Specifically, the spatial variations of the methods are investigated. Included is a discussion of the primary potential causes of these differences, such as the existence of multiple peaks in the HRIR of the contra-lateral ear for azimuths near the inter-aural axis due to multipath propagation and head/pinnae shadowing.