Head-related Impulse Responses Research Articles

DNN-based HRTF individualization for accurate spectral cues using a compact PRTF

Head-Related Transfer Function (HRTF) plays a critical role in how the auditory system perceives spatial information. The spectral cues embedded in HRTF are vital for accurately determining the elevation of sound sources. In existing approaches, deep neural networks (DNNs) have been utilized to predict the magnitude spectra of HRTF from images of the pinna, typically employing the HRTF log-magnitude as the output during training. However, HRTF encompasses the acoustic characteristics of both the head and torso, exhibiting direction-dependent patterns that pose challenges in reconstructing its spectral cues. To address this complexity, we propose an innovative method for HRTF individualization. Our model uses Pinna-Related Transfer Function (PRTF) as the output during training, which helps alleviate the impact of sound reflections from the head and torso in the head-related impulse response (HRIR). Our experimental findings, based on an HRTF dataset, illustrate that our proposed model excels in reconstructing the first and second spectral cues. Furthermore, it outperforms previous deep learning models in terms of log spectral distortion (LSD).

Comparing subjective similarity ratings and quantitative errors for the evaluation of free-field binaural panning techniques

A free-field sound source can be accurately rendered over headphones via direct convolution with head-related impulse responses (HRIRs). Using binaural panning techniques, a free-field source can be approximated, which can be thought of as HRIR reconstruction. However, a loss of fidelity occurs related to the panning algorithm’s spatial resolution. Previously, perceptual comparisons of noise bursts were made between panned sources and direct convolution with measured HRIRs. Reconstructions were generated using five different binaural panning methods, both with and without time-alignment of the HRIRs with separate application of the interaural time delay (ITD). To further explore the listening test results, the quantitative differences between the reconstructed and original HRIRs were investigated alongside the perceptual data. Several perceptually-motivated error metrics were evaluated, including errors in both ITD and interaural level difference. The fidelity of the reconstructed HRTF magnitude response was evaluated using metrics that leveraged auditory modeling steps. Both subjective and objective results support that HRIR time-alignment reduces the number of filters needed for high perceptual accuracy. A principal component-base rendering filter set produced the best subjective accuracy with only a small number of required filters (16-25). The ability to predict perceptually detectable degradation via quantitative errors will be discussed.

Auditory Spatial Bisection of Blind and Normally Sighted Individuals in Free Field and Virtual Acoustics.

Sound localization is an important ability in everyday life. This study investigates the influence of vision and presentation mode on auditory spatial bisection performance. Subjects were asked to identify the smaller perceived distance between three consecutive stimuli that were either presented via loudspeakers (free field) or via headphones after convolution with generic head-related impulse responses (binaural reproduction). Thirteen azimuthal sound incidence angles on a circular arc segment of ±24° at a radius of 3 m were included in three regions of space (front, rear, and laterally left). Twenty normally sighted (measured both sighted and blindfolded) and eight blind persons participated. Results showed no significant differences with respect to visual condition, but strong effects of sound direction and presentation mode. Psychometric functions were steepest in frontal space and indicated median spatial bisection thresholds of 11°-14°. Thresholds increased significantly in rear (11°-17°) and laterally left (20°-28°) space in free field. Individual pinna and torso cues, as available only in free field presentation, improved the performance of all participants compared to binaural reproduction. Especially in rear space, auditory spatial bisection thresholds were three to four times higher (i.e., poorer) using binaural reproduction than in free field. The results underline the importance of individual auditory spatial cues for spatial bisection, irrespective of access to vision, which indicates that vision may not be strictly necessary to calibrate allocentric spatial hearing.

Effects of visual stimuli on auditory separation of sound images spatially split by synthesized binaural signal

In order to clarify the effect of visual stimuli on the spatially split perception of sound images by interaural differences, we conducted spatially split perception experiments under conditions in which visual stimuli were presented on a head mounted display. The auditory stimuli were synthesized binaural signals consisting of two uncorrelated pink noises convolved with the receiver's own head related impulse responses. As manipulated by the listener using the mouse scroll-wheel, the visual and auditory stimuli spread to the left and right in front of the listener, and the listener was made to answer at what point the sound images were spatially split into two. As a result, the detection limit was smaller in the conditions with split visual stimuli than in the condition without visual stimulus. These results suggest that visual stimuli that give the impression of separation reduce the detection limit of the auditorily perceived spatially split sound images.

Free-field perceptual evaluation of virtual acoustic rendering algorithms using two head-related impulse response delay treatment strategies

Sound scenes can be auralized over headphones using binaural rendering techniques in conjunction with a set of head-related impulse responses (HRIRs). If the directions of the sound objects to be rendered are known, either virtual loudspeaker or Ambisonic scene-based techniques may be used, each of which introduce spatial and timbral artifacts at lower spatial resolutions. Neal and Zahorik quantitatively evaluated the effect of separately applying the HRIR delays to time-aligned HRIRs for use with virtual loudspeaker array techniques, referred to as the prior HRIR delays treatment strategy (J. Acoust. Soc. Am. 2022). The present work aims to perceptually validate their quantitative results in a listening study. Free-field point sources were binaurally rendered using five different methods: vector-based amplitude panning, Ambisonics panning, direct spherical harmonic transform of the HRIR set, MagLS, and Principal Component-Base Amplitude Panning (Neal and Zahorik, Audio Eng. Soc. AVAR Conference 2022). Renderings were simulated at various directions and spatial resolutions, both with and without the prior HRIR delays treatment strategy and compared to direct HRIR convolution. Broadband noise served as a critical test signal for revealing changes in timbre and localization and a high-resolution binaural head HRIR dataset was used. The subjective data analysis will be presented.

The influence of helmets on sound localisation in motorcyclists

Motorcyclists are among the most vulnerable road users globally, they enjoy less protection than car drivers and experience higher speed collisions than cyclists, with less mobility and spatial awareness to avoid crashes. Sound localisation (SL) while wearing military and ski helmets has been a significant area of research to improve spatial awareness, but no research has been completed thus far on the impact of motorcycle helmets on SL. This project aims to examine experimentally the impact of a motorcycle helmet on the Head Related Impulse Response (HRIR), and therefore determine the extent of the impact on SL ability of a user. A database of HRIRs as a function of position in the horizontal place was established with and without a motorcycle helmet. The variation in the signals was evaluated, and it was found that the helmet results in significant loss of higher frequencies, important for resolving front-back confusions. It is also demonstrated that the overall sound attenuation would result in reduced SL ability of the rider. A novel system to compensate for the changes in the HRIR was developed and experimentally tested to mitigate the impact on SL ability for the wearer.

A model framework for simulating spatial hearing of bilateral cochlear implant users

Bilateral cochlear implants (CIs) greatly improve spatial hearing acuity for CI users, but substantial gaps still exist compared to normal-hearing listeners. For example, CI users have poorer localization skills, little or no binaural unmasking, and reduced spatial release from masking. Multiple factors have been identified that limit binaural hearing with CIs. These include degradation of cues due to the various sound processing stages, the viability of the electrode-neuron interface, impaired brainstem neurons, and deterioration in connectivity between different cortical layers. To help quantify the relative importance and inter-relationship between these factors, computer models can and arguably should be employed. While models exploring single stages are often in good agreement with selected experimental data, their combination often does not yield a comprehensive and accurate simulation of perception. Here, we combine information from CI sound processing with computational auditory model stages in a modular and open-source framework, resembling an artificial bilateral CI user. The main stages are (a) binaural signal generation with optional head-related impulse response filtering, (b) generic CI sound processing not restricted to a specific manufacturer, (c) electrode-to-neuron transmission, (d) binaural interaction, and (e) a decision model. The function and the outputs of different model stages are demonstrated with examples of localization experiments. However, the model framework is not tailored to a specific dataset. It offers a selection of sound coding strategies and allows for third-party model extensions or substitutions; thus, it is possible to employ the model for a wide range of binaural applications and even for educational purposes.

An Algorithm for Generating Virtual Sources in Dynamic Virtual Auditory Display Based on Tensor Decomposition of Head-Related Impulse Responses

Dynamic virtual auditory displays (VADs) are increasingly used for generating various auditory objects and scenes in virtual and augmented reality. Dynamic VADs are required to generate virtual sources in various directions and distances by using HRTF- or HRIR-based binaural synthesis. In the present work, an algorithm for improving the efficiency and performance of binaural synthesis in dynamic VAD is proposed. Based on tensor decomposition, a full set of near-field HRIRs is decomposed as a combination of distance-, direction-, and time-related modes. Then, binaural synthesis in VAD can be implemented by a common set of time mode-related convolvers or filters associated with direction- and distance-related weights. Dynamic binaural signals are created by updating the weights rather than updating the HRIR-based convolvers, which enables the independent control of virtual source distance and direction and avoids the audible artifact caused by updating the HRIR-based convolvers. An example of implementation indicates that a set of eight common convolvers or filters for each ear is enough to synthesize the binaural signals with sufficient accuracy. The computational efficiency of simultaneously generating multiple virtual sources is improved when the number of virtual sources is larger than eight. A virtual-source localization experiment validates the algorithm.

The impact of head-related impulse response delay treatment strategy on psychoacoustic cue reconstruction errors from virtual loudspeaker arrays.

Known errors exist in loudspeaker array processing techniques, often degrading source localization and timbre. The goal of the present study was to use virtual loudspeaker arrays to investigate how treatment of the interaural time delay (ITD) cue from each loudspeaker impacts these errors. Virtual loudspeaker arrays rendered over headphones using head-related impulse responses (HRIRs) allow flexible control of array size. Here, three HRIR delay treatment strategies were evaluated using minimum-phase loudspeaker HRIRs: reapplying the original HRIR delays, applying the relative ITD to the contralateral ear, or separately applying the HRIR delays prior to virtual array processing. Seven array sizes were simulated, and panning techniques were used to estimate HRIRs from 3000 directions using higher-order Ambisonics, vector-base amplitude panning, and the closest loudspeaker technique. Compared to a traditional, physical array, the prior HRIR delay treatment strategy produced similar errors with a 95% reduction in the required array size. When compared to direct spherical harmonic (SH) fitting of head-related transfer functions (HRTFs), the prior delays strategy reduced errors in reconstruction accuracy of timbral and directional psychoacoustic cues. This result suggests that delay optimization can greatly reduce the number of virtual loudspeakers required for accurate rendering of acoustic scenes without SH-based HRTF representation.

Auditory model-based estimation of the effect of head-worn devices on frontal horizontal localisation

Auditory localisation accuracy may be degraded when a head-worn device (HWD), such as a helmet or hearing protector, is used. A computational method is proposed in this study for estimating how horizontal plane localisation is impaired by a HWD through distortions of interaural cues. Head-related impulse responses (HRIRs) of different HWDs were measured with a KEMAR and a binaural auditory model was used to compute interaural cues from HRIR-convolved noise bursts. A shallow neural network (NN) was trained with data from a subjective listening experiment, where horizontal plane localisation was assessed while wearing different HWDs. Interaural cues were used as features to estimate perceived direction and position uncertainty (standard deviation) of a sound source in the horizontal plane with the NN. The NN predicted the position uncertainty of localisation among subjects for a given HWD with an average estimation error of 1°. The obtained results suggest that it is possible to predict the degradation of localisation ability for specific HWDs in the frontal horizontal plane using the method.

The Presence of a Floor Improves Subjective Elevation Accuracy of Binaural Stimuli Created With Non-Individualized Head-Related Impulse Responses

The Presence of a Floor Improves Subjective Elevation Accuracy of Binaural Stimuli Created With Non-Individualized Head-Related Impulse Responses

Improved accuracy and computational efficiency in virtual acoustic rendering using principal components-based amplitude panning

Virtual loudspeaker arrays can be rendered over headphones using a head-related impulse response (HRIR) dataset, but these techniques often introduce undesirable spatial and timbral artifacts that are associated with distortions in the interaural level difference (ILD) cue. Here, a new and efficient virtual acoustic rendering technique that minimizes ILD distortions is described and evaluated. The technique uses principal components analysis (PCA) performed on the time-domain minimum-phase representation of the directionally dependent HRIRs. The resulting principal components (PCs) act as virtual acoustic filters, and the PCA scores are implemented as panning gains. These panning gains map the PC filters to source locations, similar to vector-base amplitude panning (VBAP). Thus, the technique is termed principal components-based amplitude panning (PCBAP). A spatial and spectral error analysis of ILD shows similar reconstruction accuracy is obtained using PCBAP with 15 filters compared to a 256-loudspeaker VBAP array. PCBAP is also scalable to simulate large loudspeaker arrays with a fixed set of PC filters and only small increases in computational requirements. Results from psychophysical testing suggest that perceptual equivalence between a PCBAP-rendered source and a reference source rendered with the original HRIR dataset is directionally dependent but can be achieved with as few as 10–20 PCs.

Comparing the differences in robustness between interaural time delay calculation methods

The interaural time delay (ITD) is a primary cue to lateral sound source direction and can be calculated from a head-related impulse response (HRIR). Although many ITD calculation techniques utilize cross-correlation, recent work suggests that for horizontal plane HRIRs, ITD calculation based on HRIR onset threshold may better match human perception of source location. The goal of the current study is to more comprehensively compare ITD calculation methods across a large sample of HRIRs. 2448 ITD calculation methods were compared using 42 240 HRIRs from the HUTUBS database (440 directions from 96 individuals). Within each method and individual, the calculated ITD function was compared to a spatially smoothed ITD function using a 10th-order spherical harmonic truncation. When large deviations from the smoothed version occurred (>250 μs), the method was tagged as being non-robust to calculation artifacts. Although the robustness of the onset threshold technique was found to be highly dependent upon threshold level and low-pass filter parameters, proper selection of these parameters resulted in little to no calculation artifacts and superior performance to other calculation methods. Improvements in the robustness of ITD calculation may help to optimize virtual auditory displays or HRIR personalization procedures using the minimum-phase plus ITD model.

Acoustic characteristics of a miniature dynamic speaker driver unit MT006B for measurement of head-related impulse responses by reciprocal method

Acoustic characteristics of a miniature dynamic speaker driver unit MT006B for measurement of head-related impulse responses by reciprocal method

Perceptual implications of different Ambisonics-based methods for binaural reverberation.

Reverberation is essential for the realistic auralisation of enclosed spaces. However, it can be computationally expensive to render with high fidelity and, in practice, simplified models are typically used to lower costs while preserving perceived quality. Ambisonics-based methods may be employed to this purpose as they allow us to render a reverberant sound field more efficiently by limiting its spatial resolution. The present study explores the perceptual impact of two simplifications of Ambisonics-based binaural reverberation that aim to improve efficiency. First, a "hybrid Ambisonics" approach is proposed in which the direct sound path is generated by convolution with a spatially dense head related impulse response set, separately from reverberation. Second, the reverberant virtual loudspeaker method (RVL) is presented as a computationally efficient approach to dynamically render binaural reverberation for multiple sources with the potential limitation of inaccurately simulating listener's head rotations. Numerical and perceptual evaluations suggest that the perceived quality of hybrid Ambisonics auralisations of two measured rooms ceased to improve beyond the third order, which is a lower threshold than what was found by previous studies in which the direct sound path was not processed separately. Additionally, RVL is shown to produce auralisations with comparable perceived quality to Ambisonics renderings.

Toward realistic binaural auralizations – perceptual comparison between measurement and simulation-based auralizations and the real room for a classroom scenario

In order to make full use of their potential to replace experiments in real rooms, auralizations must be as realistic as possible. Recently, it has been shown that for speech, head-tracked binaural auralizations based on measured binaural room impulse responses (BRIRs) can be so realistic, that they become indistinguishable (or nearly so) from the real room [1, 2]. In the present contribution, perceptual comparisons between the auralized and the real room are reported for auralizations based both on measured and simulated BRIRs. In the experiment, subjects sitting in the real room rated the agreement between the real and the auralized room with respect to a number of attributes. The results indicate that for most attributes, the agreement between the auralized and the real room can be very convincing (better than 7.5 on a nine-point scale). This was not only observed for auralizations based on measured BRIRs, but also for those based on simulated BRIRs. In the scenario considered here, the use of individual head-related impulse responses (HRIRs) does not seem to offer any benefit over using HRIRs from a head-and-torso-simulator.

Pre-Trained-Based Individualization Model for Real-Time Spatial Audio Rendering System

Spatial audio has attracted more and more attention in the fields of virtual reality (VR), blind navigation and so on. The individualized head-related transfer functions (HRTFs) play an important role in generating spatial audio with accurate localization perception. Existing methods only focus on one database, and do not fully utilize the information from multiple databases. In light of this, a pre-trained-based individualization model is proposed to predict HRTFs for any target user in this paper, and a real-time spatial audio rendering system built on a wearable device is implemented to produce an immersive virtual auditory display. The proposed method first builds a pre-trained model based on multiple databases using a DNN-based model combined with an autoencoder-based dimensional reduction method. This model can capture the nonlinear relationship between user-independent HRTFs and position-dependent features. Then, fine tuning is done using a transfer learning technique at a limit number of layers based on the pre-trained model. The key idea behind fine tuning is to transfer the pre-trained user-independent model to the user-dependent one based on anthropometric features. Finally, real-time issues are discussed to guarantee a fluent auditory experience during dynamic scene update, including fine-grained head-related impulse response (HRIR) acquisition, efficient spatial audio reproduction, and parallel synthesis and playback. These techniques ensure that the system is implemented with little computational cost, thus minimizing processing delay. The experimental results show that the proposed model outperforms other methods in terms of subjective and objective metrics. Additionally, our rendering system runs on HTC Vive, with almost unnoticeable delay.

Speech Intelligibility and Spatial Release From Masking Improvements Using Spatial Noise Reduction Algorithms in Bimodal Cochlear Implant Users.

This study investigated the speech intelligibility benefit of using two different spatial noise reduction algorithms in cochlear implant (CI) users who use a hearing aid (HA) on the contralateral side (bimodal CI users). The study controlled for head movements by using head-related impulse responses to simulate a realistic cafeteria scenario and controlled for HA and CI manufacturer differences by using the master hearing aid platform (MHA) to apply both hearing loss compensation and the noise reduction algorithms (beamformers). Ten bimodal CI users with moderate to severe hearing loss contralateral to their CI participated in the study, and data from nine listeners were included in the data analysis. The beamformers evaluated were the adaptive differential microphones (ADM) implemented independently on each side of the listener and the (binaurally implemented) minimum variance distortionless response (MVDR). For frontal speech and stationary noise from either left or right, an improvement (reduction) of the speech reception threshold of 5.4 dB and 5.5 dB was observed using the ADM, and 6.4 dB and 7.0 dB using the MVDR, respectively. As expected, no improvement was observed for either algorithm for colocated speech and noise. In a 20-talker babble noise scenario, the benefit observed was 3.5 dB for ADM and 7.5 dB for MVDR. The binaural MVDR algorithm outperformed the bilaterally applied monaural ADM. These results encourage the use of beamformer algorithms such as the ADM and MVDR by bimodal CI users in everyday life scenarios.

Speech Enhancement Based on Modulation-Domain Parametric Multichannel Kalman Filtering

Recently we presented a modulation-domain multichannel Kalman filtering (MKF) algorithm for speech enhancement, which jointly exploits the inter-frame modulation-domain temporal evolution of speech and the inter-channel spatial correlation to estimate the clean speech signal. The goal of speech enhancement is to suppress noise while keeping the speech undistorted, and a key problem is to achieve the best trade-off between speech distortion and noise reduction. In this paper, we extend the MKF by presenting a modulation-domain parametric MKF (PMKF) which includes a parameter that enables flexible control of the speech enhancement behaviour in each time-frequency (TF) bin. Based on the decomposition of the MKF cost function, a new cost function for PMKF is proposed, which uses the controlling parameter to weight the noise reduction and speech distortion terms. An optimal PMKF gain is derived using a minimum mean squared error (MMSE) criterion. We analyse the performance of the proposed MKF, and show its relationship to the speech distortion weighted multichannel Wiener filter (SDW-MWF). To evaluate the impact of the controlling parameter on speech enhancement performance, we further propose PMKF speech enhancement systems in which the controlling parameter is adaptively chosen in each TF bin. Experiments on a publicly available head-related impulse response (HRIR) database in different noisy and reverberant conditions demonstrate the effectiveness of the proposed method.

Investigating the perceptual accuracy of machine-learning generated personalized head-eelated impulse responses

An accurate pair of head-related impulse responses (HRIR) is necessary to render a realistic and immersive virtual auditory space. Since personalized HRIRs are challenging to measure, a method of using two machine-learning models to separately estimate the shape and time delays of the HRIRs based on an individual's anthropometric measurements of the head and pinnae is proposed. The investigation evaluates the perceptual accuracy of the resulting HRIRs with two subjective listening studies. First, an ABX test was conducted to determine if listeners could distinguish between auralizations made with an actual HRIR from the CIPIC database and the personalized HRIR based on the corresponding anthropometric measurements. In the second study, subjects performed localization tests in virtual reality to determine the perceptual accuracy of their personalized HRIR compared to an average HRIR. Each set of tests included one of three auralizations (noise, music, or speech) played in five source locations. Perceptual accuracy was evaluated by comparing the localization errors for the average and estimated HRIRs. The stimuli were repeated four times to investigate if subjects improved over time. The subjects were also asked to rate their listening experience to assess factors such as externalization and difficulty of localization.