Head-related Transfer Functions Research Articles

Effect of Interpolation Artifacts on Perceived Stability of Nearby Sources in a Navigable Reverberant Virtual Environment

Convolution with spatial Room Impulse Responses can achieve realistic auralizations. When combined with interpolation between spatially distributed RIRs, this technique can be used to create navigable virtual environments. This study explores the impact of various interpolation parameters on the perceived auditory stability of a nearby static sound source with listener movements in a reverberant environment. The auditory scene was rendered via third-order Ambisonic RIR convolution combined with magnitude-least-squares binaural decoding using nonindividualized head-related transfer functions. First, the estimated direction of arrival as a function of the listener’s position within a 2D grid of RIRs under various configurations is examined as an objective metric. The perceived stability of the auditory source is then assessed through a perceptual experiment. Participants freely explored a virtual scene reproduced over headphones and a tracked head-mounted display. They were asked to rate the stability of a nonvisual source under various conditions of RIR grid density, interpolation panning method, and reverberation time. Results indicate no need to use an RIR grid size finer than 1m to optimize source stability when using a three-nearest-neighbor interpolation scheme.

Ensemble width estimation in HRTF-convolved binaural music recordings using an auditory model and a gradient-boosted decision trees regressor

Binaural audio recordings become increasingly popular in multimedia repositories, posing new challenges in indexing, searching, and retrieval of such excerpts in terms of their spatial audio scene characteristics. This paper presents a new method for the automatic estimation of one of the most important spatial attributes of binaural recordings of music, namely “ensemble width.” The method has been developed using a repository of 23,040 binaural excerpts synthesized by convolving 192 multi-track music recordings with 30 sets of head-related transfer functions (HRTF). The synthesized excerpts represented various spatial distributions of music sound sources along a frontal semicircle in the horizontal plane. A binaural auditory model was exploited to derive the standard binaural cues from the synthesized excerpts, yielding a dataset representing interaural level and time differences, complemented by interaural cross-correlation coefficients. Subsequently, a regression method, based on gradient-boosted decision trees, was applied to the formerly calculated dataset to estimate ensemble width values. According to the obtained results, the mean absolute error of the ensemble width estimation averaged across experimental conditions amounts to 6.63° (SD 0.12°). The accuracy of the method is the highest for the recordings with ensembles narrower than 30°, yielding the mean absolute error ranging between 0.8° and 10.2°. The performance of the proposed algorithm is relatively uniform regardless of the horizontal position of an ensemble. However, its accuracy deteriorates for wider ensembles, with the error reaching 25.2° for the music ensembles spanning 90°. The developed method exhibits satisfactory generalization properties when evaluated both under music-independent and HRTF-independent conditions. The proposed method outperforms the technique based on “spatiograms” recently introduced in the literature.

DNN-based HRTF individualization for accurate spectral cues using a compact PRTF

Head-Related Transfer Function (HRTF) plays a critical role in how the auditory system perceives spatial information. The spectral cues embedded in HRTF are vital for accurately determining the elevation of sound sources. In existing approaches, deep neural networks (DNNs) have been utilized to predict the magnitude spectra of HRTF from images of the pinna, typically employing the HRTF log-magnitude as the output during training. However, HRTF encompasses the acoustic characteristics of both the head and torso, exhibiting direction-dependent patterns that pose challenges in reconstructing its spectral cues. To address this complexity, we propose an innovative method for HRTF individualization. Our model uses Pinna-Related Transfer Function (PRTF) as the output during training, which helps alleviate the impact of sound reflections from the head and torso in the head-related impulse response (HRIR). Our experimental findings, based on an HRTF dataset, illustrate that our proposed model excels in reconstructing the first and second spectral cues. Furthermore, it outperforms previous deep learning models in terms of log spectral distortion (LSD).

HRTF individualization using ear images from 3D meshes

The head-related transfer function (HRTF) describes how a human receives sound from different directions in space. It is unique to each listener. The best way to obtain an individualized HRTF is through direct measurement, but it requires sophisticated laboratory equipment and long measurement times. This work proposes an approach for HRTF individualization employing ear images from 3D meshes using a deep neural network and spherical harmonics transform (SHT). The method relies on the HUTUBS dataset, including 3D meshes and HRTFs. The model uses ear images to predict a low-dimensional representation of the HRTF. Initially, ten images of the right ear of each 3D mesh are taken in different positions. The model consists of two main parts. The first is a convolutional neural network (CNN), which is used to extract features from the ear images. The second part learns from the feature map obtained and predict the spherical harmonic coefficients. Finally, the individualized HRTF is obtained through inverse SHT. The performance of the method is evaluated by computing the log-spectral distortion (LSD) between the measured HRTF and the predicted one. The results show favorable LSD values compared to other models addressing the same problem.

Acoustic measurement around the pinna in the VHF region using a dummy head

We investigated the azimuth and vertical angle characteristics of head related transfer function (HRTF) at very high frequency region using head and torso simulator (HATS). The VHF sounds contain potential hazards for young people who can hear such sounds. Therefore, the relationship between hearing thresholds in very-high frequency (VHF) region and intensity of VHF sounds are very important for considering human impact. The former means sound pressure level (SPL) at eardrum, and the latter is SPL at center of two ears without a head. The difference between two SPLs means head-related transform function (HRTF). In this paper, we reported the HRTFs measured in VHF region for the azimuth and vertical angles by using a HATS. The complicated contour maps of HRTFs for angle and frequency show that the SPLs at eardrum in VHF region depend on among material, size and figure of a pinna.

Mesh registration-based boundary element method of head-related transfer function: unification of hair impedance boundary conditions

In the calculation of Head-Related Transfer Functions (HRTFs) using the Boundary Element Method (BEM), the head surface of a subject needs to be discretized into boundary elements with specific impedance. To simplify HRTF calculation, a method for coupling Mesh Registration and BEM (MR-BEM) is proposed in this work, which means that a template mesh for calculation is established to register and generate the calculation mesh of a new head model, to replace the tedious operations of boundary conditions configuration. To validate the proposed method, the widely-used KEMAR artificial head model was adopted to compare the HRTFs of two groups of calculation meshes manually assigned and automatically registered impedance boundary conditions, respectively. Based on the inherent similarities in head shape and hair regions across individuals, the non-rigid impedance was set only for the hair region. Although the reflections from the non-rigid impedance of hair and clothes were shielded by pinnae in practice, this work just aims to validate the feasibility of the MR-BEM, which is significant for the simplification of BEM-based HRTF calculation.

Assessment of transparent alternative listening devices' effects on localization cues using head-related transfer function measurements

The past few years have seen a meteoric rise in technological advancements in the hearing health industry. New alternative listening devices, often referred to as "hearables", aim to become real "bionic ears" offering hearing protection, amplification, monitoring and even biosensing functionalities. With the recent approval of the "Over-the-Counter Hearing Aid Act" in the United States and the introduction of so-called "OTC" hearing aids, these alternative listening devices now have the potential to revolutionize the world of traditional auditory amplification. Since these devices may be used by individuals with hearing loss, it is important to consider how they might influence spatial auditory awareness. Therefore, this paper studies the effect of alternative listening devices on localization cues using the Head Related Transfer Function (HRTF). In this purpose, we measured the HRTFs on a Head and Torso Simulator (HATS) with and without digital listening devices. To evaluate the acoustic performance of these devices, the devices were set into their "acoustically transparent" mode. These measures enable a preliminary characterization of the effect of hearables and OTC devices on localization cues.

Evaluating on the spatial localization cues of binaural sound pressure recorded via behind-the-ear microphones in a hearing aid

Spatial hearing ability enables people with normal hearing to localize a sound source and utilizes the cocktail party effect to detect the target speech information in a noise environment. Hearing-impairment causes not only hearing loss but also the degradation of spatial auditory ability. Hearing aid is desired to improve the spatial auditory ability of some hearing-impaired people, in addition to compensating hearing loss. However, the microphones for conventional behind-the-ear (BTE) hearing aid are located behind the pinnae rather than in the entrances of ear canal, which may cause distortion in spatial cues in the recorded binaural signals. The present work comprehensively evaluates the spatial hearing cues recorded by BTE-microphones. The head-related transfer functions (HRTF) for a KEMAR artificial head and eight human subjects, at three BTE positions and the entrances of blocked ear-canal, are measured respectively. Analysis from the HRTF data indicates that BTE microphone recording basically reserves the interaural time difference and interaural level difference as lateral-direction information. It also reserves the dynamic cues, but obvious distortions appear in the spectral cues at high frequencies. However, due to the redundancy among different spatial information, it can be inferred that dynamic cues alone may provide appropriate front-back and vertical-direction information.

Mimicking Human Hearing with Topologically Optimized Structure

The remarkable auditory capabilities of human ears have inspired the development of bio-inspired acoustic sensing technologies. In this study, a topology optimization problem is formulated to replicate the hearing profile of the human ear and is solved using a gradient-based optimization method. To ensure manufacturability and mesh independence, a density filter is employed to eliminate gray scale transitions between solid and void regions. The objective function is defined to minimize the error between the human's Head-Related Transfer Function (HRTF) of a reference structure and the designed domain in specific frequencies and directions. Numerical simulations demonstrate that the optimized structure exhibits similar acoustic characteristics and spatial directivity to that of the ideal human ear at the designated frequencies. The proposed structure and optimization method show great potential for applications such as 3D audio rendering, hearing- aid design, and sound source localization.

Validation of sound source localization quality in binaural audio synthesis with head-tracking

Binaural synthesis with head tracking has become a highly important tool in creating an auditory experience for users that is as similar as possible to their perception of sound in the real world. This is crucial in many different applications that rely on accurate representation of real or virtual sound environments, ranging from scientific research to gaming applications. The quality of the binaural synthesis process and the accuracy and naturalness of the produced result depend on numerous factors, ranging from the selection of suitable IMU sensor technology used for head tracking to the choice of methods for implementing binaural synthesis using generic or individual sets of Head-Related Transfer Functions (HRTF). This is especially significant in Virtual Reality (VR) systems with three or six degrees of freedom that fully immerse the users in the virtual world through visual and auditory stimuli. This paper presents the authors' experiences in addressing a variety of challenges that arose during the design of such spatial audio systems and the testing of sound source localization quality. Strong emphasis is put on the testing of sound source localization quality, where the position of the source dynamically changes in relation to the listener.

Effect of discretization on off-axis listeners in binaural synthesis based on the optimal source distribution

3D sound reproduction based on binaural synthesis over loudspeakers is one of the most important techniques in immersive spatial audio. The system inversion involved in the binaural synthesis gives rise to several problems such as dynamic range loss, distortion of the reproduction transfer function, and a lack of robustness to room reflections. The principle of Optimal Source Distribution was proposed to overcome these problems. This utilizes a pair of conceptual monopole transducers whose azimuthal location varies continuously as a function of frequency. Another potential of the principle is that it can provide listening positions for multiple off-axis listeners, in addition to the on-axis listener whose inverse matrix is given, since the phase interference state is aligned at all frequencies. However, these continuously varying transducers are too conceptual to realize in practice. It is necessary to provide discretized sound sources with appropriate frequency ranges according to the principle. In this paper, the effect of discretization of the principle on off-axis listeners' performance is numerically investigated to reveal how those off-axis sweet spots are formed as the discretization becomes finer. An experiment is also carried out using an actual discretized system and the effect of the head-related transfer functions is considered.

The contribution of dynamic cues to externalization of virtual source in virtual auditory display

It is well known that static virtual auditory display with headphone is inclined to create the auditory perception of in-head-localization, especially for target virtual sources in the median plane. By introducing the dynamic variation of binaural pressures caused by head turning, dynamic virtual auditory display eliminates the in-head-localization effectively. The present work investigates the contributions of dynamic interaural time difference (ITD) and dynamic interaural level difference (ILD) in binaural pressures to the externalization of auditory perception. By modifying the head-related transfer functions used in a dynamic virtual auditory display, binaural signals with dynamic ITD or dynamic ILD alone are created and presented by headphone. A psychoacoustic experiment is conducted to evaluate the extent of perceived externalization. The experiment results prove that, for the target virtual source in the median plane, dynamic ITD and dynamic ILD contribute to the externalization of low and high frequency components of the stimuli respectively. For wideband stimulus with low and high frequency components, both dynamic ITD and dynamic ILD contribute externalization but dynamic ITD contributes more than that of dynamic ILD.

Visar: Projecting Virtual Sound Spots for Acoustic Augmented Reality Using Air Nonlinearity

Augmented reality that integrates virtual content in real-world surroundings has attracted lots of concentration as the growing trend of the metaverse. Acoustic augmented reality (AAR) applications have proliferated due to readily available earphones and speakers. AAR can provide omnidirectional engagement through the all-around sense of spatial information. Most existing AAR approaches offer immersive experiences by playing binaural spatial audios according to head-related transfer functions (HRTF). These involve complex modeling and require the user to wear a headphone. Air nonlinearity that can reproduce audible sounds from ultrasound offers opportunities to achieve device-free and omnidirectional sound source projection in AAR. This paper proposes Visar, a device-free virtual sound spots projection system leveraging air nonlinearity. Visar achieves simultaneous tracking and sound spot generation while suppressing unintended audio leakages caused by grating lobes and nonlinear effects in mixing lobes through optimization. Considering multi-user scenarios, Visar also proposed a multi-spot scheduling scheme to mitigate the mutual interference between the spots. Extensive experiments show the tracking error is 7.83cm and the orientation estimation error is 10.06°, respectively, envisioning the considerable potential of Visar in AAR applications.

Auditory-motor adaptation: induction of a lateral shift in sound localization after biased immersive virtual reality training

IntroductionSensorimotor adaptation has often been studied in the visual modality through the Prism Adaptation (PA) paradigm. In this paradigm, a lateral shift in visual pointing was found after wearing prismatic goggles. An effect of PA has sometimes been observed on hearing, in favor of a cross-modality recalibration. However, no study has ever shown if a biased auditory-motor adaptation could induce this lateral shift, which appears essential to a better understanding of the mechanisms of auditory adaptation. The present study aimed at inducing an auditory prism-like effect.MethodsSixty healthy young adults underwent a session of active audio-proprioceptive training in immersive virtual reality based on Head Related Transfer Functions (HRTF). This training consisted of a game in which the hand-held controller emitted sounds either at its actual position in a control group or at 10° or 20° to the right of its actual position in two experimental groups. Sound localization was assessed before and after the training.ResultsThe difference between both localization tests was significantly different between the three groups. As expected, the difference was significantly leftward for the group with a 20° deviation compared to the control group. However, this effect is due to a significant rightward deviation in the control group whereas no significant difference between localization tests emerged in the two experimental groups, suggesting that other factors such as fatigue may have cumulated with the training after-effect.DiscussionMore studies are needed to determine which angle of deviation and which number of sessions of this audio-proprioceptive training are required to obtain the best after-effect. Although the coupling of hearing and vision in PA still needs to be studied, adding spatial hearing to PA programs could be a promising way to reinforce after-effects and optimize their benefits.

Auditory localization: a comprehensive practical review.

Auditory localization is a fundamental ability that allows to perceive the spatial location of a sound source in the environment. The present work aims to provide a comprehensive overview of the mechanisms and acoustic cues used by the human perceptual system to achieve such accurate auditory localization. Acoustic cues are derived from the physical properties of sound waves, and many factors allow and influence auditory localization abilities. This review presents the monaural and binaural perceptual mechanisms involved in auditory localization in the three dimensions. Besides the main mechanisms of Interaural Time Difference, Interaural Level Difference and Head Related Transfer Function, secondary important elements such as reverberation and motion, are also analyzed. For each mechanism, the perceptual limits of localization abilities are presented. A section is specifically devoted to reference systems in space, and to the pointing methods used in experimental research. Finally, some cases of misperception and auditory illusion are described. More than a simple description of the perceptual mechanisms underlying localization, this paper is intended to provide also practical information available for experiments and work in the auditory field.

The effects of extended-wear hearing aids on the localization accuracy of listeners with normal audiometric thresholds.

Extended-wear hearing aids (EWHAs) are small broadband analog amplification devices placed deeply enough in the ear canal to preserve most of the cues in the head-related transfer function. However, little is known about how EWHAs affect localization accuracy for normal hearing threshold (NHT) listeners. In this study, eight NHT participants were fitted with EWHAs and localized broadband sounds of different durations (250 ms and 4 s) and stimulus intensities (40, 50, 60, 70, and 80 dBA) in a spherical speaker array. When the EWHAs were in the active mode, localization accuracy was only slightly degraded relative to open-ear performance. However, when the EWHAs were turned off, localization performance was substantially degraded even at the highest stimulus intensities. An electro-acoustical evaluation of the EWHAs showed minimal effects of dynamic range compression on the signals and good preservation of the signal pattern for vertical polar sound localization. Between-study comparisons suggest that EWHA active mode localization accuracy is favorable compared to conventional active earplugs, and EWHA passive mode localization accuracy is comparable to conventional passive earplugs. These results suggest that the deep-insertion analog design of the EWHA is generally better at preserving localization accuracy of NHT listeners than conventional earplug devices.

A Hybrid Algorithm for Predicting Head Related Transfer Function based on Physiological Parameters

A Hybrid Algorithm for Predicting Head Related Transfer Function based on Physiological Parameters

The Effect of Training on Localizing HoloLens-Generated 3D Sound Sources.

Sound localization is a crucial aspect of human auditory perception. VR (virtual reality) technologies provide immersive audio platforms that allow human listeners to experience natural sounds based on their ability to localize sound. However, the simulations of sound generated by these platforms, which are based on the general head-related transfer function (HRTF), often lack accuracy in terms of individual sound perception and localization due to significant individual differences in this function. In this study, we aimed to investigate the disparities between the perceived locations of sound sources by users and the locations generated by the platform. Our goal was to determine if it is possible to train users to adapt to the platform-generated sound sources. We utilized the Microsoft HoloLens 2 virtual platform and collected data from 12 subjects based on six separate training sessions arranged in 2 weeks. We employed three modes of training to assess their effects on sound localization, in particular for studying the impacts of multimodal error, visual, and sound guidance in combination with kinesthetic/postural guidance, on the effectiveness of the training. We analyzed the collected data in terms of the training effect between pre- and post-sessions as well as the retention effect between two separate sessions based on subject-wise paired statistics. Our findings indicate that, as far as the training effect between pre- and post-sessions is concerned, the effect is proven to be statistically significant, in particular in the case wherein kinesthetic/postural guidance is mixed with visual and sound guidance. Conversely, visual error guidance alone was found to be largely ineffective. On the other hand, as far as the retention effect between two separate sessions is concerned, we could not find any meaningful statistical implication on the effect for all three error guidance modes out of the 2-week session of training. These findings can contribute to the improvement of VR technologies by ensuring they are designed to optimize human sound localization abilities.

A multiplayer VR showdown game for people with visual impairment

ABSTRACT Researchers have developed virtual reality (VR) systems for people with visual impairment using various audio feedback techniques. However, few studies have investigated collaborative VR systems where people with and without visual impairment can participate together. We developed a VR Showdown, similar to a real showdown game, where players with and without visual impairment can play together in a remote virtual environment. We incorporated auditory distance perception using a head-related transfer function based on the spatial position of the Showdown ball in VR. We developed two modes of VR Showdown. One is the player vs. AI agent (PvA) mode, where visually impaired people can play alone. The other is the player vs. player (PvP) mode, in which visually impaired people can play with another player in a network. The user study results showed that people with and without visual impairment enjoyed playing VR Showdown games that were well-matched.

Effect of sound-induced vibrations of the pinna on head-related transfer functions: Experimental and numerical investigations.

Numerical simulations of head-related transfer functions (HRTFs) conventionally assume a rigid boundary condition for the pinna. The human pinna, however, is an elastic deformable body that can vibrate due to incident acoustic waves. This work investigates how sound-induced vibrations of the pinna can affect simulated HRTF magnitudes. The work will motivate the research question by measuring the sound-induced vibrational patterns of an artificial pinna with a high-speed holographic interferometric system. Then, finite element simulations are used to determine HRTFs for a tabletop model of the B&K 5128 head and torso simulator for a number of directions. Two scenarios are explored: one where the pinna is modeled as perfectly rigid, and another where the pinna is modeled as linear elastic with material properties close to that of auricular cartilage. The findings suggest that pinna vibrations have negligible effects on HRTF magnitudes up to 5 kHz. The same conclusion, albeit with less certainty, is drawn for higher frequencies. Finally, the importance of the elastic domain's material properties is emphasized and possible implications for validation studies on dummy heads 1as well as the limitations of the present work are discussed in detail.