Individual Head-related Transfer Functions Research Articles

DNN-based HRTF individualization for accurate spectral cues using a compact PRTF

Head-Related Transfer Function (HRTF) plays a critical role in how the auditory system perceives spatial information. The spectral cues embedded in HRTF are vital for accurately determining the elevation of sound sources. In existing approaches, deep neural networks (DNNs) have been utilized to predict the magnitude spectra of HRTF from images of the pinna, typically employing the HRTF log-magnitude as the output during training. However, HRTF encompasses the acoustic characteristics of both the head and torso, exhibiting direction-dependent patterns that pose challenges in reconstructing its spectral cues. To address this complexity, we propose an innovative method for HRTF individualization. Our model uses Pinna-Related Transfer Function (PRTF) as the output during training, which helps alleviate the impact of sound reflections from the head and torso in the head-related impulse response (HRIR). Our experimental findings, based on an HRTF dataset, illustrate that our proposed model excels in reconstructing the first and second spectral cues. Furthermore, it outperforms previous deep learning models in terms of log spectral distortion (LSD).

HRTF individualization using ear images from 3D meshes

The head-related transfer function (HRTF) describes how a human receives sound from different directions in space. It is unique to each listener. The best way to obtain an individualized HRTF is through direct measurement, but it requires sophisticated laboratory equipment and long measurement times. This work proposes an approach for HRTF individualization employing ear images from 3D meshes using a deep neural network and spherical harmonics transform (SHT). The method relies on the HUTUBS dataset, including 3D meshes and HRTFs. The model uses ear images to predict a low-dimensional representation of the HRTF. Initially, ten images of the right ear of each 3D mesh are taken in different positions. The model consists of two main parts. The first is a convolutional neural network (CNN), which is used to extract features from the ear images. The second part learns from the feature map obtained and predict the spherical harmonic coefficients. Finally, the individualized HRTF is obtained through inverse SHT. The performance of the method is evaluated by computing the log-spectral distortion (LSD) between the measured HRTF and the predicted one. The results show favorable LSD values compared to other models addressing the same problem.

Efficient prediction of individual head-related transfer functions based on 3D meshes

Individual head-related transfer functions (HRTFs) are critical for binaural spatial audio rendering. In contrast to anthropometric parameters and pinnae images, 3D meshes allow for a more direct and comprehensive representation of the anthropometric structure, which provides highly effective inputs for modeling individualized HRTFs. This paper presents a neural network-based method for predicting individualized HRTFs in full space based on 3D meshes. Unlike many previous methods that estimate HRTF spectra at sampling grids or frequencies separately, the proposed model predicts the HRTF spectra of each vertical plane by considering the spectral correlation and continuity across adjacent sampling grids and frequencies. Evaluation results indicate that the proposed method enhances the prominence of peaks and notches in the obtained HRTF spectra and improves the speed and accuracy of HRTF individualization. The log spectral distortion of the proposed method is lower than that of state-of-the-art methods using anthropometric parameters and pinnae images. Further evaluation confirms that the proposed method requires significantly fewer points in 3D meshes when compared to numerical simulation methods. The evaluation based on localization models demonstrates that the HRTFs predicted by the proposed method are perceptually similar to the measured HRTFs.

Spatial release from masking in the median plane with non-native speakers using individual and mannequin head related transfer functions.

Spatial release from masking (SRM) in speech-on-speech tasks has been widely studied in the horizontal plane, where interaural cues play a fundamental role. Several studies have also observed SRM for sources located in the median plane, where (monaural) spectral cues are more important. However, a relatively unexplored research question concerns the impact of head-related transfer function (HRTF) personalisation on SRM, for example, whether using individually-measured HRTFs results in better performance if compared with the use of mannequin HRTFs. This study compares SRM in the median plane in a speech-on-speech virtual task rendered using both individual and mannequin HRTFs. SRM is obtained using English sentences with non-native English speakers. Our participants show lower SRM performances compared to those found by others using native English participants. Furthermore, SRM is significantly larger when the source is spatialised using the individual HRTF, and this effect is more marked for those with lower English proficiency. Further analyses using a spectral distortion metric and the estimation of the better-ear effect, show that the observed SRM can only partially be explained by HRTF-specific factors and that the effect of the familiarity with individual spatial cues is likely to be the most significant element driving these results.

Modeling of Individual Head-Related Transfer Functions (HRTFs) Based on Spatiotemporal and Anthropometric Features Using Deep Neural Networks

Modeling of Individual Head-Related Transfer Functions (HRTFs) Based on Spatiotemporal and Anthropometric Features Using Deep Neural Networks

Head-related transfer functions of rabbits within the front horizontal plane

The head-related transfer function (HRTF) describes the direction-dependent acoustic filtering by the head that occurs between a source signal in free-field space and the signal at the tympanic membrane. HRTFs contain information on sound source location via interaural differences of their magnitude or phase spectra and via the shapes of their magnitude spectra. The present study characterized HRTFs for source locations in the front horizontal plane for nine rabbits, which are a species commonly used in studies of the central auditory system. HRTF magnitude spectra shared several features across individuals, including a broad spectral peak at 2.6kHz that increased gain by 12 to 23dB depending on source azimuth; and a notch at 7.6kHz and peak at 9.8kHz visible for most azimuths. Overall, frequencies above 4kHz were amplified for sources ipsilateral to the ear and progressively attenuated for frontal and contralateral azimuths. The slope of the magnitude spectrum between 3 and 5kHz was found to be an unambiguous monaural cue for source azimuths ipsilateral to the ear. Average interaural level difference (ILD) between 5 and 16kHz varied monotonically with azimuth over ±31dB despite a relatively small head size. Interaural time differences (ITDs) at 0.5kHz and 1.5kHz also varied monotonically with azimuth over ±358 μs and ±260 μs, respectively. Remeasurement of HRTFs after pinna removal revealed that the large pinnae of rabbits were responsible for all spectral peaks and notches in magnitude spectra and were the main contribution to high-frequency ILDs (5–16kHz), whereas the rest of the head was the main contribution to ITDs and low-frequency ILDs (0.2–1.5kHz). Lastly, inter-individual differences in magnitude spectra were found to be small enough that deviations of individual HRTFs from an average HRTF were comparable in size to measurement error. Therefore, the average HRTF may be acceptable for use in neural or behavioral studies of rabbits implementing virtual acoustic space when measurement of individualized HRTFs is not possible.

Efficient representation of head-related transfer functions in continuous space–frequency domains

Utilizing spherical harmonic (SH) domain has been established as the default method of obtaining continuity over space in head-related transfer functions (HRTFs). This paper concerns different variants of extending this solution by replacing SHs with four-dimensional (4D) continuous functional models in which frequency is imagined as another physical dimension. Recently developed hyperspherical harmonic (HSH) representation is compared with models defined in spherindrical coordinate system by merging SHs with one-dimensional basis functions. The efficiency of both approaches is evaluated based on the reproduction errors for individual HRTFs from HUTUBS database, including detailed analysis of its dependency on chosen orders of approximation in frequency and space. Employing continuous functional models defined in 4D coordinate systems allows HRTF magnitude spectra to be expressed as a small set of coefficients which can be decoded back into values at any direction and frequency. The best performance was noted for HSHs and SHs merged with reverse Fourier–Bessel series, with the former featuring better compression abilities, achieving slightly higher accuracy for low number of coefficients. The presented models can serve multiple purposes, such as interpolation, compression or parametrization for machine learning applications, and can be applied not only to HRTFs but also to other types of directivity functions, e.g. sound source directivity.

The Accuracy of Dynamic Sound Source Localization and Recognition Ability of Individual Head-Related Transfer Functions in Binaural Audio Systems with Head Tracking

The use of audio systems that employ binaural synthesis with head tracking has become increasingly popular, particularly in virtual reality gaming systems. The binaural synthesis process uses the Head-Related Transfer Functions (HRTF) as an input required to assign the directions of arrival to sounds coming from virtual sound sources in the created virtual environments. Generic HRTFs are often used for this purpose to accommodate all potential listeners. The hypothesis of the research is that the use of individual HRTF in binaural synthesis instead of generic HRTF leads to improved accuracy and quality of virtual sound source localization, thus enhancing the user experience. A novel methodology is proposed that involves the use of dynamic virtual sound sources. In the experiments, the test participants were asked to determine the direction of a dynamic virtual sound source in both the horizontal and vertical planes using both generic and individual HRTFs. The gathered data are statistically analyzed, and the accuracy of localization is assessed with respect to the type of HRTF used. The individual HRTFs of the test participants are measured using a novel and efficient method that is accessible to a broad range of users.

Quantifying the impact of hearing-aid microphone placement and head-mounted devices on binaural cues from head-related transfer functions

Emerging techniques for assessment and training of spatial hearing use virtual reality (VR) devices to present auditory-visual cues. Sounds may be processed using head-related transfer functions (HRTFs), which capture directional acoustic cues, and visuals may be presented using a head-mounted display (HMD). For users of hearing devices such as hearing aids or cochlear implants, microphone placement may alter the binaural cues carried by the HRTF. Cues may also be distorted in presentations that combine HMD visuals with loudspeaker audio. This work seeks to understand these impacts by measuring and creating a database of HRTFs in the presence of hearing aids and HMDs. HRTFs were recorded within an anechoic chamber, using a standard manikin in a loudspeaker array located on the horizontal plane. Recordings were made with real sound sources of 5.625° resolution and virtual sources of 1° resolution. We will compare interaural time and level differences (ITDs and ILDs) from HRTFs measured through hearing-aid microphones with varied microphone placement (i.e., behind-the-ear and in-the-ear) and the presence versus absence of an HMD during the HRTF recording. Results will provide insights regarding HRTFs for individuals with hearing devices, as well as characterization of the HMD impact on VR-guided individual HRTF measurement.

Analysis of laser scanning and photogrammetric scanning accuracy on the numerical determination of Head-Related Transfer Functions of a dummy head

Individual Head-Related Transfer Functions (HRTFs) are necessary for the accurate rendering of virtual scenes. However, their acquisition is challenging given the complex pinna shape. Numerical methods can be leveraged to compute HRTFs on meshes originating from precise scans of a subject. Although photogrammetry can be used for the scanning, its inaccuracy might affect the spatial cues of simulated HRTFs. This paper aims to assess the significance of the photogrammetric error affecting a Neumann KU100 dummy head scan. The geometrical differences between the photogrammetric scan and a laser scan are mainly located at the pinna cavities. The computed photogrammetric HRTFs, compared to measured and simulated data using objective and perceptually inspired metrics, show deviation in high frequency spectral features, stemming from the photogrammetric scanning error. This spectral deviation hinders the modelled elevation perception with photogrammetric HRTFs to levels comparable to renderings with nonindividual data. Extracting the photogrammetric geometry at individual ear cavities and merging it to the laser mesh, an assessment of the influence of the inaccuracy at different pinna structures is conducted. Correlation analysis between acoustic and geometrical metrics computed on the results is used to identify the most relevant geometrical metrics in relation to the HRTFs.

Modeling individual head-related transfer functions from sparse measurements using a convolutional neural network.

Individual head-related transfer functions (HRTFs) are usually measured with high spatial resolution or modeled with anthropometric parameters. This study proposed an HRTF individualization method using only spatially sparse measurements using a convolutional neural network (CNN). The HRTFs were represented by two-dimensional images, in which the horizontal and vertical ordinates indicated direction and frequency, respectively. The CNN was trained by using the HRTF images measured at specific sparse directions as input and using the corresponding images with a high spatial resolution as output in a prior HRTF database. The HRTFs of a new subject can be recovered by the trained CNN with the sparsely measured HRTFs. Objective experiments showed that, when using 23 directions to recover individual HRTFs at 1250 directions, the spectral distortion (SD) is around 4.4 dB; when using 105 directions, the SD reduced to around 3.8 dB. Subjective experiments showed that the individualized HRTFs recovered from 105 directions had smaller discrimination proportion than the baseline method and were perceptually undistinguishable in many directions. This method combines the spectral and spatial characteristics of HRTF for individualization, which has potential for improving virtual reality experience.

Spatial release of masking in children and adults in non-individualized virtual environments.

The spatial release of masking (SRM) is often measured in virtual auditory environments created from head-related transfer functions (HRTFs) of a standardized adult head. Adults and children, however, differ in head dimensions and mismatched HRTFs are known to affect some aspects of binaural hearing. So far, there has been little research on HRTFs in children and it is unclear whether a large mismatch of spatial cues can degrade speech perception in complex environments. In two studies, the effect of non-individualized virtual environments on SRM accuracy in adults and children was examined. The SRMs were measured in virtual environments created from individual and non-individualized HRTFs and the equivalent real anechoic environment. Speech reception thresholds (SRTs) were measured for frontal target sentences and symmetrical speech maskers at 0° or ±90° azimuth. No significant difference between environments was observed for adults. In 7 to 12-year-old children, SRTs and SRMs improved with age, with SRMs approaching adult levels. SRTs differed slightly between environments and were significantly worse in a virtual environment based on HRTFs from a spherical head. Adult HRTFs seem sufficient to accurately measure SRTs in children even in complex listening conditions.

Estimation of the Optimal Spherical Harmonics Order for the Interpolation of Head-Related Transfer Functions Sampled on Sparse Irregular Grids

Conventional individual head-related transfer function (HRTF) measurements are demanding in terms of measurement time and equipment. For more flexibility, free body movement (FBM) measurement systems provide an easy-to-use way to measure full-spherical HRTF datasets with less effort. However, having no fixed measurement installation implies that the HRTFs are not sampled on a predefined regular grid but rely on the individual movements of the subject. Furthermore, depending on the measurement effort, a rather small number of measurements can be expected, ranging, for example, from 50 to 150 sampling points. Spherical harmonics (SH) interpolation has been extensively studied recently as one method to obtain full-spherical datasets from such sparse measurements, but previous studies primarily focused on regular full-spherical sampling grids. For irregular grids, it remains unclear up to which spatial order meaningful SH coefficients can be calculated and how the resulting interpolation error compares to regular grids. This study investigates SH interpolation of selected irregular grids obtained from HRTF measurements with an FBM system. Intending to derive general constraints for SH interpolation of irregular grids, the study analyzes how the variation of the SH order affects the interpolation results. Moreover, the study demonstrates the importance of Tikhonov regularization for SH interpolation, which is popular for solving ill-posed numerical problems associated with such irregular grids. As a key result, the study shows that the optimal SH order that minimizes the interpolation error depends mainly on the grid and the regularization strength but is almost independent of the selected HRTF set. Based on these results, the study proposes to determine the optimal SH order by minimizing the interpolation error of a reference HRTF set sampled on the sparse and irregular FBM grid. Finally, the study verifies the proposed method for estimating the optimal SH order by comparing interpolation results of irregular and equivalent regular grids, showing that the differences are small when the SH interpolation is optimally parameterized.

Interaural time difference individualization in HRTF by scaling through anthropometric parameters

Head-related transfer function (HRTF) individualization can improve the perception of binaural sound. The interaural time difference (ITD) of the HRTF is a relevant cue for sound localization, especially in azimuth. Therefore, individualization of the ITD is likely to result in better sound spatial localization. A study of ITD has been conducted from a perceptual point of view using data from individual HRTF measurements and subjective perceptual tests. Two anthropometric dimensions have been demonstrated in relation to the ITD, predicting the subjective behavior of various subjects in a perceptual test. With this information, a method is proposed to individualize the ITD of a generic HRTF set by adapting it with a scale factor, which is obtained by a linear regression formula dependent on the two previous anthropometric dimensions. The method has been validated with both objective measures and another perceptual test. In addition, practical regression formula coefficients are provided for fitting the ITD of the generic HRTFs of the widely used Brüel & Kjær 4100 and Neumann KU100 binaural dummy heads.

Toolkit for individualization of head-related transfer functions using parametric notch-peak model

Although sound image control and sound field reproduction based on head-related transfer functions (HRTFs) have been studied for years, they have not been put to practical use. The main reason is that individual differences in HRTFs have not been overcome. In the present paper, we propose the parametric notch-peak HRTF model (PNP model), which can continuously change the parameters corresponding to individual differences in HRTFs by using knowledge of spectral notches and peaks and an HRTF database. We then develop a toolkit that can generate individual HRTFs by having the listener adjust the parameters. Finally, localization tests are carried out in order to examine the performance of the individualized HRTFs using the PNP model.

On the role of head rotation in solving the HRTF inverse problem

The sound arriving at the eardrum is a complex composition of the sound source spectrum, the acoustic properties of the listener’s environment, and the directionally dependent filtering of the listener’s head-related transfer function (HRTF). Listeners only ever have access to this mixture, and it remains unknown how they are nonetheless capable of disentangling features intrinsic to the sound source from those created by the source’s position in the environment. We hypothesized that head movements, by changing only the HRTF-related components of an arriving sound, inform a listener’s judgement of source spectrum. We tested this with virtual acoustics, presenting listeners with a speech sound rendered at an elevated frontal location and asked them to judge which of two subsequent copies of the same sound rendered at the horizon had spectral noise added to them. The test sounds were either head-locked, world-locked, or moved inverted to the listener’s movements, and were rendered with either individual or non-individual HRTFs. Error patterns across HRTF and motion conditions appear to confirm the hypothesis that head rotation plays a role in the estimation of sound source spectrum, and further indicate that listeners have expectations about how the signal at the ear should change as the head moves.

Dynamic Binaural Rendering: The Advantage of Virtual Artificial Heads over Conventional Ones for Localization with Speech Signals

As an alternative to conventional artificial heads, a virtual artificial head (VAH), i.e., a microphone array-based filter-and-sum beamformer, can be used to create binaural renderings of spatial sound fields. In contrast to conventional artificial heads, a VAH enables one to individualize the binaural renderings and to incorporate head tracking. This can be achieved by applying complex-valued spectral weights—calculated using individual head related transfer functions (HRTFs) for each listener and for different head orientations—to the microphone signals of the VAH. In this study, these spectral weights were applied to measured room impulse responses in an anechoic room to synthesize individual binaural room impulse responses (BRIRs). In the first part of the paper, the results of localizing virtual sources generated with individually synthesized BRIRs and measured BRIRs using a conventional artificial head, for different head orientations, were assessed in comparison with real sources. Convincing localization performances could be achieved for virtual sources generated with both individually synthesized and measured non-individual BRIRs with respect to azimuth and externalization. In the second part of the paper, the results of localizing virtual sources were compared in two listening tests, with and without head tracking. The positive effect of head tracking on the virtual source localization performance confirmed a major advantage of the VAH over conventional artificial heads.

Head-Related Transfer Functions for Dynamic Listeners in Virtual Reality

In dynamic virtual reality, visual cues and motor actions aid auditory perception. With multimodal integration and auditory adaptation effects, generic head-related transfer functions (HRTFs) may yield no significant disadvantage to individual HRTFs regarding accurate auditory perception. This study compares two individual HRTF sets against a generic HRTF set by way of objective analysis and two subjective experiments. First, auditory-model-based predictions examine the objective deviations in localization cues between the sets. Next, the HRTFs are compared in a static subjective (N=8) localization experiment. Finally, the localization accuracy, timbre, and overall quality of the HRTF sets are evaluated subjectively (N=12) in a six-degrees-of-freedom audio-visual virtual environment. The results show statistically significant objective deviations between the sets, but no perceived localization or overall quality differences in the dynamic virtual reality.

A Graphical-User-Interface-Based Azimuth-Collection Method in Autonomous Auditory Localization of Real and Virtual Sound Sources.

Auditory localization of spatial sound sources is an important life skill for human beings. For the practical application-oriented measurement of auditory localization ability, the preference is a compromise among (i) data accuracy, (ii) the maneuverability of collecting directions, and (iii) the cost of hardware and software. The graphical user interface (GUI)-based sound-localization experimental platform proposed here (i) is cheap, (ii) can be operated autonomously by the listener, (iii) can store results online, and (iv) supports real or virtual sound sources. To evaluate the accuracy of this method, by using 12 loudspeakers arranged in equal azimuthal intervals of 30° in the horizontal plane, three groups of azimuthal localization experiments are conducted in the horizontal plane with subjects with normal hearing. In these experiments, the azimuths are reported using (i) an assistant, (ii) a motion tracker, or (iii) the newly designed GUI-based method. All three groups of results show that the localization errors are mostly within 5-12°, which is consistent with previous results from different localization experiments. Finally, the stimulus of virtual sound sources is integrated into the GUI-based experimental platform. The results with the virtual sources suggest that using individualized head-related transfer functions can achieve better performance in spatial sound source localization, which is consistent with previous conclusions and further validates the reliability of this experimental platform.

Sensitivity analysis of pinna morphology on head-related transfer functions simulated via a parametric pinna model.

The head-related transfer function (HRTF) defines the acoustic path from a source to the two ears of a listener in a manner that is highly dependent on direction. This directional dependence arises from the highly individual morphology of the pinna, which results in complex reflections and resonances. While this notion is generally accepted, there has been little research on the importance of different structural elements of the pinna on the HRTF. A parametric three-dimensional ear model was used to investigate the changes in shape of the pinna in a systematic manner with a view to determining important contributing morphological parameters that can be used for HRTF individualization. HRTFs were simulated using the boundary element method. The analysis comprised objective comparisons between the directional transfer function and diffuse field component. The mean spectral distortion was used for global evaluation of HRTF similarity across all simulated positions. A perceptual localization model was used to determine correspondences between perceptual cues and objective parameters. A reasonable match was found between the modelled perceptual results and the mean spectral distortion. Modifications to the shape of the concha were found to have an important impact on the HRTF, as did those in proximity to the triangular fossa. Furthermore, parameters that control the relief of the pinna were found to be at least as important as more frequently cited side-facing parameters, highlighting limitations in previous morphological/HRTF studies.