Set Of Head-related Transfer Functions Research Articles

On the generalization of accommodation to head-related transfer functions.

To date, there is strong evidence indicating that humans with normal hearing can adapt to non-individual head-related transfer functions (HRTFs). However, less attention has been given to studying the generalization of this adaptation to untrained conditions. This study investigated how adaptation to one set of HRTFs can generalize to another set of HRTFs. Participants were divided into two groups and trained to localize a speech stimulus reproduced binaurally using either individual or non-individual HRTFs. Training led to an improved localization performance with the trained HRTFs for both groups of participants. Results also showed that there was no difference in the localization performance improvement between the trained and untrained HRTFs for both groups, indicating a generalization of adaptation to HRTFs. The findings did not allow to precisely determine which type of learning (procedural or perceptual) primarily contributed to the generalization, thus highlighting the potential need to expose participants to longer training protocols.

A Perceptual Study on the Plausibility of Binaural Rendering in Anechoic Environments

Plausibility has become a widely used paradigm for evaluating binaural audio for augmented reality. Although several studies have evaluated the plausibility of different binaural rendering techniques, it is not clear which parameters contribute to plausibility and to what extent. The present study attempts to fill this gap and presents the results of five listening experiments systematically investigating the impact of different parameters. A first plausibility test performed in anechoic conditions assessed the plausibility of nonindividual head-related transfer-function rendering. In four subsequent experiments, the influence of spherical harmonics truncation order, visual cues, small translational head movements, and different head-related transfer-function sets is assessed. Overall, the results indicate that even large manipulations of the rendering system have surprisingly little impact on the estimated d’.

Ensemble width estimation in HRTF-convolved binaural music recordings using an auditory model and a gradient-boosted decision trees regressor

Binaural audio recordings become increasingly popular in multimedia repositories, posing new challenges in indexing, searching, and retrieval of such excerpts in terms of their spatial audio scene characteristics. This paper presents a new method for the automatic estimation of one of the most important spatial attributes of binaural recordings of music, namely “ensemble width.” The method has been developed using a repository of 23,040 binaural excerpts synthesized by convolving 192 multi-track music recordings with 30 sets of head-related transfer functions (HRTF). The synthesized excerpts represented various spatial distributions of music sound sources along a frontal semicircle in the horizontal plane. A binaural auditory model was exploited to derive the standard binaural cues from the synthesized excerpts, yielding a dataset representing interaural level and time differences, complemented by interaural cross-correlation coefficients. Subsequently, a regression method, based on gradient-boosted decision trees, was applied to the formerly calculated dataset to estimate ensemble width values. According to the obtained results, the mean absolute error of the ensemble width estimation averaged across experimental conditions amounts to 6.63° (SD 0.12°). The accuracy of the method is the highest for the recordings with ensembles narrower than 30°, yielding the mean absolute error ranging between 0.8° and 10.2°. The performance of the proposed algorithm is relatively uniform regardless of the horizontal position of an ensemble. However, its accuracy deteriorates for wider ensembles, with the error reaching 25.2° for the music ensembles spanning 90°. The developed method exhibits satisfactory generalization properties when evaluated both under music-independent and HRTF-independent conditions. The proposed method outperforms the technique based on “spatiograms” recently introduced in the literature.

Validation of sound source localization quality in binaural audio synthesis with head-tracking

Binaural synthesis with head tracking has become a highly important tool in creating an auditory experience for users that is as similar as possible to their perception of sound in the real world. This is crucial in many different applications that rely on accurate representation of real or virtual sound environments, ranging from scientific research to gaming applications. The quality of the binaural synthesis process and the accuracy and naturalness of the produced result depend on numerous factors, ranging from the selection of suitable IMU sensor technology used for head tracking to the choice of methods for implementing binaural synthesis using generic or individual sets of Head-Related Transfer Functions (HRTF). This is especially significant in Virtual Reality (VR) systems with three or six degrees of freedom that fully immerse the users in the virtual world through visual and auditory stimuli. This paper presents the authors' experiences in addressing a variety of challenges that arose during the design of such spatial audio systems and the testing of sound source localization quality. Strong emphasis is put on the testing of sound source localization quality, where the position of the source dynamically changes in relation to the listener.

Grid-agnostic personalized head-related transfer function modeling with neural fields

The spatial filtering effect brought on by sound propagation from the sound source to the outer ear is referred to as the head-related transfer function (HRTF). The personalization of HRTF is essential to enhance the personalized immersive audio experience in virtual and augmented reality. Our work aims to employ deep learning to predict the customized HRTF from anthropometric measurements. However, existing measured HRTF databases each employ a different geographic sampling, making it difficult to combine these databases into training data-hungry deep learning methods while each of them only contains dozens of subjects. Following our previous work, we use a neural field, a neural network that maps the spherical coordinates to the magnitude spectrum to represent each subject’s set of HRTFs. We constructed a generative model to learn the latent space across subjects using such a consistent representation of HRTF across datasets. In this work, by learning the mapping of the anthropometric measurements to the latent space and then reconstructing the HRTF, we further investigate the neural field representation to carry out HRTF personalization. Thanks to the grid-agnostic nature of our method, we are able to train on combined datasets and even validate the performance on grids unseen during training.

Estimation of the Optimal Spherical Harmonics Order for the Interpolation of Head-Related Transfer Functions Sampled on Sparse Irregular Grids

Conventional individual head-related transfer function (HRTF) measurements are demanding in terms of measurement time and equipment. For more flexibility, free body movement (FBM) measurement systems provide an easy-to-use way to measure full-spherical HRTF datasets with less effort. However, having no fixed measurement installation implies that the HRTFs are not sampled on a predefined regular grid but rely on the individual movements of the subject. Furthermore, depending on the measurement effort, a rather small number of measurements can be expected, ranging, for example, from 50 to 150 sampling points. Spherical harmonics (SH) interpolation has been extensively studied recently as one method to obtain full-spherical datasets from such sparse measurements, but previous studies primarily focused on regular full-spherical sampling grids. For irregular grids, it remains unclear up to which spatial order meaningful SH coefficients can be calculated and how the resulting interpolation error compares to regular grids. This study investigates SH interpolation of selected irregular grids obtained from HRTF measurements with an FBM system. Intending to derive general constraints for SH interpolation of irregular grids, the study analyzes how the variation of the SH order affects the interpolation results. Moreover, the study demonstrates the importance of Tikhonov regularization for SH interpolation, which is popular for solving ill-posed numerical problems associated with such irregular grids. As a key result, the study shows that the optimal SH order that minimizes the interpolation error depends mainly on the grid and the regularization strength but is almost independent of the selected HRTF set. Based on these results, the study proposes to determine the optimal SH order by minimizing the interpolation error of a reference HRTF set sampled on the sparse and irregular FBM grid. Finally, the study verifies the proposed method for estimating the optimal SH order by comparing interpolation results of irregular and equivalent regular grids, showing that the differences are small when the SH interpolation is optimally parameterized.

Spatial up-sampling of HRTF sets using generative adversarial networks: A pilot study

Headphones-based spatial audio simulations rely on Head-related Transfer Functions (HRTFs) in order to reconstruct the sound field at the entrance of the listener’s ears. A HRTF is strongly dependent on the listener’s specific anatomical structures, and it has been shown that virtual sounds recreated with someone else’s HRTF result in worse localisation accuracy, as well as altering other subjective measures such as externalisation and realism. Acoustic measurements of the filtering effects generated by ears, head and torso has proven to be one of the most reliable ways to obtain a personalised HRTF. However this requires a dedicated and expensive setup, and is time-intensive. In order to simplify the measurement setup, thereby improving the scalability of the process, we are exploring strategies to reduce the number of acoustic measurements without degrading the spatial resolution of the HRTF. Traditionally, spatial up-sampling of HRTF sets is achieved through barycentric interpolation or by employing the spherical harmonics framework. However, such methods often perform poorly when the provided HRTF data is spatially very sparse. This work investigates the use of generative adversarial networks (GANs) to tackle the up-sampling problem, offering an initial insight about the suitability of this technique. Numerical evaluations based on spectral magnitude error and perceptual model outputs are presented on single spatial dimensions, therefore considering sources positioned only in one of the three main planes: Horizontal, median, and frontal. Results suggest that traditional HRTF interpolation methods perform better than the proposed GAN-based one when the distance between measurements is smaller than 90°, but for the sparsest conditions (i.e., one measurement every 120°–180°), the proposed approach outperforms the others.

Spatial Audio Scene Characterization (SASC): Automatic Localization of Front-, Back-, Up-, and Down-Positioned Music Ensembles in Binaural Recordings

The automatic localization of audio sources distributed symmetrically with respect to coronal or transverse planes using binaural signals still poses a challenging task, due to the front–back and up–down confusion effects. This paper demonstrates that the convolutional neural network (CNN) can be used to automatically localize music ensembles panned to the front, back, up, or down positions. The network was developed using the repository of the binaural excerpts obtained by the convolution of multi-track music recordings with the selected sets of head-related transfer functions (HRTFs). They were generated in such a way that a music ensemble (of circular shape in terms of its boundaries) was positioned in one of the following four locations with respect to the listener: front, back, up, and down. According to the obtained results, CNN identified the location of the ensembles with the average accuracy levels of 90.7% and 71.4% when tested under the HRTF-dependent and HRTF-independent conditions, respectively. For HRTF-dependent tests, the accuracy decreased monotonically with the increase in the ensemble size. A modified image occlusion sensitivity technique revealed selected frequency bands as being particularly important in terms of the localization process. These frequency bands are largely in accordance with the psychoacoustical literature.

Automatic discrimination between front and back ensemble locations in HRTF-convolved binaural recordings of music

One of the greatest challenges in the development of binaural machine audition systems is the disambiguation between front and back audio sources, particularly in complex spatial audio scenes. The goal of this work was to develop a method for discriminating between front and back located ensembles in binaural recordings of music. To this end, 22, 496 binaural excerpts, representing either front or back located ensembles, were synthesized by convolving multi-track music recordings with 74 sets of head-related transfer functions (HRTF). The discrimination method was developed based on the traditional approach, involving hand-engineering of features, as well as using a deep learning technique incorporating the convolutional neural network (CNN). According to the results obtained under HRTF-dependent test conditions, CNN showed a very high discrimination accuracy (99.4%), slightly outperforming the traditional method. However, under the HRTF-independent test scenario, CNN performed worse than the traditional algorithm, highlighting the importance of testing the algorithms under HRTF-independent conditions and indicating that the traditional method might be more generalizable than CNN. A minimum of 20 HRTFs are required to achieve a satisfactory generalization performance for the traditional algorithm and 30 HRTFs for CNN. The minimum duration of audio excerpts required by both the traditional and CNN-based methods was assessed as 3 s. Feature importance analysis, based on a gradient attribution mapping technique, revealed that for both the traditional and the deep learning methods, a frequency band between 5 and 6 kHz is particularly important in terms of the discrimination between front and back ensemble locations. Linear-frequency cepstral coefficients, interaural level differences, and audio bandwidth were identified as the key descriptors facilitating the discrimination process using the traditional approach.

Head-Related Transfer Functions for Dynamic Listeners in Virtual Reality

In dynamic virtual reality, visual cues and motor actions aid auditory perception. With multimodal integration and auditory adaptation effects, generic head-related transfer functions (HRTFs) may yield no significant disadvantage to individual HRTFs regarding accurate auditory perception. This study compares two individual HRTF sets against a generic HRTF set by way of objective analysis and two subjective experiments. First, auditory-model-based predictions examine the objective deviations in localization cues between the sets. Next, the HRTFs are compared in a static subjective (N=8) localization experiment. Finally, the localization accuracy, timbre, and overall quality of the HRTF sets are evaluated subjectively (N=12) in a six-degrees-of-freedom audio-visual virtual environment. The results show statistically significant objective deviations between the sets, but no perceived localization or overall quality differences in the dynamic virtual reality.

Impact of non-individualised head related transfer functions on speech-in-noise performances within a synthesised virtual environment.

When performing binaural spatialisation, it is widely accepted that the choice of the head related transfer functions (HRTFs), and in particular the use of individually measured ones, can have an impact on localisation accuracy, externalization, and overall realism. Yet the impact of HRTF choices on speech-in-noise performances in cocktail party-like scenarios has not been investigated in depth. This paper introduces a study where 22 participants were presented with a frontal speech target and two lateral maskers, spatialised using a set of non-individual HRTFs. Speech reception threshold (SRT) was measured for each HRTF. Furthermore, using the SRT predicted by an existing speech perception model, the measured values were compensated in the attempt to remove overall HRTF-specific benefits. Results show significant overall differences among the SRTs measured using different HRTFs, consistently with the results predicted by the model. Individual differences between participants related to their SRT performances using different HRTFs could also be found, but their significance was reduced after the compensation. The implications of these findings are relevant to several research areas related to spatial hearing and speech perception, suggesting that when testing speech-in-noise performances within binaurally rendered virtual environments, the choice of the HRTF for each individual should be carefully considered.

Head-related transfer function recommendation based on perceptual similarities and anthropometric features.

Individualization of head-related transfer functions (HRTFs) can improve the quality of binaural applications with respect to the localization accuracy, coloration, and other aspects. Using anthropometric features (AFs) of the head, neck, and pinna for individualization is a promising approach to avoid elaborate acoustic measurements or numerical simulations. Previous studies on HRTF individualization analyzed the link between AFs and technical HRTF features. However, the perceptual relevance of specific errors might not always be clear. Hence, the effects of AFs on perceived perceptual qualities with respect to the overall difference, coloration, and localization error are directly explored. To this end, a listening test was conducted in which subjects rated differences between their own HRTF and a set of nonindividual HRTFs. Based on these data, a machine learning model was developed to predict the perceived differences using ratios of a subject's individual AFs and those of presented nonindividual AFs. Results show that perceived differences can be predicted well and the HRTFs recommended by the models provide a clear improvement over generic or randomly selected HRTFs. In addition, the most relevant AFs for the prediction of each type of error were determined. The developed models are available under a free cultural license.

Comparison of direct and indirect perceptual head-related transfer function selection methods.

When a personalized set of head-related transfer functions (HRTFs) is not available, a common solution is identifying a perceptually appropriate substitute from a database. There are various approaches to this selection process whether based on localization cues, subjective evaluations, or anthropomorphic similarities. This study investigates whether HRTF rankings that stem from different selection methods yield comparable results. A perceptual study was carried out using a basic source localization method and a subjective quality judgment method for a common set of eight HRTFs. HRTF rankings were determined according to different metrics from each method for each subject and the respective results were compared. Results indicate a significant and positive mean correlation between certain metrics. The best HRTFs selected according to one method had significant above-average rating scores according to metrics in the second method.

An Efficient Implementation of Parallel Parametric HRTF Models for Binaural Sound Synthesis in Mobile Multimedia

The extended use of mobile multimedia devices in applications like gaming, 3D video and audio reproduction, immersive teleconferencing, or virtual and augmented reality, is demanding efficient algorithms and methodologies. All these applications require real-time spatial audio engines with the capability of dealing with intensive signal processing operations while facing a number of constraints related to computational cost, latency and energy consumption. Most mobile multimedia devices include a Graphics Processing Unit (GPU) that is primarily used to accelerate video processing tasks, providing high computational capabilities due to its inherent parallel architecture. This paper describes a scalable parallel implementation of a real-time binaural audio engine for GPU-equipped mobile devices. The engine is based on a set of head-related transfer functions (HRTFs) modelled with a parametric parallel structure, allowing efficient synthesis and interpolation while reducing the size required for HRTF data storage. Several strategies to optimize the GPU implementation are evaluated over a well-known kind of processor present in a wide range of mobile devices. In this context, we analyze both the energy consumption and real-time capabilities of the system by exploring different GPU and CPU configuration alternatives. Moreover, the implementation has been conducted using the OpenCL framework, guarantying the portability of the code.

Short-term effects of sound localization training in virtual reality

Head-related transfer functions (HRTFs) capture the direction-dependant way that sound interacts with the head and torso. In virtual audio systems, which aim to emulate these effects, non-individualized, generic HRTFs are typically used leading to an inaccurate perception of virtual sound location. Training has the potential to exploit the brain’s ability to adapt to these unfamiliar cues. In this study, three virtual sound localization training paradigms were evaluated; one provided simple visual positional confirmation of sound source location, a second introduced game design elements (“gamification”) and a final version additionally utilized head-tracking to provide listeners with experience of relative sound source motion (“active listening”). The results demonstrate a significant effect of training after a small number of short (12-minute) training sessions, which is retained across multiple days. Gamification alone had no significant effect on the efficacy of the training, but active listening resulted in a significantly greater improvements in localization accuracy. In general, improvements in virtual sound localization following training generalized to a second set of non-individualized HRTFs, although some HRTF-specific changes were observed in polar angle judgement for the active listening group. The implications of this on the putative mechanisms of the adaptation process are discussed.

Spectral manipulation improves elevation perception with non-individualized head-related transfer functions.

Spatially rendering sounds using head-related transfer functions (HRTFs) is an important part of creating immersive audio experiences for virtual reality applications. However, elevation perception remains challenging when generic, non-personalized HRTFs are used. This study investigated whether digital audio effects applied to a generic set of HRTFs could improve sound localization in the vertical plane. Several of the tested effects significantly improved elevation judgment, and trial-by-trial variability in spectral energy between 2 and 10 kHz correlated strongly with perceived elevation. Digital audio effects may therefore be a promising strategy to improve elevation perception where personalized HRTFs are not available.

Do We Need Individual Head-Related Transfer Functions for Vertical Localization? The Case Study of a Spectral Notch Distance Metric

This paper deals with the issue of individualizing the head-related transfer function (HRTF) rendering process for auditory elevation perception. Is it possible to find a nonindividual, personalized HRTF set that allows a listener to have an equally accurate localization performance than with his/her individual HRTFs? We propose a psychoacoustically motivated, anthropometry based mismatch function between HRTF pairs that exploits the close relation between the listener's pinna geometry and localization cues. This is evaluated using an auditory model that computes a mapping between HRTF spectra and perceived spatial locations. Results on a large number of subjects in the center for image processing and integrated computing (CIPIC) and acoustics research institute (ARI) HRTF databases suggest that there exists a nonindividual HRTF set, which allows a listener to have an equally accurate vertical localization than with individual HRTFs. Furthermore, we find the optimal parameterization of the proposed mismatch function, i.e., the one that best reflects the information given by the auditory model. Our findings show that the selection procedure yields statistically significant improvements with respect to dummy-head HRTFs or random HRTF selection, with potentially high impact from an applicative point of view.

A Recommender System for Improving Median Plane Sound Localization Performance Based on a Nonlinear Representation of HRTFs

We propose a new method to improve median plane sound localization performance using a nonlinear representation of head-related transfer functions (HRTFs) and a recommender system. First, we reduce the dimensionality of an HRTF data set with multiple subjects using manifold learning in conjunction with a customized intersubject graph which takes into account relevant prior knowledge of HRTFs. Then, we use a sound localization model to estimate a subject’s localization performance in terms of polar error and quadrant error rate. These metrics are merged to form a single rating per HRTF pair that we feed into a recommender system. Finally, the recommender system takes the low-dimensional HRTF representation as well as the ratings obtained from the localization model to predict the best HRTF set, possibly constructed by mixing HRTFs from different individuals, that minimizes a subject’s localization error. The simulation results show that our method is capable of choosing a set of HRTFs that improves the median plane localization performance with respect to the mean localization performance using non-individualized HRTFs. Moreover, the localization performance achieved by our HRTF recommender system shows no significant difference to the localization performance observed with the best matching non-individualized HRTFs but with the advantage of not having to perform listening tests with all individuals’ HRTFs from the database.

Head-Related Transfer Function Selection Using Neural Networks

AbstractIn binaural audio systems, for an optimal virtual acoustic space a set of head-related transfer functions (HRTFs) should be used that closely matches the listener’s ones. This study aims to select the most appropriate HRTF dataset from a large database for users without the need for extensive listening tests. Currently, there is no way to reliably reduce the number of datasets to a smaller, more manageable number without risking discarding potentially good matches. A neural network that estimates the appropriateness of HRTF datasets based on input vectors of anthropometric measurements is proposed. The shapes and sizes of listeners’ heads and pinnas were measured using digital photography; the measured anthropometric parameters form the feature vectors used by the neural network. A graphical user interface (GUI) was developed for participants to listen to music transformed using different HRTFs and to evaluate the fitness of each HRTF dataset. The listening scores recorded were the target outputs used to train the neural networks. The aim was to learn a mapping between anthropometric parameters and listener’s perception scores. Experimental validations were performed on 30 subjects. It is demonstrated that the proposed system produces a much more reliable HRTF selection than previously used methods.

Contributions of head-related transfer function choice and head tracking to virtual loudspeaker binaural rendering

This presentation will provide an overview of recent and ongoing studies regarding evaluations of sound fields using virtual loudspeaker binaural synthesis. Of specific interest is an identification of perceptual attributes affected by Head-Related Transfer Function (HRTF) choice beyond basic localization error and the sensitivity of listeners to head tracking with regards to latency and externalization judgments. A list of perceptual attributes, created using a Consensus Vocabulary Protocol elicitation method, and validated through listening tests, resulted in eight valid perceptual attributes for describing the perceptual dimensions affected by HRTF set variations. Employing prescribed head movements, sensitivity to head tracker latency showed small but significant differences between single and multichannel audio source scenes. A similar protocol was employed to comparing the sense of externalization as a function of head rotation with and without head tracking. In contrast to several previous studies, results showed that head movements can substantially enhance externalization, especially for frontal and rear sources, and that externalization can persist even though the subject has stopped moving his/her head. These works were carried out during the course of the French funded BiLi (Binaural Listening) project (FUI-AAP14).