Smartphone-derived 3D head and pinnae modeling for personalized 3D virtual audio

Abstract

3D audio is commonplace augmented or virtual reality environments, and is used simulate the direction of arrival for sounds presented over headphones. The Head-Related Transfer Function (HRTF) enables 3D audio by characterizing how sound is received from a sound source in every direction. Optimal sound localization performance with HRTF-based 3D audio is thought to require personalized HRTFs because the localization cues are dependent on the size and shape of the head and pinnae. In this work, we propose a prototype pipeline to generate HRTFs from high resolution smartphone videos of the head and ears, and we validate the localization performance of the resulting HRTFs through human-subject behavioral experiments. To create a personalized 3D HRTF, a video was taken of a subject’s torso, left ear and right ear using a smartphone camera. Acoustic simulation with Boundary Element Modeling was performed in COMSOL, taking an approximately 1 h to complete. Preliminary virtual sound localization results were obtained on a pilot subject for each of 3 different HRTF types: A generic HRTF, a gold standard acoustically measured HRTF, and the smartphone video HRTF. We found a mean angle error of 36.6, 21.1, and 14.0 deg, respectively, indicating that the smartphone-based HRTF developed in this study provides higher localization accuracy than other methods on this pilot subject. Further testing will add additional subjects, and compare to free-field localization over loudspeakers.