Virtual Acoustic Environments Research Articles

Virtual acoustic environments for perception research

Human sound perception includes the ability to process auditory information from dynamic acoustic situations, which is an essential component for communication and orientation in everyday life. In contrast, standard psychoacoustic experiments don't include real-life situations.They include rather simple acoustic stimuli with nominally high reproducibility but low significance for situations in real life. The gap in research on auditory perception between the laboratory and real life can be closed now. Virtual Reality technology is a very powerful tool which connects real-time computer simulation with 3D devices for auditory-visual impressions. Virtual acoustics can incorporate any situational context and scene conditions with sources and sound propagation features in given environments. In consequence, ground-breaking research in auditory perception can be expected with higher degree of realism and with still high reproducibility. The presentation will highlight guidelines for creation of Virtual Acoustic Environments and examples for application in noise assessment, psychoacoustic research, and hearing diagnosis and rehabilitation.

Measurement of Head-Related Transfer Functions: A Review

A head-related transfer function (HRTF) describes an acoustic transfer function between a point sound source in the free-field and a defined position in the listener’s ear canal, and plays an essential role in creating immersive virtual acoustic environments (VAEs) reproduced over headphones or loudspeakers. HRTFs are highly individual, and depend on directions and distances (near-field HRTFs). However, the measurement of high-density HRTF datasets is usually time-consuming, especially for human subjects. Over the years, various novel measurement setups and methods have been proposed for the fast acquisition of individual HRTFs while maintaining high measurement accuracy. This review paper provides an overview of various HRTF measurement systems and some insights into trends in individual HRTF measurements.

Mapping the perceptual topology of auditory space permits the creation of hyperstable virtual acoustic environments

Mapping the perceptual topology of auditory space permits the creation of hyperstable virtual acoustic environments

SCaLAr – A surrounding spherical cap loudspeaker array for flexible generation and evaluation of virtual acoustic environments

Introduction: Surrounding spherical loudspeaker arrays facilitate the application of various spatial audio reproduction methods and can be used for a broad range of acoustic measurements and perceptual evaluations. Methods: Installed in an anechoic chamber, the design and implementation of such an array of 68 coaxial loudspeakers, sampling a spherical cap with a radius of 1.35 m on an equal-area grid, is presented. A network-based audio backbone enables low-latency signal transmission with low-noise amplifiers providing a high signal-to-noise ratio. To address batch-to-batch variations, the loudspeaker transfer functions were equalised by individually designed 512-taps finite impulse response filters. Time delays and corresponding level adjustments further helped to minimise radial mounting imperfections. Results: The equalised loudspeaker transfer functions measured under ideal conditions and when mounted, their directivity patterns, and in-situ background noise levels satisfy key criteria towards applicability. Advantages and shortcomings of the selected decoders for panning-based techniques, as well as the influence of loudspeaker positioning errors, are analysed in terms of simulated performance metrics. An evaluation of the achievable channel separation allows deriving recommendations of feasible subset layouts for loudspeaker-based binaural reproduction. Conclusion: The combination of electroacoustic properties, simulated sound field synthesis performance and measured channel separation classifies the system as suitable for its target applications.

MobiLab – A Mobile Laboratory for On-Site Listening Experiments in Virtual Acoustic Environments

Virtual acoustic environments have demonstrated their versatility for conducting studies in various research areas as they allow easy manipulations of experimental test conditions or simulated acoustic scenes, while providing expansion possibilities to related interdisciplinary and multimodal fields. Although the evolution of auditory and cognitive models is consistently pursued, listening experiments are still considered the gold standard, usually necessitating a large amount of resources, including travel expenses of study participants. In order to facilitate practical and efficient study execution, we therefore implemented a mobile hearing laboratory by acoustically optimising the interior of a caravan. All necessary technical facilities were integrated to perform listening experiments in virtual acoustic environments under controlled conditions directly on site, for example, in front of schools or senior residential centers. The design and construction of this laboratory are presented and evaluated based on insulation properties, selected room acoustic parameters, and interior ambient noise levels that are to be expected during operation at representative test sites. Limitations, particularly in low-frequency insulation performance, should provide incentives for further optimisations in similar future projects.

A Toolbox for Rendering Virtual Acoustic Environments in the Context of Audiology

A toolbox for creation and rendering of dynamic virtual acoustic environments (TASCAR) that allows direct user interaction was developed for application in hearing aid research and audiology. This technical paper describes the general software structure and the time-domain simulation methods, i.e., transmission model, image source model, and render formats, used to produce virtual acoustic environments with moving objects. Implementation-specific properties are described, and the computational performance of the system was measured as a function of simulation complexity. Results show that on commercially available commonly used hardware the simulation of several hundred virtual sound sources is possible in the time domaina

Real-time auralization of a talker’s own voice in virtual rooms

While much has been done in the field of sound auralization in virtual rooms, the problem of hearing one’s own voice in these environments has received less attention. A robust and feasible system for real-time auralization of talkers who are also listeners is needed. To address this requirement, a real-time convolution system (RTCS) was designed with the specific goal of “placing” a talker/listener in virtual acoustic environments. This system necessitated the development of several tools and methods. Oral-binaural room impulse responses were measured and characterized for a variety of room. The RTCS improved on past systems, in part through the derivation and inclusion of compensation filters, which corrected the linear auditory distortions of the RTCS components. Objective measures in the time- and frequency-domains were developed to assess the validity of the system. A jury-based listening study also indicated that RTCS users could speak and listen to their own voices in the virtual acoustic environments in a natural manner.While much has been done in the field of sound auralization in virtual rooms, the problem of hearing one’s own voice in these environments has received less attention. A robust and feasible system for real-time auralization of talkers who are also listeners is needed. To address this requirement, a real-time convolution system (RTCS) was designed with the specific goal of “placing” a talker/listener in virtual acoustic environments. This system necessitated the development of several tools and methods. Oral-binaural room impulse responses were measured and characterized for a variety of room. The RTCS improved on past systems, in part through the derivation and inclusion of compensation filters, which corrected the linear auditory distortions of the RTCS components. Objective measures in the time- and frequency-domains were developed to assess the validity of the system. A jury-based listening study also indicated that RTCS users could speak and listen to their own voices in the virtual acoustic environme...

Spatial release from masking in reverberation for children and adults with normal hearing

The ability to perform spatial release from masking (SRM), i.e., benefiting from a spatial separation between target and distracting talkers, is an important developmental skill for children to navigate noisy environments such as classrooms. While studies on children’s SRM were mostly conducted in anechoic rooms or sound booths to date, little is yet known how realistic room acoustics including reverberation in everyday listening affect this ability. In this study, we measured SRM from children and adults with normal hearing in virtual acoustic environments that mimic typical classrooms with different acoustic conditions. Two virtual classrooms were simulated with mean mid-frequency reverberation times (RTs) of 0.4 and 1.1 s, one within and the other poorer than the recommendation of classroom acoustics standard (ANSI S12.60). Overall, children performed more poorly than adults on both speech intelligibility and SRM. Children’s speech intelligibility of the target speech masked by distractor voices was better under the less reverberant condition. Interestingly, while adults showed better SRM under RT = 0.4 s, children did not perform any differently between the two RT conditions. The findings will be discussed for further investigation and implications for classroom acoustic designs. [Work supported by the EU 7th Framework Programme, ITN FP7-607139.]

Neural tracking of auditory motion is reflected by delta phase and alpha power of EEG

It is of increasing practical interest to be able to decode the spatial characteristics of an auditory scene from electrophysiological signals. However, the cortical representation of auditory space is not well characterized, and it is unclear how cortical activity reflects the time-varying location of a moving sound. Recently, we demonstrated that cortical response measures to discrete noise bursts can be decoded to determine their origin in space. Here we build on these findings to investigate the cortical representation of a continuously moving auditory stimulus using scalp recorded electroencephalography (EEG).In a first experiment, subjects listened to pink noise over headphones which was spectro-temporally modified to be perceived as randomly moving on a semi-circular trajectory in the horizontal plane. While subjects listened to the stimuli, we recorded their EEG using a 128-channel acquisition system. The data were analysed by 1) building a linear regression model (decoder) mapping the relationship between the stimulus location and a training set of EEG data, and 2) using the decoder to reconstruct an estimate of the time-varying sound source azimuth from the EEG data. The results showed that we can decode sound trajectory with a reconstruction accuracy significantly above chance level. Specifically, we found that the phase of delta (<2 Hz) and power of alpha (8–12 Hz) EEG track the dynamics of a moving auditory object.In a follow-up experiment, we replaced the noise with pulse train stimuli containing only interaural level and time differences (ILDs and ITDs respectively). This allowed us to investigate whether our trajectory decoding is sensitive to both acoustic cues. We found that the sound trajectory can be decoded for both ILD and ITD stimuli. Moreover, their neural signatures were similar and even allowed successful cross-cue classification. This supports the notion of integrated processing of ILD and ITD at the cortical level. These results are particularly relevant for application in devices such as cognitively controlled hearing aids and for the evaluation of virtual acoustic environments.

Effects of non-individual head-related transfer functions and visual mismatch on speech recognition in virtual acoustic environments

There is widespread research and clinical interest in quantifying how the acoustics of real-world environments, such as background noise and reverberation, impede a listener’s ability to recognize speech. Conventional methods used to quantify these effects include dichotic listening via headphones in sound-attenuated booths or loudspeakers in anechoic or low-reverberant environments, which lack the capability of manipulating room acoustics. Using a state-of-the-art Variable Room Acoustics System housed in a virtual sound room (ViSoR), this study aims to systematically assess the effects of non-individual head-related transfer functions (HRTFs) and mismatched visual perception on speech recognition in virtual acoustic environments. Young adults listened to and repeated sentences presented amidst a co-located two-talker speech competitor with reverberation times ranging from 0.4 to 1.25 s. Sentences were presented in three listening conditions: through a loudspeaker array in ViSoR with the participants’ own HRTFs (Condition 1); via headphones in a sound-attenuated booth with non-individual HRTFs(Condition 2); and using the same binaural reproduction as Condition 2 in ViSoR (Condition 3). Condition 3 serves as a control condition, allowing us to quantify the separate effects of non-individual HRTFs and visual mismatch on speech recognition. Discussion will address the validity and use of virtual acoustics in research and clinical settings.

High-resolution room virtualization using first-order source-and-receiver-directional room impulse responses

Virtual acoustic environments benefit from involving the source and receiver directivity to become fully interactive. In this contribution we discuss virtualization of a room based on measured source-and-receiver-directional room impulse responses (SRD RIR), captured by first-order-directional source and receiver arrays. Low order arrays may in some ways overcome a directional versus temporal resolution trade off one encounters in directional measurements. This is done by employing the spatial decomposition method of room impulse responses on both the first-order source and receiver side. We discuss the result of these resolution-enhanced and efficiently measured SRD RIR based on a comparative listening experiment involving the variable-directivity icosahedral loudspeaker (IKO), a spherical beamforming loudspeaker array, whose perceptual sculptural effects have been studied in a room. For the virtualization of the IKO, high resolution directivity measurement of the IKO are inserted, and for its rendering, a high-resolution set of head related impulse responses (HRIRs).

A hybrid real-time auralization system for binaural reproduction of virtual acoustic environments with simulated room acoustics adapted for hearing aid users

A recently developed binaural real-time reproduction system has been extended with an interface to research hearing aids allowing for the conduction of auditory research on subjects with hearing loss. Simulated hearing aid signals based on measurements of generic hearing aid-related transfer functions are additionally processed on a master hearing aid to emulate conventional hearing aid algorithms and played back through the hearing aid receivers. Designed for subjects with mild to moderate hearing loss, the system also facilitates the use of residual hearing capabilities by simulating an external sound field based on generic head-related transfer functions which is reproduced via loudspeakers and acoustic crosstalk cancellation filters. For increased ecological validity, plausible room acoustics are simulated using adjusted simulation models relying on geometrical acoustics. The proposed system was evaluated objectively on different levels by investigating the listening environment and various system components, running a benchmark analysis on the acoustical simulation and auralization, and measuring the combined system latency.

An Extended Binaural Real-Time Auralization System With an Interface to Research Hearing Aids for Experiments on Subjects With Hearing Loss.

Theory and implementation of acoustic virtual reality have matured and become a powerful tool for the simulation of entirely controllable virtual acoustic environments. Such virtual acoustic environments are relevant for various types of auditory experiments on subjects with normal hearing, facilitating flexible virtual scene generation and manipulation. When it comes to expanding the investigation group to subjects with hearing loss, choosing a reproduction system which offers a proper integration of hearing aids into the virtual acoustic scene is crucial. Current loudspeaker-based spatial audio reproduction systems rely on different techniques to synthesize a surrounding sound field, providing various possibilities for adaptation and extension to allow applications in the field of hearing aid-related research. Representing one option, the concept and implementation of an extended binaural real-time auralization system is presented here. This system is capable of generating complex virtual acoustic environments, including room acoustic simulations, which are reproduced as combined via loudspeakers and research hearing aids. An objective evaluation covers the investigation of different system components, a simulation benchmark analysis for assessing the processing performance, and end-to-end latency measurements.

Angle-Dependent Distortions in the Perceptual Topology of Acoustic Space.

By moving sounds around the head and asking listeners to report which ones moved more, it was found that sound sources at the side of a listener must move at least twice as much as ones in front to be judged as moving the same amount. A relative expansion of space in the front and compression at the side has consequences for spatial perception of moving sounds by both static and moving listeners. An accompanying prediction that the apparent location of static sound sources ought to also be distorted agrees with previous work and suggests that this is a general perceptual phenomenon that is not limited to moving signals. A mathematical model that mimics the measured expansion of space can be used to successfully capture several previous findings in spatial auditory perception. The inverse of this function could be used alongside individualized head-related transfer functions and motion tracking to produce hyperstable virtual acoustic environments.

Interactive reproduction of virtual acoustic environments for the evaluation of hearing devices—Methods and validation

Virtual acoustic environments are increasingly used for evaluating hearing devices in complex acoustic conditions. In this talk we propose an interactive simulation method via multi-channel loudspeaker systems or headphones. The method focuses on the time-domain simulation of the direct path and a geometric image source model, which simulates air absorption and in case of motion the Doppler effect of all primary and image sources. To establish the feasibility of this approach, the interaction between reproduction method and technical and perceptual hearing aid performance measures was investigated using computer simulations. Three spatial audio reproduction methods were compared in regular circular loudspeaker arrays with 4 to 72 channels. The influence of reproduction method and array size on performance measures of multi-microphone hearing aid algorithms was analyzed. In addition to the analysis of reproduction methods, algorithm performance was tested in a number of different virtual acoustic environments in order to assess the underlying factors of decreased hearing aid performance in complex environments. The results confirm previous findings that spatial complexity has a major impact on hearing aid benefit, and demonstrate the potential of virtual acoustic environments for hearing aid evaluation. [Funded by DFG FOR1732 “Individualized hearing acoustics.”]

Binaural reproduction of self-generated sound in virtual acoustic environments

Virtual acoustics aims to immerse the user in a virtual acoustic environment (VAE). However, most VAE systems do not feed self-generated sound back into the virtual room, even though there is evidence that adequate reproduction of self-generated sound affects the user’s perception and might even enhance immersion. Thus, if at all possible, sonic interaction between the user and the virtual room is very limited in most current systems. This work presents a VAE system that is able to capture and reproduce self-generated sound in real time. Hence, the VAE is complemented with a reactive component providing the acoustic response to the actions of the user. The major difference compared to the few reactive VAEs introduced so far is that the system presented here considers the varying directivity of the user or the sound source, and that it generally works with any arbitrary source. The study includes a first technical evaluation of the system as well as an example application of a virtual concert hall. The reactive VAE can be used as a virtual practice room for musicians, or as a tool for psychoacoustic experiments investigating the influence of self-generated sound on human perceptual processes.

The perceptual evaluation of acoustical environments I: Simulated environments

The successful design and development of numerical models for the sound propagation in rooms crucially depends on the existence of appropriate measures for the quality assessment of the modeling results. In the perceptual domain, these should include single-number measures for the overall assessment of the simulation as well as differential measures, enabling experts to identify specific shortcomings in the selected modeling approach and its implementation. In the SEACEN consortium, the “authenticity” and the “plausibility” of virtual acoustic environments has been established, measuring the perceived identity with an external (given) or internal reference. Moreover, a Spatial Audio Quality Inventory (SAQI) was developed by a focus group of experts for virtual acoustics, as a metric for the differential diagnosis of virtual environments.

Simulation of acoustic environments—Part 1: Computer models

Simulation and auralization techniques are established in applications of room acoustics for quite a while. Research projects are currently focusing on a coordinated effort to improve the complete signal chain from the numerical modeling, the data acquisition within numerical or real sound fields, the coding and transmission to the electro-acoustic reproduction by binaural technology or by sound field synthesis. Approaches for the comparative evaluation of real and simulated environments will enable the evaluation of the plausibility and/or the authenticity of virtual acoustic environments. The state of the art is revisited and discussed along the series of the three past “round robins on room acoustic computer simulation.”

Open community platform for hearing aid algorithm research

The project “Open community platform for hearing aid algorithm research” funded by the National Institutes of Health (NIH Grant 1R01DC015429-01) aims at sustainable, focused research toward improvement and new types of assistive hearing systems. To this end, an open-source software platform for real-time audio signal processing will be developed and made available to the research community including a standard set of reference algorithms. Furthermore, novel algorithms for dynamic and frequency compression, auditory-scene-analysis based noise suppression and speech enhancement, and feedback management will be investigated. For a realistic assessment of the benefits of hearing aid algorithms and combinations thereof, instrumental measures of performance in virtual acoustic environments of varying complexity will be included in the algorithm design and optimization. With such a quasi-standard set of benchmarks and the means to develop and integrate own signal-processing methods and measures in the same framework, the platform enables reproducible, comparative studies and collaborative research efforts. Beyond an implementation for PC hardware the system will also be made usable for ARM-processor based hardware to allow pre-development of wearable audio devices—so-called “hearables.” This contribution will present underlying previous work and the goals and plans of the project that has started midyear 2016. www.openMHA.org.

Multi-channel loudspeaker reproduction and virtual acoustic environments in the context of hearing aid evaluation

Virtual acoustic environments reproduced via multiple loudspeakers are increasingly used for evaluating hearing aids in complex acoustic conditions. To further establish the feasibility of this approach, this study investigated the interaction between spatial resolution of different reproduction methods and technical and perceptual hearing aid performance measures using computer simulations. Three spatial audio reproduction methods—discrete speakers, vector base amplitude panning and higher order ambisonics—were compared in regular circular loudspeaker arrays with 4 to 72 channels. The influence of reproduction method and array size on performance measures of multi-microphone hearing aid algorithm classes with spatially distributed microphones and a representative single channel noise-reduction algorithm was analyzed. In addition to the analysis of reproduction methods, algorithm performance was tested in a number of different virtual acoustic environments in order to assess the underlying factors of decr...