Binaural Room Impulse Responses Research Articles

A multi-loudspeaker binaural room impulse response dataset with high-resolution translational and rotational head coordinates in a listening room

A multi-loudspeaker binaural room impulse response dataset with high-resolution translational and rotational head coordinates in a listening room

An end-to-end approach for blindly rendering a virtual sound source in an audio augmented reality environment

Audio augmented reality (AAR), a prominent topic in the field of audio, requires understanding the listening environment of the user for rendering an authentic virtual auditory object. Reverberation time (RT60\\documentclass[12pt]{minimal} \\usepackage{amsmath} \\usepackage{wasysym} \\usepackage{amsfonts} \\usepackage{amssymb} \\usepackage{amsbsy} \\usepackage{mathrsfs} \\usepackage{upgreek} \\setlength{\\oddsidemargin}{-69pt} \\begin{document}$$RT_{60}$$\\end{document}) is a predominant metric for the characterization of room acoustics and numerous approaches have been proposed to estimate it blindly from a reverberant speech signal. However, a single RT60\\documentclass[12pt]{minimal} \\usepackage{amsmath} \\usepackage{wasysym} \\usepackage{amsfonts} \\usepackage{amssymb} \\usepackage{amsbsy} \\usepackage{mathrsfs} \\usepackage{upgreek} \\setlength{\\oddsidemargin}{-69pt} \\begin{document}$$RT_{60}$$\\end{document} value may not be sufficient to correctly describe and render the acoustics of a room. This contribution presents a method for the estimation of multiple room acoustic parameters required to render close-to-accurate room acoustics in an unknown environment. It is shown how these parameters can be estimated blindly using an audio transformer that can be deployed on a mobile device. Furthermore, the paper also discusses the use of the estimated room acoustic parameters to find a similar room from a dataset of real BRIRs that can be further used for rendering the virtual audio source. Additionally, a novel binaural room impulse response (BRIR) augmentation technique to overcome the limitation of inadequate data is proposed. Finally, the proposed method is validated perceptually by means of a listening test.

Real-time impulse response: a methodology based on Machine Learning approaches for a rapid impulse response generation for real-time Acoustic Virtual Reality systems

Simulation of high-definition binaural room impulse responses using conventional approaches involves a significant amount of computational resources, resulting in high computational time, making these approaches incapable of performing real-time high quality acoustic virtual reality. This research implemented a methodology for the rapid impulse response generation using the position of a moving listener inside a fixed sound field. The rapid generation of the impulse response is performed using its representative compressed dimension, with a smaller dimension than the original impulse response, learned by variational autoencoders and long short-term memory neural networks. First, the methodology selects a representative number of impulse responses covering the area of interest using a reliable room acoustic simulator. Second, it generates a dataset with sufficient impulse responses uniformly distributed through a data augmentation approach using a modified bilinear interpolation from the impulse responses previously simulated. Third, it applies an unsupervised model to positionally cluster the impulse responses to reduce the variability of the impulse responses in the given environment. Fourth, it splits the impulse response into time segments and generates a dataset per segment and cluster. Fifth, it trains a variational autocoder with a long short-term memory neural network model for each time segment cluster of impulse responses to infer the correspondent compressed impulse response part. In summary, the impulse response is generated by assigning the current listener position to the corresponding cluster and executing the decoders of the variational autoencoders with long short-term memory, trained previously. The findings are encouraging; the normalized mean absolute error of the impulse responses gathered by the interpolator and the impulse responses generated by the proposed model is less than 15% in the 88% of impulse responses reserved for testing. Moreover, the average of the absolute error of the interaural cross-correlation coefficient between the impulse responses obtained from the simulator and the proposed model is around 16%, implying that most acoustic characteristics of the real impulse are preserved. In addition, the computational time for generating the impulse response segment of 300 ms is approximately 65 ms, which is almost haft than the total system latency for a realistic auralization, 112 ms.

Rapid BRIR generation approach using Variational Auto-Encoders and LSTM neural networks

This article proposes a rapid binaural room impulse response (BRIR) generation for real-time auralization with a moving receiver inside a room and a fixed sound source. The proposed method uses generative models, such as variational auto-encoders and long short-term memory neural networks. First, a number of static receivers are modeled within a given room, covering a representative area where listeners can typically be found. Then, a BRIR is simulated in each receiver, and from them, more BRIRs uniformly distributed in the scenario are synthesized through a data augmentation procedure. In the sequel, by applying an unsupervised model to cluster the receivers positionally, the BRIRs are organized into groups. Furthermore, variational autoencoder training is performed to reduce computational complexity. The results when comparing the reconstructed BRIRs with the original ones show that the error is within the just noticeable difference threshold. A computational cost analysis in terms of the total number of operations is carried out, achieving a computational gain of over 75%, and a storage reduction of more than 99% with respect to classic BRIR generation techniques. This opens the possibility of obtaining a reliable real-time auralization with a moving receiver inside the room.

Impact of direct-to-reverberation ratio and interaural cross-correlation on externalization in binaural audio rendering

To achieve realistic binaural reproduction of spatial audio in a headphone scenario, addressing inside-the-head-locatedness is crucial, which can be accomplished through appropriate externalization techniques. Two acoustic cues, the direct-to-reverberation ratio (DRR) and interaural cross-correlation (IACC), are known to influence perceived externalization. However, the precise relationship between these cues and their combined effect has not been extensively studied. In this investigation, we aim to explore the relative contribution of DRR and IACC to externalization by examining various combinations of these two acoustic cues. To this end, participants were exposed to various binaural room impulse responses (BRIRs) synthesized to produce three distinct levels of DRRs and IACCs for two different source directions. The listening test was conducted in an anechoic chamber with participants’ eyes blindfolded to minimize the influence of room divergence effect. We also investigated the influence of head tracking by conducting experiments with and without head trackers. The correlation maps between externalization and the two acoustic cues were generated for each source direction. The study reveals that the contribution of the two acoustic cues differs with and without head tracking, and the joint contribution also varies depending on the source direction and amount of reflections included in the BRIR.

A binaural room impulse response dataset and Shorelining psychophysical task for the evaluation of auditory sensory augmentation

Sensory augmentation using spatial sound presented in augmented-reality (AR) can assist people with low vision or blindness in navigating their environment (Katz et al., 2012). Nonetheless, in many situations, the poor quality of the binaural sound rendered using current tool sets limits the potential capability of the assistive technology. In particular, acoustic environments with near-field sources and reflections pose significant challenges. In this work, we provide a reference binaural room impulse response (BRIR) dataset with near-field sources and reflections and an associated shorelining psychophysical task that is useful for the evaluation of AR spatial audio. The dataset consists of 12∼small loudspeakers arranged on a 3-by-4 grid in a complex acoustic environment. BRIR measurements are recorded using the Head and Torso Simulator (HATS) for 17∼different receiver positions with a 5o angular resolution. Room impulse response measurements are also recorded using the Eigenmike for each of the 17 receiver positions. Using the Razer Anzu smart glasses to render the binaural AR spatial audio, we compare psychophysical performance on the shorelining navigation task using the recorded dataset and various existing binaural AR tool sets.

The localization of virtual source reproduced by the crosstalk cancellation system under different direct-to-reverberation energy ratios conditions

This study explores the localization of virtual sources reproduced by the crosstalk cancellation (CTC) system under reverberation environments with different direct-to-reverberation energy ratios (DRRs). Binaural room impulse responses in rectangle rooms are simulated by high-order image source method. Two independent key variables, the reverberation time (RT) and the distance of the sound source to the listener, are employed to manipulate DRR through simulation. Subsequently, a subjective localization experiment based on the CTC system is virtually conducted using earphone reproduction. The results show that the azimuth localization in horizontal plane is almost impervious by RT but is subjected to the change of the source distance. Specifically, as the distance of sound source increases, the perceived azimuth of lateral virtual sources exhibits a bias towards the front, accompanied by larger variances. The results indicate that reflections with small delays have a more detrimental effect on sound localization compared to the late reverberation with large intensity, which may be associated with the inhibition of late reflections based on the precedence effect. Additionally, the localization data are analyzed and explained using binaural cues, including the interaural cross-correlation coefficient, the interaural time difference (ITD), and interaural level difference (ILD).

Significance of Binaural Synthesis on the Spatial Acoustic Impression of Enclosures

Listening tests have been carried out to quantify the significance of binaural auralization over monaural auralization in accordance with the acoustic properties of the enclosure. To this end, acoustic rendering of three different rooms were generated based on synthesized monaural (two channels with the same audio material) and binaural room impulse responses. The auralizations were evaluated by means of subjective tests using headphones with non-individualized equalization. Parameters, such as localization, spatial impression and realism, were taken into consideration to determine the relevance of providing binaural information for the auralization of a given room. The analysis of the data has been conducted following a statistical approach based on ANOVA and Pearson correlation. The results indicate that spatial perception is strongly dependent on the acoustic characteristics of the rooms and on the listening condition of the audio material. Furthermore, as expected, advantages of binaural rendering in terms of source localization was also confirmed.

The effect of listener head orientation on front-to-rear speech intelligibility in an automotive cabin

The listener head orientation significantly affects speech intelligibility (SI) in the small acoustic space of an automobile, owing to near-field binaural listening and special acoustic conditions such as early reflections and seat-back occlusions. However, situations involving a back-row listener and a front-row speaker (i.e., front-to-rear scenarios) have not yet been studied. Such scenarios are dominated by non-uniform early reflections rather than direct sounds, making them distinct from cases considered in previous studies. This study investigates the effect of listener head orientation on SI in front-to-rear scenarios in an automobile, using both the speech transmission index (STI) and subjective experiments. A virtual speaker (i.e., a loudspeaker) is located in the driver’s seat and faces forward. Binaural room impulse responses are measured on a dummy head in the three rear-row seats with different head orientations. Using the binaural room impulse responses, the corresponding STIs are calculated, and the Chinese SI scores are measured virtually via headphones. The results show that the SI variation range for various listener head orientations in front-to-rear scenarios is greatly reduced compared with that in the rear-to-front scenarios considered in previous studies. Overall, the effect of listener head orientation on SI is mostly negligible when a listener in the rear row listens to a speaker in the front, with variations in the SI score of no more than 5%. The present study provides a supplement to previous studies and helps to deepen understanding of the effect of multiple factors on SI in an automotive cabin.

Binaural Auralization of Room Acoustics with a Highly Scalable Wave-Based Acoustics Simulation

This paper presents a proposal of an efficient binaural room-acoustics auralization method, an essential goal of room-acoustics modeling. The method uses a massively parallel wave-based room-acoustics solver based on a dispersion-optimized explicit time-domain finite element method (TD-FEM). The binaural room-acoustics auralization uses a hybrid technique of first-order Ambisonics (FOA) and head-related transfer functions. Ambisonics encoding uses room impulse responses computed by a parallel wave-based room-acoustics solver that can model sound absorbers with complex-valued surface impedance. Details are given of the novel procedure for computing expansion coefficients of spherical harmonics composing the FOA signal. This report is the first presenting a parallel wave-based solver able to simulate room impulse responses with practical computational times using an HPC cloud environment. A meeting room problem and a classroom problem are used, respectively, having 35 million degrees of freedom (DOF) and 100 million DOF, to test the parallel performance of up to 6144 CPU cores. Then, the potential of the proposed binaural room-acoustics auralization method is demonstrated via an auditorium acoustics simulation of up to 5 kHz having 750,000,000 DOFs. Room-acoustics auralization is performed with two acoustics treatment scenarios and room-acoustics evaluations that use an FOA signal, binaural room impulse response, and four room acoustical parameters. The auditorium acoustics simulation showed that the proposed method enables binaural room-acoustics auralization within 13,000 s using 6144 cores.

Effect of speaker orientation on speech intelligibility in an automotive environment

In addition to background noise, binaural effect, and the acoustic boundary conditions of a given space, speech intelligibility (SI) is affected by the source/speaker directivity or orientation, the latter of which has not been researched in small enclosed spaces, such as an automobile. Thus, this study aims to investigate the effect of speaker orientation on SI in automobiles, using both the speech transmission index (STI) and subjective experiments. Given the limited variation range for the orientation of rear-row speakers in an actual scenario, only the situation for front-row speakers was considered in this study. An artificial mouth was regarded as the speaker, which is placed at the driver’s seat. Binaural room impulse responses were measured for different listener locations and speaker orientations in an automobile and a listening room with negligible reflections. Subsequently, STIs were calculated and subjective speech reception thresholds (SRTs) in Mandarin Chinese were evaluated for the different listener locations and speaker orientations. The result shows that the SI in the listening room considerably decreases when the speaker orientation deviates from the listener due to the speaker directivity, with a best-ear STI (SRT) variation of up to 0.24 (7.4 dB). In comparison with the result in the listening room, early reflections improve SI in the automobile by a best-ear STI (SRT) increase (decrease) of up to 0.3 (7.7 dB). The speaker deviating from the listener signifies that the early reflections in the automobile afford a higher SI improvement. Thus, the variation range of SI with various speaker orientations in the automobile was greatly reduced compared with that in the listening room, with a best-ear STI (SRT) variation of no more than 0.1 (2.7 dB). Overall, the speaker orientation mostly does not affect SI except when the speaker is facing the listener due to rich early reflections inside an automobile. Moreover, the comparison between the subjective and objective results shows that the best-ear STI can well express the variation in the SRT in the automobile under various speaker orientations. This study bridges the gap in the research of SI considering various speaker orientations in an automotive environment and helps understand the combined effect of various factors on SI under the special acoustic condition in an automobile.

Perceptual Similarities between Artificial Reverberation Algorithms and Real Reverberation

This paper presents a study evaluating the perceptual similarity between artificial reverberation algorithms and acoustic measurements. An online headphone-based listening test was conducted and data were collected from 20 expert assessors. Seven reverberation algorithms were tested in the listening test, including the Dattorro, Directional Feedback Delay Network (DFDN), Feedback Delay Network (FDN), Gardner, Moorer, and Schroeder reverberation algorithms. A new Hybrid Moorer–Schroeder (HMS) reverberation algorithm was included as well. A solo cello piece, male speech, female singing, and a drumbeat were rendered with the seven reverberation algorithms in three different reverberation times (0.266 s, 0.95 s and 2.34 s) as the test conditions. The test was conducted online and based on the Multiple Stimuli with Hidden Reference and Anchor (MUSHRA) paradigm. The reference conditions consisted of the same audio samples convolved with measured binaural room impulse responses (BRIRs) with the same three reverberation times. The anchor was dual-mono 3.5 kHz low pass filtered audio. The similarity between the test audio and the reference audio was scored on a scale of zero to a hundred. Statistical analysis of the results shows that the Gardner and HMS reverberation algorithms are good candidates for exploration of artificial reverberation in Augmented Reality (AR) scenarios in future research.

Perceptual detection thresholds for numerical dispersion in binaural auralizations of two acoustically different rooms.

Room acoustic simulations using the finite-difference time-domain method on a wide frequency range can be computationally expensive and typically contain numerical dispersion. Numerical dispersion can be audible and, thus, constitutes an artifact in auralizations. There is a need to measure perceptual thresholds for numerical dispersion to achieve an optimal balance between computational complexity and audibility of dispersion. This work measures the perceptual detection thresholds for numerical dispersion in binaural auralizations of two acoustically different rooms. Numerical dispersion is incorporated into measured binaural room impulse responses (BRIRs) by the means of filters that represent the dispersion that plane waves experience, which propagate in the simulation in the direction of the worst-case dispersion error. The results show that the perceptual detection threshold is generally lower for the most reverberant room and greatly depends on the source signal independently of the room in which the threshold is measured. It is the most noticeable in the pure BRIRs, i.e., with an impulse as source signal, and almost unnoticeable with speech. The results also show that there was no statistical evidence that the perceptual thresholds for the conditions where numerical dispersion was present or absent in the direct path of the BRIRs be different.

Effect of electric-acoustic cochlear implant stimulation and coding strategies on spatial cues of speech signals in reverberant room

The comparison of spatial cues changes in different setups and coding strategies used in cochlear implants (CI) is investigated. In this experiment, we implement three voice coder setups, such as bilateral CI, bimodal CI, and electro-acoustic stimulation (EAS). Two well-known coding strategies are used, which are continuous interleaved sampling (CIS) and spectral peak (SPEAK). Speech signals are convoluted with appropriate binaural room impulse response (BRIR), creating reverberant spatial stimuli. Five different reverberant conditions (including anechoic) were applied to the stimuli. Interaural level and time differences (ILD and ITD) are evaluated objectively and subjectively, and their relationship with the intelligibility of speech is observed. Prior objective evaluation with CIS reveals that clarity (C50) becomes a more important factor in spatial cue change than reverberation time. Vocoded conditions (bilateral CI) show an increment in ILD value (compression has not been implemented yet on the vocoder processing), when the value of ITD gets more different (decreased) from the middle point. Reverberation degrades the intelligibility rate at various rates depending on the C50 value, both in unvocoded and vocoded conditions. In the vocoded condition, decrement on spatial cues was also followed by the decreement on the intelligibility of spatial stimuli.

Effect of spatial separation of sources on speech intelligibility in an automotive environment

Speech intelligibility has been shown to be enhanced when the target speech source is spatially separated from the interferer, which is related to the spatial unmasking phenomenon. However, this issue has not been studied in the small acoustical space of an automobile. This study, thus, examines the effect of spatial separation of sources on the speech reception threshold (SRT) in an automobile, and compared with the results in a weak-reflective listening room. The target was always presented at the front-passenger seat, and the interferer was presented at the front-passenger seat, right-back seat, mid-back seat, and left-back seat in sequence. The stimuli were synthesized using convolution with binaural room impulse responses measured on a dummy head in driver seat under different interferer locations. Sentence SRTs in Mandarin Chinese were measured via headphones virtually in an automobile and a listening room. Accordingly, the spatial release from masking (SRM) was obtained based on the SRT result. Results show that the SRM in automobile is always smaller than that in listening room, because the early reflections and seat-back occlusions cannot only obscure the binaural cues related to the source localization such as the interaural-time-difference and interaural-level-difference, but also influence the target-to-interferer ratio.

Effect of listener head orientation on speech reception threshold in an automotive environment

The head orientation of the listener significantly affects speech intelligibility (SI) in automobiles due to the effect of binaural listening and special acoustic conditions such as early reflections and seat-back occlusions. However, this issue has not been studied with subjective tests yet. This study investigates SI with various head orientations of a listener in automobiles with an subjective experiment. The sentence speech reception thresholds (SRT) in Mandarin Chinese are measured via headphones virtually in an automobile environment, and compared with results in a weak-reflective listening room. A virtual speaker is located in the front-passenger seat, the right-back seat and the left-back seat in sequence, by convolving the target speech with corresponding binaural room impulse responses measured on a dummy head in the driver’s seat with five head orientations. Result shows that the SRT variations caused by head orientations are up to 5 dB in automobile, lower than that of 9 dB in listening room. Under various head orientations, the SRT in automobile decreases as the virtual speaker moves closer to the front lateral direction, which is similar to the change rule in listening room that is mainly determined by the effect of binaural listening. Overall, a lower SRT in automobile can be obtained when the listener in the driver seat turns their head inward, i.e., towards the right. In comparison with the result in listening room, early reflections improve SI in automobile by an SRT decrease of up to 3.5 dB, while seat-back occlusions reduce SI by an SRT increase of up to 5.3 dB. The early reflections play a more significant role when the listener is in an adverse position, e.g., for the head orientations and speaker locations making direct sounds difficult to reach the listener’s ears. Comparison between the SRTs based on speech transmission index (STI) and STI-SI models and the SRTs measured by subjective experiment indicates that the STI-based objective method only partially expresses the variation in SI obtained from subjective experiment, and the difference between the subjective results and the objective results is mainly caused by the inapplicability of the STI-SI models derived from the traditional room in the automotive environment. The present work is relevant to understanding the combined effect of various factors on SI under such a special acoustic condition in automobile.

Impact Thresholds of Parameters of Binaural Room Impulse Responses (BRIRs) on Perceptual Reverberation

This paper presents a study on the perceived importance of different acoustic parameters of Binaural Room Impulse Response (BRIR) rendering. A headphone-based listening test was conducted with twenty expert participants. Three BRIRs generated from simulations of three different rooms were convolved with a dry speech signal and used as reference audio samples. Four BRIR parameters, Initial Time Delay Gap (ITDG), Forward Early Reflections (FER), Reverse Early Reflections (RER) and Late Reverberation (LR) were systematically altered and convolved with a speech signal to generate the test conditions. A staircase method was used to obtain the threshold at which each BRIR parameter was perceived as different from the reference audio sample. The average perceived impact threshold of each parameter was then calculated across the twenty participants. Results show that RER removal and ITDG extension have a clear impact on the perceptual reverberation of speech audio. Subjects were less sensitive to FER removal. The effect of LR removal on perceptual reverberation is hard to distinguish. Therefore, RER and ITDG are of particular importance when designing artificial reverberation algorithms, whilst more research is needed to understand the perceptual contribution of LR. Minor changes in FER and LR are less significant.

Favorable reverberation time in concert halls revisited for piano and violin solos.

A favorable reverberation time in concert halls is a fundamental issue for room acoustic design, and various recommendations have been suggested so far. Nevertheless, one must track back more than half a century when it comes to systematic subjective experiments on this topic. In this study, binaural room impulse responses (RIRs) measured with a dodecahedral loudspeaker in concert halls, where orchestra concerts are regularly held, are used. First, signal processing is applied to RIR to equalize the dodecahedral loudspeaker response as flat and broad as possible within the audible frequency range. Then, anechoic recordings of music by piano and violin, excerpts from cadenzas, are convoluted with the equalized RIR. Next, subjective experiments are conducted to seek favorable reverberation times. Reverberance and clarity were judged by 16 music experts. From this research, the favorable reverberation times RTM (octave band average for 500 and 1000 Hz) for piano and violin solos are from 1.2 to 2.0 s and 1.8 to 2.4 s, respectively. However, the clarity index C80,3 (octave band average for 500, 1000, and 2000 Hz) needs to range from 0 to 2.4 dB and -1.6 to 0.7 dB, respectively, to meet the optimum reverberance for piano and violin.

Binaural reproduction of dummy head and spherical microphone array data-A perceptual study on the minimum required spatial resolution.

Dynamic binaural synthesis requires binaural room impulse responses (BRIRs) for each head orientation of the listener. Such BRIRs can either be measured with a dummy head or calculated from the spherical microphone array (SMA) data. Because the dense dummy head measurements require enormous effort, alternatively sparse measurements can be performed and then interpolated in the spherical harmonics domain. The real-world SMAs, on the other hand, have a limited number of microphones, resulting in spatial undersampling artifacts. For both of the methods, the spatial order N of the underlying sampling grid influences the reproduction quality. This paper presents two listening experiments to determine the minimum spatial order for the direct sound, early reflections, and reverberation of the dummy head or SMA measurements required to generate the horizontally head-tracked binaural synthesis perceptually indistinguishable from a high-resolution reference. The results indicate that for direct sound, N = 9-13 is required for the dummy head BRIRs, but significantly higher orders of N = 17-20 are required for the SMA BRIRs. Furthermore, significantly lower orders are required for the late parts with N = 4-5 for the early reflections and reverberation of the dummy head BRIRs but N = 12-13 for the early reflections and N = 6-9 for the reverberation of the SMA BRIRs.

The Influence of Seat Occlusion on Driver’s Binaural Signal in Automobile based on FEM-RTM Simulation

Automotive environment is increasingly becoming an indispensable listening space in our daily life. Affected by the seat occlusions on sound propagation between the front-row and rear-row space, the speech communication between front and rear passengers in automobile is a well-known problem. However, this issue has been paid less attention. To this issue, the numerical simulation is a common, low-cost and high-efficiency technical approach. However, it is a well-known limitation on the effective frequencies to common numerical methods. To extend the region of effective frequencies, a combined approach of the finite element method (FEM) for low-frequency and the ray-tracing method (RTM) for high-frequency (i.e., FEM-RTM) was adopted in current work. The full-bandwidth binaural room impulse responses for a listener in driver’s seat were simulated with the speaker at mid-right seats, and both conditions with seats and without seats were taken into account. Result showed that the seats occlusions mainly affected the left ear (i.e., contralateral ear). The seat occlusions reduced the early decay time, speech transmission index and magnitude spectrum for left ear considerably. The inter-aural correlation coefficient under condition without seats was lower than that under condition with seats.