Late Reverberation Research Articles

Echo-aware room impulse response generation.

In real-time applications, like interactive virtual reality environments, there is a significant need for low-complexity simulation of room impulse responses in highly complex virtual scenes, but this remains a challenging issue. In particular, simulating late reverberation using physically based acoustic modeling requires much computational effort, contrary to the early reflections that can be modeled by simpler techniques, e.g., the image source method. To tackle this computational complexity issue, we propose a neural network-based hybrid artificial reverberation framework (Echo2Reverb) that generates late reverberation from given early reflections. The proposed model can control both temporal texture and frequency-dependent energy decay, i.e., echo density and spectral energy distribution, of the generated reverberations by extracting spectral and echo-related features and filtering sampled sparse sequences and Gaussian noises using estimated features. To support the end-to-end training with controlled echo density, a differentiable approximation of the normalized echo density profile is proposed. We train and test the model not only for nearly diffuse but also distinct echoes prominent in late reverberations, such as with flutter echoes in narrow corridors. Evaluation results demonstrate that the proposed model can accurately reproduce frequency-dependent energy decay and temporal texture of a room impulse response using only early reflections.

Non-Exponential Reverberation Modeling Using Dark Velvet Noise

Previous research on late-reverberation modeling has mainly focused on exponentially decaying room impulse responses, whereas methods for accurately modeling non-exponential reverberation remain challenging. This paper extends the previously proposed basic darkvelvet- noise reverberation algorithm and proposes a parametrization scheme for modeling late reverberation with arbitrary temporal energy decay. Each pulse in the velvet-noise sequence is routed to a single dictionary filter that is selected from a set of filters based on weighted probabilities. The probabilities control the spectral evolution of the late-reverberation model and are optimized to fit a target impulse response via non-negative least-squares optimization. In this way, the frequency-dependent energy decay of a target late-reverberation impulse response can be fitted with mean and maximum reverberation-time errors of 4% and 8%, respectively, requiring about 50% less coloration filters than a previously proposed filteredvelvet- noise algorithm. Furthermore, the extended dark-velvet-noise reverberation algorithm allows the modeled impulse response to be gated, the frequency-dependent reverberation time to be modified, and the model’s spectral evolution and broadband decay to be decoupled. The proposed method is suitable for the parametric late-reverberation synthesis of various acoustic environments, especially spaces that exhibit a non-exponential energy decay, motivating its use in musical audio and virtual reality.

Joint dereverberation and blind source separation using a hybrid autoregressive and convolutive transfer function-based model

Most frequency-domain blind source separation (BSS) methods are based on the multiplicative narrowband assumption, which is not valid in long reverberation environments. In contrast, convolutive transfer function (CTF)-based BSS methods do not rely on the narrowband assumption, and the separation performance is significantly improved compared to the traditional algorithms in long reverberation environments. However, the CTF-based BSS methods and their variants, e.g., autoregressive (AR) BSS methods, introduce modeling errors to some extent, due to the truncation or approximation during the optimization process. To address this problem, we propose a frequency-domain BSS method employing a hybrid AR and CTF model, which can provide more precise representations of the early reflections and late reverberations. Furthermore, we utilize the Gaussian noise model to deal with the BSS problem in noisy reverberant environments. We formulate the objective function using the maximum log-likelihood criterion, and derive an efficient iterative algorithm for parameter estimation with the block coordinate descent (BCD) method. Experimental results show that the proposed method has a better separation performance than the existing methods in long reverberation environments.

Directional reverberation time and the image source method for rectangular parallelepipedal rooms.

The image source (IS) method is a commonly used geometrical acoustics simulation technique in room and virtual acoustics. In particular, it has been used in the analysis of room reverberation under different choices of geometry and wall conditions. Under a simple rectangular parallelepipedal geometry, reverberation time is known to be dependent on the direction of arrival of reflections relative to the room axes. In this article, a closed-form expression for the directional energy decay and reverberation time is derived, which is valid in the late response, and may be used in the case of either angle-independent or angle-dependent reflection. The expression reduces to an easily evaluated formula in the case of an omnidirectional energy decay curve (EDC). Various numerical results are presented, including the validation of the closed-form expression against EDCs and late reverberation times drawn directly from the IS method.

Directional distribution of the pseudo intensity vector in anisotropic late reverberation.

The pseudo intensity vector (PIV) is often used to analyze the directional properties of spatial room impulse responses. In the early part of the response, it is capable of estimating the directions of individual reflections. However, thus far, its behaviour in the late field is unclear. Specifically, it is unknown whether anisotropy, i.e., a direction-dependent energy distribution, is captured by the directional estimates. In this study, a closed-form expression of the directional distribution of the pressure-normalized pseudo intensity vector contingent on a general stochastical model of anisotropic fields was analytically derived. This paper shows that the probability density function of this PIV is a multivariate Cauchy distribution, which does indeed depend on the energy distribution of the field, yet the directional distribution has very limited degrees of freedom. The derived distribution is compared to the results of Monte Carlo simulations and fields captured with a microphone array in a real room. These results facilitate better understanding of the behaviour of parametric spatial room impulse response methods and may enable improved directional estimators for anisotropic fields.

Co-immersion in Audio Augmented Virtuality: the Case Study of a Static and Approximated Late Reverberation Algorithm.

In immersive Audio Augmented Reality, a virtual sound source should be indistinguishable from the existing real ones. This property can be evaluated with the co-immersion criterion, which encompasses scenes constituted by arbitrary configurations of real and virtual objects. Thus, we introduce the term Audio Augmented Virtuality (AAV) to describe a fully virtual environment consisting of auditory content captured from the real world, augmented by synthetic sound generation. We propose an experimental design in AAV investigating how simplified late reverberation (LR) affects the co-immersion of a sound source. Participants listened to simultaneous virtual speakers dynamically rendered through spatial Room Impulse Responses, and were asked to detect the presence of an impostor, i.e., a speaker rendered with one of two simplified LR conditions. Detection rates were found to be close to chance level, especially for one condition, suggesting a limited influence on co-immersion of the simplified LR in the evaluated AAV scenes. This methodology can be straightforwardly extended and applied to different acoustics scenes, complexities, i.e., the number of simultaneous speakers, and rendering parameters in order to further investigate the requirements for immersive audio technologies in AAR and AAV applications.

Room-aware portable Auditory Augmented Reality: Real-time spatial audio generation with geometric data analysis

This study proposes an innovative and practical approach to spatial acoustic rendering for Auditory Augmented Reality (AAR) in small room usage. AAR aims to enhance user experiences by overlaying spatially appropriate sounds in the real environment, creating the illusion that virtual sounds coexist in the physical world. However, current technological constraints in hardware and AAR rendering present challenges in creating a truly immersive sensation of “being there.” To address these challenges, we present a dynamic processing framework for a typical mobile AAR setting. This framework combines a game-audio engine with 3-D visual computation both specialized for real-time processing, providing a plausible spatial impression with lightweight computation. The process involves capturing room information by the Light Detection And Ranging (LiDAR) sensor mounted on the device, followed by estimating its dominant acoustic materials. The generated data will be used to compute dynamic early reflections and late reverberation. Early reflections are calculated up to the fourth reflection using the image source method, considering the acoustic materials. Late reverberation is generated using the Feedback Delay Network (FDN), with room frequency characteristics and spatial directivity varying relative to the listener's position. All components will be represented in fifth-order Ambisonics and binauralized with head-tracking. Notably, this dynamic rendering method could be light enough to handle all processing on a single mobile device.

The localization of virtual source reproduced by the crosstalk cancellation system under different direct-to-reverberation energy ratios conditions

This study explores the localization of virtual sources reproduced by the crosstalk cancellation (CTC) system under reverberation environments with different direct-to-reverberation energy ratios (DRRs). Binaural room impulse responses in rectangle rooms are simulated by high-order image source method. Two independent key variables, the reverberation time (RT) and the distance of the sound source to the listener, are employed to manipulate DRR through simulation. Subsequently, a subjective localization experiment based on the CTC system is virtually conducted using earphone reproduction. The results show that the azimuth localization in horizontal plane is almost impervious by RT but is subjected to the change of the source distance. Specifically, as the distance of sound source increases, the perceived azimuth of lateral virtual sources exhibits a bias towards the front, accompanied by larger variances. The results indicate that reflections with small delays have a more detrimental effect on sound localization compared to the late reverberation with large intensity, which may be associated with the inhibition of late reflections based on the precedence effect. Additionally, the localization data are analyzed and explained using binaural cues, including the interaural cross-correlation coefficient, the interaural time difference (ITD), and interaural level difference (ILD).

Crossband Filtering for Weighted Prediction Error-Based Speech Dereverberation

Weighted prediction error (WPE) is a linear prediction-based method extensively used to predict and attenuate the late reverberation component of an observed speech signal. This paper introduces an extended version of the WPE method to enhance the modeling accuracy in the time–frequency domain by incorporating crossband filters. Two approaches to extending the WPE while considering crossband filters are proposed and investigated. The first approach improves the model’s accuracy. However, it increases the computational complexity, while the second approach maintains the same computational complexity as the conventional WPE while still achieving improved accuracy and comparable performance to the first approach. To validate the effectiveness of the proposed methods, extensive simulations are conducted. The experimental results demonstrate that both methods outperform the conventional WPE regarding dereverberation performance. These findings highlight the potential of incorporating crossband filters in improving the accuracy and efficacy of the WPE method for dereverberation tasks.

Objective Assessment of Binaural Benefit from Acoustical Treatment in Real Primary School Classrooms.

Providing students with an adequate acoustic environment is crucial for ensuring speech intelligibility in primary school classrooms. Two main approaches to control acoustics in educational facilities consist of reducing background noise and late reverberation. Prediction models for speech intelligibility have been developed and implemented to evaluate the effects of these approaches. In this study, two versions of the Binaural Speech Intelligibility Model (BSIM) were used to predict speech intelligibility in realistic spatial configurations of speakers and listeners, considering binaural aspects. Both versions shared the same binaural processing and speech intelligibility backend processes but differed in the pre-processing of the speech signal. An Italian primary school classroom was characterized in terms of acoustics before (reverberation, T20 = 1.6 ± 0.1 s) and after (T20 = 0.6 ± 0.1 s) an acoustical treatment to compare BSIM predictions to well-established room acoustic measures. With shorter reverberation time, speech clarity and definition improved, as well as speech recognition thresholds (SRTs) (by up to ~6 dB), particularly when the noise source was close to the receiver and an energetic masker was present. Conversely, longer reverberation times resulted (i) in poorer SRTs (by ~11 dB on average) and (ii) in an almost non-existent spatial release from masking at an angle (SRM).

A neural network-supported two-stage algorithm for lightweight dereverberation on hearing devices

A two-stage lightweight online dereverberation algorithm for hearing devices is presented in this paper. The approach combines a multi-channel multi-frame linear filter with a single-channel single-frame post-filter. Both components rely on power spectral density (PSD) estimates provided by deep neural networks (DNNs). By deriving new metrics analyzing the dereverberation performance in various time ranges, we confirm that directly optimizing for a criterion at the output of the multi-channel linear filtering stage results in a more efficient dereverberation as compared to placing the criterion at the output of the DNN to optimize the PSD estimation. More concretely, we show that training this stage end-to-end helps further remove the reverberation in the range accessible to the filter, thus increasing the early-to-moderate reverberation ratio. We argue and demonstrate that it can then be well combined with a post-filtering stage to efficiently suppress the residual late reverberation, thereby increasing the early-to-final reverberation ratio. This proposed two-stage procedure is shown to be both very effective in terms of dereverberation performance and computational demands, as compared to, e.g., recent state-of-the-art DNN approaches. Furthermore, the proposed two-stage system can be adapted to the needs of different types of hearing-device users by controlling the amount of reduction of early reflections.

Computationally-Efficient Simulation of Late Reverberation for Inhomogeneous Boundary Conditions and Coupled Rooms

Computationally-Efficient Simulation of Late Reverberation for Inhomogeneous Boundary Conditions and Coupled Rooms

Real-time co-location in dynamic virtual soundscapes across multi-occupant immersive rooms

Immersive rooms, a type of virtual reality system consisting of human-scale panoramic visual and acoustic display systems and distributed sensing apparatus for occupant motion, have been increasingly adopted for dynamic and interactive applications. While these applications enable multi-user audiovisual immersion and navigation from a single physical location, they have yet to propagate along multiple homogeneous system infrastructures in a networked manner. In this work, we intend to co-locate two physically-remote immersive rooms – at EMPAC and the CRAIVE-Lab, respectively – in a single system of shared environments developed in Unity and embedded with virtual soundscapes. This system actively monitors spatial properties of both immersive rooms’ dynamic virtual footprint and their corresponding occupants. It generates virtual sound sources. both procedurally and through spatially-aware user inputs. The sound sources are rendered in real time via an algorithm synthesizing a ray-traced early reflection window and a parameterized late reverberation estimate from in-scene geometries. The co-located virtual soundscapes, displayed in individual immersive rooms through their respective multi-channel wave field synthesis loudspeaker systems, are shared as such that the user interaction in one physical location has holistic effects on the experience of virtual environments across all associated physical locations. [Work supported by NSF IIS-1909229 & CNS-1229391.]

Multi-Threaded Sound Propagation Algorithm to Improve Performance on Mobile Devices

We propose a multi-threaded algorithm that can improve the performance of geometric acoustic (GA)-based sound propagation algorithms in mobile devices. In general, sound propagation algorithms require high computational cost because they perform based on ray tracing algorithms. For this reason, it is difficult to operate sound propagation algorithms in mobile environments. To solve this problem, we processed the early reflection and late reverberation steps in parallel and verified the performance in four scenes based on eight sound sources. The experimental results showed that the performance of the proposed method was on average 1.77 times better than that of the single-threaded method, demonstrating that our algorithm can improve the performance of mobile devices.

Common-Slope Modeling of Late Reverberation

Common-Slope Modeling of Late Reverberation

Fast processing models effects of reflections on binaural unmasking

Sound reflections and late reverberation alter energetic and binaural cues of a target source, thereby affecting its detection in noise. Two experiments investigated detection of harmonic complex tones, centered around 500 Hz, in noise, in a virtual room with different modifications of simulated room impulse responses (RIRs). Stimuli were auralized using the Simulated Open Field Environment’s (SOFE’s) loudspeakers in anechoic space. The target was presented from the front (0°) or 60° azimuth, while an anechoic noise masker was simultaneously presented at 0°. In the first experiment, early reflections were progressively added to the RIR and detection thresholds of the reverberant target were measured. For a frontal sound source, detection thresholds decreased while adding early reflections within the first 45 ms, whereas for a lateral sound source, thresholds remained constant. In the second experiment, early reflections were removed while late reflections were kept along with the direct sound. Results for a target at 0° show that even reflections as late as 150 ms reduce detection thresholds compared to only the direct sound. A binaural model with a sluggishness component following the computation of binaural unmasking in short windows predicts measured and literature results better than when large windows are used.

Direct and Residual Subspace Decomposition of Spatial Room Impulse Responses

Psychoacoustic experiments have shown that directional properties of the direct sound, salient reflections, and the late reverberation of an acoustic room response can have a distinct influence on the auditory perception of a given room. Spatial room impulse responses (SRIRs) capture those properties and thus are used for direction-dependent room acoustic analysis and virtual acoustic rendering. This work proposes a subspace method that decomposes SRIRs into a direct part, which comprises the direct sound and the salient reflections, and a residual, to facilitate enhanced analysis and rendering methods by providing individual access to these components. The proposed method is based on the generalized singular value decomposition and interprets the residual as noise that is to be separated from the other components of the reverberation. Large generalized singular values are attributed to the direct part, which is then obtained as a low-rank approximation of the SRIR. By advancing from the end of the SRIR toward the beginning while iteratively updating the residual estimate, the method adapts to spatio-temporal variations of the residual. The method is evaluated using a spatio-spectral error measure and simulated SRIRs of different rooms, microphone arrays, and ratios of direct sound to residual energy. The proposed method creates lower errors than existing approaches in all tested scenarios, including a scenario with two simultaneous reflections. A case study with measured SRIRs shows the applicability of the method under real-world acoustic conditions. A reference implementation is provided.

Walkable auralizations for experiential learning in an immersive classroom.

This paper proposes an experiential method for learning acoustics and consequences of room design through the rapid creation of audio-visual congruent walkable auralizations. An efficient method produces auralizations of acoustical landmarks using a two-dimensional ray-tracing algorithm and publicly available floor plans for a 128-channel wave-field synthesis system. Late reverberation parameters are calculated using additional volumetric data. Congruent visuals are produced using a web-based interface accessible via personal devices, which automatically formats for and transmits to the immersive display. Massive user-contributed online databases are harnessed through application programming interfaces, such as those offered by the Google Maps Platform, to provide near-instant access to innumerable locations. The approach allows the rapid sonic recreation of historical concert venues with adequate sound sources. Listeners can walk through these recreations over an extended user area (12 m × 10 m).

Time-frequency-dependent directional analysis of room reflections using eigenbeam processing and von Mises-Fisher clustering.

The knowledge of frequency-dependent spatiotemporal features of the reflected soundfield is essential in optimizing the perception quality of spatial audio applications. For this purpose, we need a reliable room acoustic analyzer that can conceive the spatial variations in a decaying reflected soundfield according to the frequency-dependent surface properties and source directivity. This paper introduces a time-frequency-dependent angular reflection power distribution model represented by a von Mises-Fisher (vMF) mixture function to facilitate manifold analysis of a reverberant soundfield. The proposed approach utilizes the spatial correlation of higher-order eigenbeams to deduce the directional reflection power vectors, which are then synthesized into a vMF mixture model. The experimental study demonstrates the directional power variations of early reflections and late reverberations across different frequencies. This work also introduces a measure called the directivity time-span to quantify the duration of anisotropic reflections before it decays into a totally diffused field. We validate the subband performance by comparing it with the eigenbeam multiple signal classification method. The results prove the influence of source position, source directivity, and room environment in the distribution of reflection power, whereas the directivity time-span behaves independent of the source positions.

Predicting reflection patterns from binaural activity maps using deep neural networks

A new model architecture is presented to predict room acoustical parameters from a running binaural signal. For this purpose, a deep neural network architecture is combined with a precedence effect model to extract the spatial and temporal locations of the direct signal and early reflections. The precedence effect model builds on the modified BICAM algorithm [Braasch, J. Acoust. Soc. Am. 140, EL143], for which the 1st layer auto-/cross correlation functions are replaced with a Cepstrum method. The latter allows a better separation of features relating to the source signal's early reflections and harmonic structure. The precedence effect model is used to create binaural activity maps that are analyzed by the neural network for pattern recognition. Anechoic orchestral recordings were reverberated by adding four early reflections and late reverberation to test the model. Head-related transfer functions were used to spatialize direct sound and early reflections. The model can identify the main reflection characteristics of a room, offering applications in numerous fields, including room acoustical assessment, acoustical analysis for virtual-reality applications, and modeling of human perception. [Work supported by the National Science Foundation under Grant No. IIS-1909229.]