Real Reverberant Environments Research Articles

Auditory-visual interactions in egocentric distance perception: Ventriloquism effect and aftereffect.

This study describes data on auditory-visual integration and visually-guided adaptation of auditory distance perception using the ventriloquism effect (VE) and ventriloquism aftereffect (VAE). In an experiment, participants judged egocentric distance of interleaved auditory or auditory-visual stimuli with the auditory component located from 0.7 to 2.04 m in front of listeners in a real reverberant environment. The visual component of auditory-visual stimuli was displaced 30% closer (V-closer), 30% farther (V-farther), or aligned (V-aligned) with respect to the auditory component. The VE and VAE were measured in auditory and auditory-visual trials, respectively. Both effects were approximately independent of target distance when expressed in logarithmic units. The VE strength, defined as a difference of V-misaligned and V-aligned response bias, was approximately 72% of the auditory-visual disparity regardless of the visual-displacement direction, while the VAE was stronger in the V-farther (44%) than the V-closer (31%) condition. The VAE persisted to post-adaptation auditory-only blocks of trials, although it was diminished. The rates of build-up/break-down of the VAE were asymmetrical, with slower adaptation in the V-closer condition. These results suggest that auditory-visual distance integration is independent of the direction of induced shift, while the re-calibration is stronger and faster when evoked by more distant visual stimuli.

A Novel Scheme for Single-Channel Speech Dereverberation

This paper presents a novel scheme for speech dereverberation. The core of our method is a two-stage single-channel speech enhancement scheme. Degraded speech obtains a sparser representation of the linear prediction residual in the first stage of our proposed scheme by applying orthogonal matching pursuit on overcomplete bases, trained by the K-SVD algorithm. Our method includes an estimation of reverberation and mixing time from a recorded hand clap or a simulated room impulse response, which are used to create a time-domain envelope. Late reverberation is suppressed at the second stage by estimating its energy from the previous envelope and removed with spectral subtraction. Further speech enhancement is applied on minimizing the background noise, based on optimal smoothing and minimum statistics. Experimental results indicate favorable quality, compared to two state-of-the-art methods, especially in real reverberant environments with increased reverberation and background noise.

CRNN-Based Multiple DoA Estimation Using Acoustic Intensity Features for Ambisonics Recordings

Localizing audio sources is challenging in real reverberant environments, especially when several sources are active. We propose to use a neural network built from stacked convolutional and recurrent layers in order to estimate the directions of arrival of multiple sources from a first-order Ambisonics recording. It returns the directions of arrival over a discrete grid of a known number of sources. We propose to use features derived from the acoustic intensity vector as inputs. We analyze the behavior of the neural network by means of a visualization technique called layerwise relevance propagation. This analysis highlights which parts of the input signal are relevant in a given situation. We also conduct experiments to evaluate the performance of our system in various environments, from simulated rooms to real recordings, with one or two speech sources. The results show that the proposed features significantly improve performances with respect to raw Ambisonics inputs.

Localization and Characterization of Multiple Harmonic Sources

We introduce a new and intuitive algorithm to characterize and localize multiple harmonic sources intersecting in the spatial and frequency domains. It jointly estimates their fundamental frequencies, their respective amplitudes, and their directions of arrival based on an intelligent non-parametric signal representation. To obtain these parameters, we first apply variable-scale sampling on unbiased cross-correlation functions between pairs of microphone signals to generate a joint parameter space. Then, we employ a multidimensional maxima detector to represent the parameters in a sparse joint parameter space. In comparison to others, our algorithm solves the issue of pitch-period doubling when using cross-correlation functions, it estimates multiple harmonic sources with a signal power smaller than the signal power of the dominant harmonic source, and it associates the estimated parameters to their corresponding sources in a multidimensional sparse joint parameter space, which can be directly fed into a tracker. We tested our algorithm and three others on synthetic data and speech data recorded in a real reverberant environment and evaluated their performance by employing the joint recall measure, the root-mean-square error, and the cumulative distribution function of fundamental frequencies and directions of arrival. The evaluations show promising results: Our algorithm outperforms the others in terms of the joint recall measure, and it can achieve root-mean-square errors of 1 Hz or 1 $^\circ$ and smaller, which facilitates, e.g., distant-speech enhancement or source separation.

Environment-dependent denoising autoencoder for distant-talking speech recognition

In this paper, we propose an environment-dependent denoising autoencoder (DAE) and automatic environment identification based on a deep neural network (DNN) with blind reverberation estimation for robust distant-talking speech recognition. Recently, DAEs have been shown to be effective in many noise reduction and reverberation suppression applications because higher-level representations and increased flexibility of the feature mapping function can be learned. However, a DAE is not adequate in mismatched training and test environments. In a conventional DAE, parameters are trained using pairs of reverberant speech and clean speech under various acoustic conditions (that is, an environment-independent DAE). To address the above problem, we propose two environment-dependent DAEs to reduce the influence of mismatches between training and test environments. In the first approach, we train various DAEs using speech from different acoustic environments, and the DAE for the condition that best matches the test condition is automatically selected (that is, a two-step environment-dependent DAE). To improve environment identification performance, we propose a DNN that uses both reverberant speech and estimated reverberation. In the second approach, we add estimated reverberation features to the input of the DAE (that is, a one-step environment-dependent DAE or a reverberation-aware DAE). The proposed method is evaluated using speech in simulated and real reverberant environments. Experimental results show that the environment-dependent DAE outperforms the environment-independent one in both simulated and real reverberant environments. For two-step environment-dependent DAE, the performance of environment identification based on the proposed DNN approach is also better than that of the conventional DNN approach, in which only reverberant speech is used and reverberation is not blindly estimated. And, the one-step environment-dependent DAE significantly outperforms the two-step environment-dependent DAE.

Source Traffic Modelling in WSN for Acoustic Sensing in Reverberant Environment

Recent research efforts show that Wireless Sensor Networks (WSN) for acoustic sensing have the potential for ubiquitous sensing. The key challenge in delivering such WSN architecture is the trade-off between the cost of wireless sensors and acoustic analysis complexity. In this paper, we are proposing a source traffic modelling considering a system in which sensors are connected to a cloud based processing service. Acoustic sensing in practice generates great amount of traffic, and therefore a new model of sensing in a reverberant environment is necessary for better understanding of traffic nature and its volume. To lower the sensor cost, we have considered low-complexity acoustic event detection on the sensor side for preventing unnecessary traffic. The proposed acoustic sensor acts like a noise gate. Sound propagation in reverberant environment is modelled using the modified Image-Source Modelling (ISM) method. The source traffic model is validated in Matlab simulation and in real reverberant environment. Results show an example of bandwidth used by a simple acoustic sensor. DOI: http://dx.doi.org/10.5755/j01.eee.21.5.13335

Single-channel Dereverberation for Distant-Talking Speech Recognition by Combining Denoising Autoencoder and Temporal Structure Normalization

In this paper, we propose a robust distant-talking speech recognition by combining cepstral domain denoising autoencoder (DAE) and temporal structure normalization (TSN) filter. For the proposed method, after applying a DAE in the cepstral domain of speech to suppress reverberation, we apply a post-processing technology based on temporal structure normalization (TSN) filter to reduce the noise and reverberation effects by normalizing the modulation spectra to reference spectra of clean speech. The proposed method was evaluated using speech in simulated and real reverberant environments. By combining a cepstral-domain DAE and TSN, the average Word Error Rate (WER) was reduced from 25.2% of the baseline system to 21.2% in simulated environments and from 47.5% to 41.3% in real environments, respectively.

A Wrapped Kalman Filter for Azimuthal Speaker Tracking

We present the wrapped Kalman filter (WKF) for tracking the azimuth of a speaker with a compact, 3-channel microphone array. Traditional extended and unscented filters assume that the observation is a rotating vector in \BBR2. However, the azimuth inhabits a 1-D subspace: the unit circle. We model the state variable with a wrapped Gaussian distribution and show that this achieves a lower mean squared error than 2-D methods. We demonstrate the superior tracking performance of the WKF in simulated and real reverberant environments.

An MTF-based blind restoration of temporal power envelopes as a front-end processor for automatic speech recognition systems in reverberant environments

To reduce speech degradation in reverberant environments, we previously proposed a modulation transfer function (MTF) based method for speech dereverberation. It is based on the MTF relation that the sub-band temporal power envelope of reverberant speech can be represented as the convolution between temporal power envelopes of clean speech and the room impulse response. Therefore, the sub-band power envelope of clean speech can be estimated using inverse MTF filtering without measuring the room impulse response. We tested the effectiveness of this method as a front-end for automatic speech recognition (ASR) in both artificial and real reverberant environments. Reverberant speech signals were created by simple convolution of clean speech (AURORA-2J) and artificially-produced or real room impulse responses. The relative spectral filtering of the auditory-power-spectrum based method was used as a baseline. Compared with the baseline, our proposed method had 36.64% and 21.68% improvements in error reduction rate for artificial reverberant environments (reverberation times from 0.2 to 2.0 s) and real reverberant environments (43 reverberant impulse responses), respectively. These results indicate that our proposed method can be used as a robust front-end for ASR. [Work supported by a Grant-in-Aid for Science Research from the Japanese Ministry of Education (No. 18680017).]

Comparative evaluation of modulation-transfer-function-based blind restoration of sub-band power envelopes of speech as a front-end processor for automatic speech recognition systems

To reduce speech degradation in reverberant environments, we previously proposed a modulation-transfer-function (MTF)-based method of speech dereverberation. By considering the temporal modulation properties of speech, and the exponential decay properties of the power envelope of the impulse response of room acoustics, we obtained the following MTF relation: the sub-band power envelope of reverberant speech that can be represented as a convolution between the sub-band power envelope of clean speech and the power envelope of the impulse response of room acoustics. On the basis of the MTF relation, inverse MTF filtering can be applied to restoring the power envelopes of reverberant speech. Therefore, the impulse response of the room acoustics in this restoration dose not need to be measured at any time since we model the power envelope of the impulse response as an exponential decay function. We have tested how effective this method is as a front-end for automatic speech recognition (ASR) systems in artificial and real reverberant environments. Reverberant speech signals were created by simply convoluting clean speech (AURORA-2J database) with the artificially produced or real impulse responses of room acoustics. A method based on the auditory power spectrum was used as a baseline for comparison. Compared with the baseline, the proposed method for artificial reverberant environments produced a 35.67% relative improvement in the error reduction rate (on average, for reverberation times from 0.2 to 2.0 s), and for real reverberant environments (43 reverberant impulse responses), it produced a 25.78% relative improvement in the error reduction rate. The results demonstrate that our new approach can improve the robustness of speech-recognition systems in reverberant environments, and it performs better than conventional methods.

Blind Model Selection for Automatic Speech Recognition in Reverberant Environments

This communication presents a new method for automatic speech recognition in reverberant environments. Our approach consists in the selection of the best acoustic model out of a library of models trained on artificially reverberated speech databases corresponding to various reverberant conditions. Given a speech utterance recorded within a reverberant room, a Maximum Likelihood estimate of the fullband room reverberation time is computed using a statistical model for short-term log-energy sequences of anechoic speech. The estimated reverberation time is then used to select the best acoustic model, i.e., the model trained on a speech database most closely matching the estimated reverberation time, which serves to recognize the reverberated speech utterance. The proposed model selection approach is shown to improve significantly recognition accuracy for a connected digit task in both simulated and real reverberant environments, outperforming standard channel normalization techniques.

Speech enhancement using sub-band adaptive Griffiths–Jim signal processing

Results are presented from intelligibility tests of a two-microphone sub-band adaptive Griffiths–Jim (SBAGJ) processing scheme that has possible application to future hearing aids as a method of improving speech intelligibility and quality in a noisy reverberant environment. This SBAGJ scheme combines sub-band processing with a Griffiths–Jim “front-end” that delivers a simple system more robust to the “causality” issue inherent with some multi-microphone adaptive noise cancelling configurations. Intelligibility testing is described and results presented of an assessment of the SBAGJ scheme using ten normal hearing listeners and signals from a real reverberant environment. The results support the hypothesis that speech processing using the SBAGJ scheme can provide a statistically and practically significant improvement in speech intelligibility. Analysis of mean opinion score data shows a corresponding statistically significant improvement in subjective quality.