Modulation Energy Ratio Research Articles

Preserving the beamforming effect for spatial cue-based pseudo-binaural dereverberation of a single source

Reverberations are unavoidable in enclosures, resulting in reduced intelligibility for hearing impaired and non-native listeners and even for the normal hearing listeners in noisy circumstances. It also degrades the performance of machine listening applications. In this paper, we propose a novel approach of binaural dereverberation of a single speech source, using the differences in the interaural cues of the direct path signal and the reverberations. Two beamformers, spaced at an interaural distance, are used to extract the reverberations from the reverberant speech. The interaural cues generated by these reverberations and those generated by the direct path signal act as a two-class dataset, used for the training of U-Net (a deep convolutional neural network). After its training, the beamformers are removed and the trained U-Net along with the maximum likelihood estimation (MLE) algorithm is used to discriminate between the direct path cues from the reverberation cues, when the system is exposed to the interaural spectrogram of the reverberant speech signal. Our proposed model has outperformed the classical signal processing dereverberation model ‘weighted prediction error’ in terms of cepstral distance (CEP), frequency weighted segmental signal to noise ratio (fwsegSNR) and signal-to-reverberation modulation energy ratio (SRMR) by 1.4 points, 8 dB and 0.6 dB. It has achieved better performance than the deep learning based dereverberation model by gaining 1.3 points improvement in CEP with comparable fwsegSNR, using training dataset which is almost 8 times smaller than required for that model. The proposed model also sustained its performance under relatively similar unseen acoustic conditions and at positions in the vicinity of its training position.

An improved two-stage dereverberation method based on bayesian estimation of a speech source

It is not easy to extract speech emanating from a specific direction in a large reverberant room with additional sound sources. Many of the existing methods work well only if there is a single sound source, and their performance degrades if additional sound sources are present. In this work, we have addressed this problem by maximizing the posterior probability of speech signal as computed by use of Bayes' Theorem. Our de-reverb method is two-staged. We have evaluated the efficacy of our method against several popular methods in terms of five objective measures; Signal-to-Interference Ratio, Log-Likelihood Ratio, Cepstrum Distance, Frequency-Weighted Segment SNR, and Speechto-Reverberation Modulation Energy Ratio. We show that on each of these parameters, the method proposed in this work performs better than the one we have benchmarked it against.

Individual differences across caregivers in acoustic implementation of infant-directed and adult-directed speech: Modeling impacts on intelligibility in children with cochlear implants

Caregivers modify their speaking style from adult-directed speech (ADS) to infant-directed speech (IDS) when talking to infants. However, it is unclear how individual caregivers acoustically implement differences between ADS and IDS, and how these differences may affect experienced speech intelligibility by infants, particularly those with cochlear implants (CIs). Seven female talkers spoke fifteen utterances both in IDS and ADS. We analyzed these utterances and their cochlear implant-simulated versions (using 22-channel noise vocoders) to investigate how acoustic distances between ADS and IDS varied across talkers (based on Mel-frequency cepstral coefficients, MFCCs) and how the effect of these shifts in speaking style on utterance intelligibility were different between talkers (using the acoustic index of speech-to-reverberation modulation energy ratio tailored to CI devices, SRMR-CI). Results showed substantial variability across talkers comparing ADS and IDS for caregivers’ acoustic profiles from MFCCs and for speech intelligibility from SRMR-CI. These findings suggest that acoustic choices by individual mothers may differentially affect recoverability of intelligible words from speech signals by children with CIs, which may contribute to differences in these children’s language outcomes. [Work supported by NIH grant R01DC008581.]

Phase and reverberation aware DNN for distant-talking speech enhancement

Enhancing reverberant speech with Deep Neural Networks (DNNs) is an interesting yet challenging topic. The performance of speech enhancement degrades significantly when test and training conditions are mismatched. In this paper we propose a Static Reverberation Aware Training (SRAT)-based dereverberation through which the reverberation estimate is obtained by averaging over broken down frame. This method significantly reduces the input dimensions of the and enables the DNN to learn the relations between clean and reverberant speech more efficiently. Most speech enhancement approaches ignore phase information due to its complicated structure. As phase correlates closely to speech signal we exploited this relationship to achieve better performance using DNN. Phase information was augmented with magnitude information and used as the input for DNN. We denote this method as phase aware DNN. Finally, both phase information and reverberation were added to reverberant speech to achieve better speech enhancement performance in a distant-talking condition. Features of the reverberant speech, phase and reverberation were used during the training and testing stages. This is because the DNN could use both reverberation and phase information to better generalize the speech signal. The proposed method was evaluated using the REVERB CHALLENGE 2014 database. Results are significantly improved results with respect to both reconstructed speech quality (PESQ: Perceptual Evaluation of Speech Quality) and influence of reverberation (SRMR: Speech to Reverberation Modulation Energy Ratio). As compared to the conventional DNN-based approach, this proposed one improved SRMR from 4.84 to 5.92 and PESQ from 2.34 to 2.70, indicating that our proposed method could efficiently enhance speech severely corrupted by reverberation.

Guided spectrogram filtering for speech dereverberation

Guided filtering is a computationally efficient and powerful technique used in image processing applications, such as edge-preserving smoothing, details enhancing and single image dehazing. In this paper, we propose a novel single channel speech dereverberation method using guided spectrogram filtering by considering a speech spectrogram as an image. The proposed method requires neither room acoustic parameter estimation nor late reverberant spectral variance estimation. Objective test results show the validity of the guided spectrogram filtering method for speech dereverberation. Compared with state-of-the-art speech dereverberation methods, the proposed method has better performance in terms of perceptual evaluation of speech quality (PESQ), speech-to-reverberation modulation energy ratio (SRMR) and short-time objective intelligibility (STOI) in most cases.

Speech dereverberation for enhancement and recognition using dynamic features constrained deep neural networks and feature adaptation

This paper investigates deep neural networks (DNN) based on nonlinear feature mapping and statistical linear feature adaptation approaches for reducing reverberation in speech signals. In the nonlinear feature mapping approach, DNN is trained from parallel clean/distorted speech corpus to map reverberant and noisy speech coefficients (such as log magnitude spectrum) to the underlying clean speech coefficients. The constraint imposed by dynamic features (i.e., the time derivatives of the speech coefficients) are used to enhance the smoothness of predicted coefficient trajectories in two ways. One is to obtain the enhanced speech coefficients with a least square estimation from the coefficients and dynamic features predicted by DNN. The other is to incorporate the constraint of dynamic features directly into the DNN training process using a sequential cost function. In the linear feature adaptation approach, a sparse linear transform, called cross transform, is used to transform multiple frames of speech coefficients to a new feature space. The transform is estimated to maximize the likelihood of the transformed coefficients given a model of clean speech coefficients. Unlike the DNN approach, no parallel corpus is used and no assumption on distortion types is made. The two approaches are evaluated on the REVERB Challenge 2014 tasks. Both speech enhancement and automatic speech recognition (ASR) results show that the DNN-based mappings significantly reduce the reverberation in speech and improve both speech quality and ASR performance. For the speech enhancement task, the proposed dynamic feature constraint help to improve cepstral distance, frequency-weighted segmental signal-to-noise ratio (SNR), and log likelihood ratio metrics while moderately degrades the speech-to-reverberation modulation energy ratio. In addition, the cross transform feature adaptation improves the ASR performance significantly for clean-condition trained acoustic models.

Objective Quality and Intelligibility Prediction for Users of Assistive Listening Devices.

This article presents an overview of twelve existing objective speech quality and intelligibility prediction tools. Two classes of algorithms are presented, namely intrusive and non-intrusive, with the former requiring the use of a reference signal, while the latter does not. Investigated metrics include both those developed for normal hearing listeners, as well as those tailored particularly for hearing impaired (HI) listeners who are users of assistive listening devices (i.e., hearing aids, HAs, and cochlear implants, CIs). Representative examples of those optimized for HI listeners include the speech-to-reverberation modulation energy ratio, tailored to hearing aids (SRMR-HA) and to cochlear implants (SRMR-CI); the modulation spectrum area (ModA); the hearing aid speech quality (HASQI) and perception indices (HASPI); and the PErception MOdel - hearing impairment quality (PEMO-Q-HI). The objective metrics are tested on three subjectively-rated speech datasets covering reverberation-alone, noise-alone, and reverberation-plus-noise degradation conditions, as well as degradations resultant from nonlinear frequency compression and different speech enhancement strategies. The advantages and limitations of each measure are highlighted and recommendations are given for suggested uses of the different tools under specific environmental and processing conditions.

A Non-Intrusive Quality and Intelligibility Measure of Reverberant and Dereverberated Speech

A modulation spectral representation is investigated for non-intrusive quality and intelligibility measurement of reverberant and dereverberated speech. The representation is obtained by means of an auditory-inspired filterbank analysis of critical-band temporal envelopes of the speech signal. Modulation spectral insights are used to develop an adaptive measure termed speech to reverberation modulation energy ratio. Experimental results show the proposed measure outperforming three standard algorithms for tasks involving estimation of multiple dimensions of perceived coloration, as well as quality measurement and intelligibility estimation of reverberant and dereverberated speech.

Temporal Dynamics for Blind Measurement of Room Acoustical Parameters

In this paper, short- and long-term temporal dynamic information is investigated for the blind measurement of room acoustical parameters. In particular, estimators of room reverberation time (T60) and direct-to-reverberant energy ratio (DRR) are proposed. Short-term temporal dynamic information is obtained from differential (delta) cepstral coefficients. The statistics computed from the zeroth-order delta cepstral sequence serve as input features to a support vector T60 estimator. Long-term temporal dynamic cues, on the other hand, are obtained from an auditory spectrotemporal representation of speech commonly referred to as modulation spectrum. A measure termed as reverberation-to-speech modulation energy ratio, which is computed per modulation frequency band, is proposed and serves as input to T60 and DRR estimators. Experiments show that the proposed estimators outperform a baseline system in scenarios involving reverberant speech with and without the presence of acoustic background noise. Experiments also suggest that estimators of subjective perception of spectral coloration, reverberant tail effect, and overall speech quality can be obtained with an adaptive speech-to-reverberation modulation energy ratio measure.