Objective Speech Quality Measures Research Articles

Single-channel speech enhancement using implicit Wiener filter for high-quality speech communication

Speech enables easy human-to-human communication as well as human-to-machine interaction. However, the quality of speech degrades due to background noise in the environment, such as drone noise embedded in speech during search and rescue operations. Similarly, helicopter noise, airplane noise, and station noise reduce the quality of speech. Speech enhancement algorithms reduce background noise, resulting in a crystal clear and noise-free conversation. For many applications, it is also necessary to process these noisy speech signals at the edge node level. Thus, we propose implicit Wiener filter-based algorithm for speech enhancement using edge computing system. In the proposed algorithm, a first order recursive equation is used to estimate the noise. The performance of the proposed algorithm is evaluated for two speech utterances, one uttered by a male speaker and the other by a female speaker. Both utterances are degraded by different types of non-stationary noises such as exhibition, station, drone, helicopter, airplane, and white Gaussian stationary noise with different signal-to-noise ratios. Further, we compare the performance of the proposed speech enhancement algorithm with the conventional spectral subtraction algorithm. Performance evaluations using objective speech quality measures demonstrate that the proposed speech enhancement algorithm outperforms the spectral subtraction algorithm in estimating the clean speech from the noisy speech. Finally, we implement the proposed speech enhancement algorithm, in addition to the spectral subtraction algorithm, on the Raspberry Pi 4 Model B, which is a low power edge computing device.

A Two-Stage Deep Neuroevolutionary Technique for Self-Adaptive Speech Enhancement

This paper presents a novel self-adaptive approach for speech enhancement in the context of highly nonstationary noise. A two-stage deep neuroevolutionary technique for speech enhancement is proposed. The first stage is composed of a deep neural network (DNN) method for speech enhancement. Two DNN methods were tested at this stage, namely, both a deep complex convolution recurrent network (DCCRN) and a residual long short-term memory neural network (ResLSTM). The ResLSTM method was combined with a minimum mean-square error method to perform a preliminary enhancement. The ResLSTM network is used as an <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">a priori</i> signal-to-noise ratio (SNR) estimator. The second stage implements a self-adaptive multiband spectral subtraction enhancement method using tuning optimization based on a genetic algorithm. The proposed two-stage technique is evaluated using objective measures of speech quality and intelligibility. The experiments are carried out using the NOIZEUS noisy speech corpus using conditions of real-world stationary, colored, and nonstationary noise sources at multiple SNR levels. These experiments demonstrate the advantage of building a cooperative approach using evolutionary and deep learning-based techniques that are capable of achieving robust speech enhancement in adverse conditions. Indeed, the experimental tests show that the proposed two-stage technique outperformed a baseline implementation using a state-of-the-art deep learning approach by an average 13% and 6% improvement for six noise conditions at a −5 dB and a 0 dB input SNR, respectively.

Deep Q-network-based noise suppression for robust speech recognition

This study develops the deep Q-network (DQN)-based noise suppression for robust speech recognition purposes under ambient noise. We thus design a reinforcement algorithm that combines DQN training with a deep neural networks (DNN) to let reinforcement learning (RL) work for complex and high dimensional environments like speech recognition. For this, we elaborate on the DQN training to choose the best action that is the quantized noise suppression gain by the observation of noisy speech signal with the rewards of DQN including both the word error rate (WER) and objective speech quality measure. Experiments demonstrate that the proposed algorithm improves speech recognition in various noisy conditions while reducing the computational burden compared to the DNN-based noise suppression method.

The Speech Enhancement via Attention Masking Network (SEAMNET): An End-to-end System for Joint Suppression of Noise and Reverberation

This paper proposes the Speech Enhancement via Attention Masking Network (SEAMNET), a neural network-based end-to-end single-channel speech enhancement system designed for joint suppression of noise and reverberation. It formalizes an end-to-end network architecture, referred to as b-Net , which accomplishes noise suppression through attention masking in a learned embedding space. A key contribution of SEAMNET is that the b-Net architecture contains both an enhancement and an autoencoder path. This paper proposes a novel loss function which simultaneously trains both the enhancement and the autoencoder paths, so that disabling the masking mechanism during inference causes SEAMNET to reconstruct the input speech signal. This allows dynamic control of the level of suppression applied by SEAMNET via a minimum gain level, which is not possible in other state-of-the-art approaches to end-to-end speech enhancement. This paper also proposes a perceptually-motivated waveform distance measure. In addition to the b-Net architecture, this paper proposes a novel method for designing target waveforms for network training, so that joint suppression of additive noise and reverberation can be performed by an end-to-end enhancement system, which has not been previously possible. Experimental results show the SEAMNET system to outperform a variety of state-of-the-art baselines systems, both in terms of objective speech quality measures and subjective listening tests. Finally, this paper draws parallels between SEAMNET and conventional statistical model-based enhancement approaches, offering interpretability of many network components.

Deep learning for minimum mean-square error approaches to speech enhancement

Recently, the focus of speech enhancement research has shifted from minimum mean-square error (MMSE) approaches, like the MMSE short-time spectral amplitude (MMSE-STSA) estimator, to state-of-the-art masking- and mapping-based deep learning approaches. We aim to bridge the gap between these two differing speech enhancement approaches. Deep learning methods for MMSE approaches are investigated in this work, with the objective of producing intelligible enhanced speech at a high quality. Since the speech enhancement performance of an MMSE approach improves with the accuracy of the used a priori signal-to-noise ratio (SNR) estimator, a residual long short-term memory (ResLSTM) network is utilised here to accurately estimate the a priori SNR. MMSE approaches utilising the ResLSTM a priori SNR estimator are evaluated using subjective and objective measures of speech quality and intelligibility. The tested conditions include real-world non-stationary and coloured noise sources at multiple SNR levels. MMSE approaches utilising the proposed a priori SNR estimator are able to achieve higher enhanced speech quality and intelligibility scores than recent masking- and mapping-based deep learning approaches. The results presented in this work show that the performance of an MMSE approach to speech enhancement significantly increases when utilising deep learning.Availability: The proposed a priori SNR estimator is available at: https://github.com/anicolson/DeepXi.

Adversarial Training for Speech Super-Resolution

Speech super-resolution or speech bandwidth expansion aims to upsample a given speech signal by generating the missing high-frequency content. In this paper, we propose a deep neural network approach exploiting the adversarial training ideas that have been shown effective in image super-resolution. Specifically, our proposed network follows the generative adversarial networks setup, where the generator network uses a convolutional autoencoder architecture with one-dimensional convolution kernels to generate high-frequency log-power spectra from the low-frequency log-power spectra of the input speech. We propose to use both the reconstruction loss and the adversarial loss for training, and we employ a recent regularization method that penalizes the gradient norms of the discriminator to stabilize the training. We compare our proposed approach with two state-of-the-art neural network baselines and evaluate these methods with both objective speech quality measures and subjective perceptual and intelligibility tests. Results show that our proposed method outperforms both baselines in terms of both objective and subjective evaluations. To gain insights of the network architecture, we analyze key parameters of the proposed network including the number of layers, the number of convolution kernels, and the relative weight of the reconstruction and adversarial losses. Besides, we analyze the computational complexity of our method and the baselines and discuss ways for phase estimation. We further develop a noise-resilient version of the proposed approach by training the network with noisy speech inputs. Objective evaluation validates the noise-resilient property on unseen noise types.

Learning-Based Reference-Free Speech Quality Measures for Hearing Aid Applications

Objective measures of speech quality are highly desirable in benchmarking and monitoring the performance of hearing aids (HAs). Existing HA speech quality indices such as the hearing aid speech quality index (HASQI) are intrusive in that they require a properly time-aligned and frequency-shaped reference signal to predict the quality of HA output. Two new reference-free HA speech quality indices are proposed in this paper, based on a model that amalgamates perceptual linear prediction (PLP), hearing loss (HL) modeling, and machine learning concepts. For the first index, HL-modified PLP coefficients and their statistics were used as the feature set, which was subsequently mapped to the predicted quality scores using support vector regression (SVR). For the second index, HL-impacted gammatone auditory filterbank energies and their second-order statistics constituted the feature set, which was again mapped using SVR. Two databases involving HA recordings were collected and utilized for the evaluation of the robustness and generalizability of the two indices. Experimental results showed that the index based on the gammatone filterbank energies not only correlated well with HA quality ratings by hearing impaired listeners, but also exhibited robust performance across different test conditions and was comparable to the full-reference HASQI performance.

Evaluating noise suppression methods for recovering the Lombard speech from vocal output in an external noise field

Speech production is affected by noise due to the Lombard effect. The traditional method of investigation is through headphone delivery of noise to allow speech to be recorded in quiet, but this could create an occlusion effect artefact during speech production. It is also not directly applicable when wearing hearing protectors, hearing aids, or other devices due to physical interference by the headphones. In these situations, the Lombard effect needs to be elicited by an external noise field and speech recorded in the presence of noise. This is a more challenging measurement situation, but one that preserves perception of own voice and the surrounding noise in interaction with the hearing device worn. Two methods, direct waveform subtraction and adaptive noise cancellation, were evaluated for suppressing the background noise in the recorded speech..The effects of sound recording configuration on performance was investigated for two microphone types (omnidirectional and directional) at two distances (50 and 25 cm) in different noises and in the presence of real talker’s movement. Results show that the amount of noise reduction with both suppression methods is greater for fluctuating than continuous noises. Overall, the best recording configuration for noise reduction was with the omnidirectional microphone at 25 cm. Pitch extraction, energy level, and objective speech intelligibility and quality measures show that both suppression methods provide adequate noise reduction for SNRs as low as − 10 dB, which is suitable to successfully recover Lombard speech produced in an external noise field with open ears and when wearing hearing protectors.

Ideal neighbourhood mask for speech enhancement

A novel approach for speech enhancement applications by applying spectral mask estimation is introduced. The new application uses the local binary patterns to estimate an ideal neighbourhood mask. This will indicate which time–frequency units of the noisy speech are dominated by the noise. The performance assessment of the proposed application in conjunction with the traditional mask techniques, i.e. ideal binary mask and ideal ratio mask, are carried out under various environments in terms of the objective speech quality measures, as well as word error rate performance in speech recognition systems using deep neural networks. Results indicated that the proposed mask yielded significantly better performance than the conventional techniques.

Real-time Simulink implementation of noise adaptive speech processing pipeline of cochlear implants

This paper presents the real-time Simulink implementation of a noise adaptive speech processing pipeline for cochlear implants that was developed in a previous work. After providing an overview of each component or module in the pipeline, it is described how each module is implemented in Simulink so that the input audio frames are processed in real-time. This Simulink implementation allows the same code to be run on different hardware boards that are supported by Simulink. The performance of this implemented pipeline is evaluated in terms of six objective measures of speech quality. The results obtained indicate the effectiveness of suppressing noise in this speech processing pipeline when using the implemented automatic mechanism to identify the noise environment.

Neural decoding of attentional selection in multi-speaker environments without access to clean sources

Objective. People who suffer from hearing impairments can find it difficult to follow a conversation in a multi-speaker environment. Current hearing aids can suppress background noise; however, there is little that can be done to help a user attend to a single conversation amongst many without knowing which speaker the user is attending to. Cognitively controlled hearing aids that use auditory attention decoding (AAD) methods are the next step in offering help. Translating the successes in AAD research to real-world applications poses a number of challenges, including the lack of access to the clean sound sources in the environment with which to compare with the neural signals. We propose a novel framework that combines single-channel speech separation algorithms with AAD. Approach. We present an end-to-end system that (1) receives a single audio channel containing a mixture of speakers that is heard by a listener along with the listener’s neural signals, (2) automatically separates the individual speakers in the mixture, (3) determines the attended speaker, and (4) amplifies the attended speaker’s voice to assist the listener. Main results. Using invasive electrophysiology recordings, we identified the regions of the auditory cortex that contribute to AAD. Given appropriate electrode locations, our system is able to decode the attention of subjects and amplify the attended speaker using only the mixed audio. Our quality assessment of the modified audio demonstrates a significant improvement in both subjective and objective speech quality measures. Significance. Our novel framework for AAD bridges the gap between the most recent advancements in speech processing technologies and speech prosthesis research and moves us closer to the development of cognitively controlled hearable devices for the hearing impaired.

Neural decoding of attentional selection in multi-speaker environments without access to separated sources.

People who suffer from hearing impairments can find it difficult to follow a conversation in a multi-speaker environment. Modern hearing aids can suppress background noise; however, there is little that can be done to help a user attend to a single conversation without knowing which speaker is being attended to. Cognitively controlled hearing aids that use auditory attention decoding (AAD) methods are the next step in offering help. A number of challenges exist, including the lack of access to the clean sound sources in the environment with which to compare with the neural signals. We propose a novel framework that combines single-channel speech separation algorithms with AAD. We present an end-to-end system that 1) receives a single audio channel containing a mixture of speakers that is heard by a listener along with the listener's neural signals, 2) automatically separates the individual speakers in the mixture, 3) determines the attended speaker, and 4) amplifies the attended speaker's voice to assist the listener. Using invasive electrophysiology recordings, our system is able to decode the attention of a subject and detect switches in attention using only the mixed audio. We also identified the regions of the auditory cortex that contribute to AAD. Our quality assessment of the modified audio demonstrates a significant improvement in both subjective and objective speech quality measures. Our novel framework for AAD bridges the gap between the most recent advancements in speech processing technologies and speech prosthesis research and moves us closer to the development of cognitively controlled hearing aids.

Spectral difference for statistical model-based speech enhancement in speech recognition

In this paper, we propose a statistical model-based speech enhancement technique using the spectral difference scheme for the speech recognition in virtual reality. In the analyzing step, two principal parameters, the weighting parameter in the decision-directed (DD) method and the long-term smoothing parameter in noise estimation, are uniquely determined as optimal operating points according to the spectral difference under various noise conditions. These optimal operating points, which are specific according to different spectral differences, are estimated based on the composite measure, which is a relevant criterion in terms of speech quality. An efficient mapping function is also presented to provide an index of the metric table associated with the spectral difference so that operating points can be determined according to various noise conditions for an on-line step. In the on-line speech enhancement step, different parameters are chosen on a frame-by-frame basis under the metric table of the spectral difference. The performance of the proposed method is evaluated using objective and subjective speech quality measures in various noise environments. Our experimental results show that the proposed algorithm yields better performances than conventional algorithms.

Speech quality assessment using 2D neurogram orthogonal moments

This study proposes a new objective speech quality measure using the responses of a physiologically-based computational model of auditory nerve (AN). The population response of the model AN fibers to a speech signal is represented by a 2D neurogram, and features of the neurogram are extracted by orthogonal moments. A special type of orthogonal moment, the orthogonal Tchebichef-Krawtchouk moment, is used in this study. The proposed measure is compared to the subjective scores from two standard databases, the NOIZEUS and the supplement 23 to the P series (P.Sup23) of ITU-T Recommendations. The NOIZEUS database is used in the assessment of 11 speech enhancement algorithms whereas the P.Sup23 database is used in the ITU-T 8 kbit/s codec (Recommendation G.729) characterization test. The performance of the proposed speech quality measure is also compared to the results from some traditional objective quality measures. In general, the proposed neural-response-based metric yielded better results than most of the traditional acoustic-property-based quality measures. The proposed metric can be applied to evaluate the performance of various speech-enhancement algorithms and compression systems.

Novel adaptive muting technique for packet loss concealment of ITU-T G.722 using optimized parametric shaping functions

Adaptive muting method using an optimized parametric shaping function as a part of the ITU-T G.722 Appendix IV packet loss concealment algorithm is proposed. The packet loss concealment algorithm incorporating an adaptive muting scheme is known to prevent the generation of unpleasant sounds during packet loss concealment. However, original muting uses a piece-wise linear muting curve according to packet errors so that our muting approach uses non-linear parametric shaping functions including sigmoid and raised-cosine. Training is substantially performed to determine the parameters of the two shaping functions in terms of objective speech quality measures, and optimal parameters are finally selected by subjective speech quality. Through extensive experiments, this proposed muting technique turns out to improve the performance of the reference muting mechanisms in the packet loss concealment algorithm of the G.722 Appendix IV under various experimental conditions.

SPEECH ENHANCEMENT USING SPECTRAL SUBTRACTION TECHNIQUE WITH MINIMIZED CROSS SPECTRAL COMPONENTS

The aim of speech enhancement is to get significant reduction of noise and enhanced speech from noisy speech. There are several approaches for speech enhancement .earlier approaches didn’t consider cross spectral terms into account. Cross spectral terms become prominent when processing window size becomes small i.e. 20ms-30ms. In this paper, an enhancement method is proposed for significant reduction of noise, and improvement in the quality and perceptibility of speech degraded by correlated additive background noise. The proposed method is based on the spectral subtraction technique. The simple spectral subtraction technique results in poor reduction of noise. One of the main reasons for this is neglecting the cross spectral terms of speech and noise, based on the appropriation that clean speech and noise signals are completely uncorrelated to each other, which is not true on short time basis. In this paper an improvement in reduction of the noise is achieved as compared to the earlier methods. This fact is mainly attributed to the cross spectral terms between speech and noise. This algorithm can be implemented and used in hearing aids for the benefit of hearing impaired people. Objective speech quality measures, spectrogram analyses and subjective listening tests conforms the proposed method is more effective in comparison with earlier speech enhancement techniques.

Supervised Single-Microphone Multi-Talker Speech Separation with Conditional Random Fields

We apply conditional random field (CRF) for single-microphone speech separation in a supervised learning scenario. We train the parameters with mixture data in which the sources are competing with the same average signal power. Compared with factorial hidden Markov model (HMM) baselines, the CRF settings require fewer training mixture data to improve objective speech quality measures and speech recognition accuracy of the reconstructed sources, when mixing ratios of training and testing mixture data are matched. The CRF settings also handle minor mixing ratio mismatch after adjusting the gain factors of the sources with non-linear mappings inspired from the mixture-maximization model. When the mixing ratio mismatch further increases such that the speech mixture is dominated by only one source, factorial HMM finally catches up with and performs better than the CRF settings due to improved model accuracy. We also develop a convex statistical inference simplification based on linear-chain CRFs. The simplification achieves the same performance level as the original CRF settings after integrating additional observations.

Reconstruction of audio waveforms from spike trains of artificial cochlea models.

Spiking cochlea models describe the analog processing and spike generation process within the biological cochlea. Reconstructing the audio input from the artificial cochlea spikes is therefore useful for understanding the fidelity of the information preserved in the spikes. The reconstruction process is challenging particularly for spikes from the mixed signal (analog/digital) integrated circuit (IC) cochleas because of multiple non-linearities in the model and the additional variance caused by random transistor mismatch. This work proposes an offline method for reconstructing the audio input from spike responses of both a particular spike-based hardware model called the AEREAR2 cochlea and an equivalent software cochlea model. This method was previously used to reconstruct the auditory stimulus based on the peri-stimulus histogram of spike responses recorded in the ferret auditory cortex. The reconstructed audio from the hardware cochlea is evaluated against an analogous software model using objective measures of speech quality and intelligibility; and further tested in a word recognition task. The reconstructed audio under low signal-to-noise (SNR) conditions (SNR < –5 dB) gives a better classification performance than the original SNR input in this word recognition task.

Assessment of the impact of system phase response non-linearity on the speech signals quality

It is shown that phase distortion of speech signals are acceptable for human auditory system when the maximum difference of group delay times in the high and low frequencies is below 50 ms - the interference between adjacent vowels and consonants is not perceived with such a difference of group delay. There were founded values of objective measures of speech quality in the form of a segmental signal-to-noise ratio (SSNR), the log-spectral distortion (LSD), bark spectral distortion (BSD) and perceptual evaluation of speech quality (PESQ), according to the detected threshold value of 50 ms. Bibl. 7, Fig. 6, Tab. 1.

On pre-image iterations for speech enhancement.

In this paper, we apply kernel PCA for speech enhancement and derive pre-image iterations for speech enhancement. Both methods make use of a Gaussian kernel. The kernel variance serves as tuning parameter that has to be adapted according to the SNR and the desired degree of de-noising. We develop a method to derive a suitable value for the kernel variance from a noise estimate to adapt pre-image iterations to arbitrary SNRs. In experiments, we compare the performance of kernel PCA and pre-image iterations in terms of objective speech quality measures and automatic speech recognition. The speech data is corrupted by white and colored noise at 0, 5, 10, and 15 dB SNR. As a benchmark, we provide results of the generalized subspace method, of spectral subtraction, and of the minimum mean-square error log-spectral amplitude estimator. In terms of the scores of the PEASS (Perceptual Evaluation Methods for Audio Source Separation) toolbox, the proposed methods achieve a similar performance as the reference methods. The speech recognition experiments show that the utterances processed by pre-image iterations achieve a consistently better word recognition accuracy than the unprocessed noisy utterances and than the utterances processed by the generalized subspace method.