Speech Enhancement Performance Research Articles

Speech Enhancement based on Compressive Sensing Algorithm

There are various methods, in performance of speech enhancement, have been proposed over the years. The accurate method for the speech enhancement design mainly focuses on quality and intelligibility. The method proposed with high performance level. A novel speech enhancement by using compressive sensing (CS) is a new paradigm of acquiring signals, fundamentally different from uniform rate digitization followed by compression, often used for transmission or storage. Using CS can reduce the number of degrees of freedom of a sparse/compressible signal by permitting only certain configurations of the large and zero/small coefficients, and structured sparsity models. Therefore, CS is significantly provides a way of reconstructing a compressed version of the speech in the original signal by taking only a small amount of linear and non-adaptive measurement. The performance of overall algorithms will be evaluated based on the speech quality by optimise using informal listening test and Perceptual Evaluation of Speech Quality (PESQ). Experimental results show that the CS algorithm perform very well in a wide range of speech test and being significantly given good performance for speech enhancement method with better noise suppression ability over conventional approaches without obvious degradation of speech quality.

Improved Speech Enhancement Algorithm Based on Bark Bands Noise-Estimation for Non-Stationary Environment

The conventional spectrum subtraction algorithm cannot effectively suppress the noise under highly non-stationary environment and results in the remaining music noise is often heard in the enhanced speech. In order to improve the speech enhancement performance, a novel denoising algorithm is proposed, which is based on speech endpoint detection using spectrum variance and the dynamic spectrum subtraction in Bark bands. According to human auditory characteristics, the Bark bands spectrums of the noisy speech signal are firstly calculated, and the noise power spectrum of each Bark band is then tracked and estimated by the improved minima controlled recursive averaging method. This noise estimation is adjustable frame by frame and more accurate for non-stationary environment. The experiment results showed that the proposed method can suppress the noise more efficiently than the conventional spectrum subtraction and the remaining music noise is almost eliminated.

Two-channel multi-stage speech enhancement for noisy fMRI environment

Strong acoustic noise interferes with speech communication during functional magnetic resonance imaging (fMRI) scans employing echo planar imaging (EPI) sequences. Performance of many speech enhancement (SE) methods degrades with such a low signal-to-noise ratio (SNR). In this paper, an adaptive SE method is proposed enabling noise-free online speech communication with patients and recording of clean speech. The proposed method is a two-channel, two-stage algorithm with a switching technique. In the first stage, least mean square (LMS) based algorithm and linear prediction error filtering, for specific parts of the noisy acoustic signal, are used. The first stage operations improve the performance of minimum mean square error log-spectral amplitude estimator (Log-MMSE) used for enhancing the speech signal in the second stage. A soft-switching technique is also employed which significantly reduces the audible noise spikes which occur during the transient periods between successive EPI sequencing. The proposed algorithm is tested online during the scans in a noisy fMRI room. Moreover, this paper provides the results of real time implementation of the algorithm on a dedicated laboratory test-bed setup using Texas Instruments digital signal processing platform.

Multichannel Two-Stage Beamforming with Unconstrained Beamformer and Distortion Reduction

This paper proposes a novel multichannel speech enhancement technique for reverberant rooms that is effective when noise sources are spatially stationary, such as a projector fan noise, an air-conditioner noise, and unwanted speech sources at the back of microphones. Speech enhancement performance of the conventional multichannel Wiener filter (MWF) degrades when the Signal-to-Noise Ratio (SNR) of the current microphone input signal changes from the noise-only period. Furthermore, the MWF structure is computationally inefficient, because the MWF updates the whole spatial beamformer periodically to track switching of the speakers (e.g. turn-taking). In contrast to the MWF, the proposed method reduces noise independently of the SNR. The proposed method has a novel two-stage structure, which reduces noise and distortion of the desired source signal in a cascade manner by using two different beamformers. The first beamformer focuses on noise reduction without any constraint on the desired source, which is insensitive to SNR variation. However, the output signal after the first beamformer is distorted. The second beamformer focuses on distortion reduction of the desired source signal. Theoretically, complete elimination of distortion is assured. Additionally, the proposed method has a computationally efficient structure optimized for spatially stationary noise reduction problems. The first beamformer is updated only when the speech enhancement system is initialized. Only the second beamformer is updated periodically to track switching of the active speaker. The experimental results indicate that the proposed method can reduce spatially stationary noise source signals effectively with less distortion of the desired source signal even in a reverberant conference room.

Improvement method of speech enhancement performance of two-microphone array system by applying cascade connection of recursive and non-recursive filters

Improvement method of speech enhancement performance of two-microphone array system by applying cascade connection of recursive and non-recursive filters

Speech enhancement using harmonics regeneration based on multiband excitation

This paper proposes an algorithm that adopts the harmonic regeneration as post-processing to improve the performance of speech enhancement using traditional Short Time Spectral Amplitude (STSA). The proposed algorithm aims to alleviate the distortion of the high harmonics of enhanced speech via the traditional STSA, and consequently improves the speech quality. We first detect the pitch, or fundamental frequency, of the enhanced speech via the traditional STSA, and then, divide the whole spectrum into multiple sub-bands which center on each harmonic. After that, a series of specially designed windows centered on each harmonic are applied to all the sub-bands, in order to redistribute the energy in the sub-bands. The results of experiment demonstrate that the method has both theoretical and practical basis.

Joint Speech Enhancement and Speaker Identification Using Approximate Bayesian Inference

We present a variational Bayesian algorithm for joint speech enhancement and speaker identification that makes use of speaker dependent speech priors. Our work is built on the intuition that speaker dependent priors would work better than priors that attempt to capture global speech properties. We derive an iterative algorithm that exchanges information between the speech enhancement and speaker identification tasks. With cleaner speech we are able to make better identification decisions and with the speaker dependent priors we are able to improve speech enhancement performance. We present experimental results using the TIMIT data set which confirm the speech enhancement performance of the algorithm by measuring signal-to-noise (SNR) ratio improvement and perceptual quality improvement via the Perceptual Evaluation of Speech Quality (PESQ) score. We also demonstrate the ability of the algorithm to perform voice activity detection (VAD). The experimental results also demonstrate that speaker identification accuracy is improved.

Speech Enhancement in the Karhunen-Loève Expansion Domain

This book is devoted to the study of the problem of speech enhancement whose objective is the recovery of a signal of interest (i.e., speech) from noisy observations. Typically, the recovery process is accomplished by passing the noisy observations through a linear filter (or a linear transformation). Since both the desired speech and undesired noise are filtered at the same time, the most critical issue of speech enhancement resides in how to design a proper optimal filter that can fully take advantage of the difference between the speech and noise statistics to mitigate the noise effect as much as possible while maintaining the speech perception identical to its original form. The optimal filters can be designed either in the time domain or in a transform space. As the title indicates, this book will focus on developing and analyzing optimal filters in the Karhunen-Loeve expansion (KLE) domain. We begin by describing the basic problem of speech enhancement and the fundamental rinciples to solve it in the time domain. We then explain how the problem can be equivalently formulated in the KLE domain. Next, we divide the general problem in the KLE domain into four groups, depending on whether interframe and interband information is accounted for, leading to four linear models for speech enhancement in the KLE domain. For each model, we introduce signal processing measures to quantify the performance of speech enhancement, discuss the formation of different cost functions, and address the optimization of these cost functions for the derivation of different optimal filters. Both theoretical analysis and experiments will be provided to study the performance of these filters and the links between the KLE-domain and time-domain optimal filters will be examined. Table of Contents: Introduction / Problem Formulation / Optimal Filters in the Time Domain / Linear Models for Signal Enhancement in the KLE Domain / Optimal Filters in the KLE Domain with Model 1 / Optimal ilters in the KLE Domain with Model 2 / Optimal Filters in the KLE Domain with Model 3 / Optimal Filters in the KLE Domain with Model 4 / Experimental Study

Two-Dimensional FIR Filter Design on Wideband Speech Enhancement in Microphone Array

Due to complexity of sound environment, the performance of speech enhancement is degraded greatly based on traditional beam-forming when directivity of arrival of sound and number of sound are not been estimated precisely. In this paper, according to characteristics of handle-communicated equipment, a method on suppression of environmental noise and disturbance is provided. Beam-forming on speech wideband in desired angle is realized by designing a two-dimensional FIR filter which has fan structure in pass-band and wedge-shaped structure in transition-band. The filter can suppress noise and disturbance out of main-lobe desired. The simulations demonstrated that the filter designed is effective.

Speech Enhancement and Recognition in Meetings With an Audio–Visual Sensor Array

<para xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> This paper addresses the problem of distant speech acquisition in multiparty meetings, using multiple microphones and cameras. Microphone array beamforming techniques present a potential alternative to close-talking microphones by providing speech enhancement through spatial filtering. Beamforming techniques, however, rely on knowledge of the speaker location. In this paper, we present an integrated approach, in which an audio– visual multiperson tracker is used to track active speakers with high accuracy. Speech enhancement is then achieved using microphone array beamforming followed by a novel postfiltering stage. Finally, speech recognition is performed to evaluate the quality of the enhanced speech signal. The approach is evaluated on data recorded in a real meeting room for stationary speaker, moving speaker, and overlapping speech scenarios. The results show that the speech enhancement and recognition performance achieved using our approach are significantly better than a single table-top microphone and are comparable to a lapel microphone for some of the scenarios. The results also indicate that the audio–visual-based system performs significantly better than audio-only system, both in terms of enhancement and recognition. This reveals that the accurate speaker tracking provided by the audio–visual sensor array proved beneficial to improve the recognition performance in a microphone array-based speech recognition system. </para>

Noisy Speech Feature Estimation on the Aurora2 Database using a Switching Linear Dynamic Model

This paper presents an approach to enhance speech feature estimation in the log spectral domain under additive noise environments. A switching linear dynamic model (SLDM) is explored as a parametric model for the clean speech distribution, enforcing a state transition in the feature space and capturing the smooth time evolution of speech conditioned on the state sequence. Experimental results using the Aurora2 database show that the new SLDM approach can improve speech enhancement performance in terms of recognition accuracy.

Evaluating the perceptual quality of speech signals enhanced using the Modified Phase Opponency model

Previously, we presented a speech enhancement algorithm based on the auditorily motivated MPO model. It was shown that the MPO‐based speech enhancement scheme is robust to fluctuating noise. Compared to the output of several other speech enhancement techniques, the MPO‐enhanced speech signals exhibit the lowest increase in Itakura‐Saito, log‐area‐ratio, and log‐likelihood‐ratio‐based distortion measures as the SNR reduces. In the present work, the performance of the MPO speech enhancement is compared with that of the other techniques using subjective evaluations. Three normal‐hearing subjects were recruited to perform two‐alternative, forced‐choice preference tasks. Each sentence was presented twice: one of them was enhanced using the MPO scheme and the other was either left unprocessed or enhanced using one of three other speech enhancement techniques. Sentences are 5‐digits long and corrupted by additive subway noise at −5, 0, 5, or 15‐dB SNR. Most of the time, the MPO‐enhanced speech signal was preferred weakly. Moreover, the MPO‐processed clean speech signal was preferred over clean unprocessed speech signal just as many times as the clean unprocessed speech signal was preferred, indicating that the MPO processing does not introduce any perceptual artifacts in clean speech. [Work supported by NSF BCS0236707 DOD H9823005C0425.]

On-line Gaussian mixture modeling in the log-power domain for signal-to-noise ratio estimation and speech enhancement

We present on-line Gaussian mixture modeling (GMM) in the log-power domain of actual noisy speech and its applications to segmental signal-to-noise ratio (SNR) estimation and speech enhancement. The basic idea in this method is the use of conventional two-component GMM modeling in the log-power domain to estimate the distributions of noise and noisy speech subspaces in each speech segment of a length of 0.5–2 s. Given the subspace distributions, the statistical estimation method is adopted in the applications. For the segmental SNR estimation, the average speech level is estimated from noisy speech using a nonlinear moment of modeled distributions. This method is suitable under real conditions, when neither reference signals nor speech activity is available, and is shown to be more robust and accurate than conventional methods, particularly under low-SNR conditions. The proposed GMM model is extended to the multiband log-power domains for noise estimation. We use long-term information, which is obtained by GMM modeling in each segment of 0.5 s, to update the local distributions of noise and noisy speech power at each actual time–frequency index. The cumulative distribution function equalization (CDFE) is then used to estimate the noise and subtract it from the noisy speech power. The advantage of the CDFE method for noise estimation is that the estimation is given in the logarithmic domain without any approximation. The proposed speech enhancement is tested using the AURORA-2J database. We also compare the proposed method to the conventional minimum statistic and quantile-based noise estimation. The proposed method is found to be superior to the conventional in the speech recognition rate over most noise environments and shown to provide very good compromise between speech enhancement and speech recognition performance.

Application of spectral subtraction method on enhancement of electrolarynx speech

Although electrolarynx (EL) serves as an important method of phonation for the laryngectomees, the resulting speech is of poor intelligibility due to the presence of a steady background noise caused by the instrument, even worse in the case of additive noise. This paper investigates the problem of EL speech enhancement by taking into account the frequency-domain masking properties of the human auditory system. One approach is incorporating an auditory masking threshold (AMT) for parametric adaptation in a subtractive-type enhancement process. The other is the supplementary AMT (SAMT) algorithm, which applies a cross-correlation spectral subtraction (CCSS) approach as a post-processing scheme to enhancing EL speech dealt with the AMT method. The performance of these two algorithms was evaluated as compared to the power spectral subtraction (PSS) algorithm. The best performance of EL speech enhancement was associated with the SAMT algorithm, followed by the AMT algorithm and the PSS algorithm. Acoustic and perceptual analyses indicated that the AMT and SAMT algorithms achieved the better performances of noise reduction and the enhanced EL speech was more pleasant to human listeners as compared to the PSS algorithm.

Single-Channel Multiple Regression for In-Car Speech Enhancement

We address issues for improving hands-free speech enhancement and speech recognition performance in different car environments using a single distant microphone. This paper describes a new single-channel in-car speech enhancement method that estimates the log spectra of speech at a close-talking microphone based on the nonlinear regression of the log spectra of noisy signal captured by a distant microphone and the estimated noise. The proposed method provides significant overall quality improvements in our subjective evaluation on the regression-enhanced speech, and performed best in most objective measures. Based on our isolated word recognition experiments conducted under 15 real car environments, the proposed adaptive nonlinear regression approach shows an advantage in average relative word error rate (WER) reductions of 50.8% and 13.1%, respectively, compared to original noisy speech and ETSI advanced front-end (ETSI ES 202 050).

A new approach for speech enhancement based on the adaptive thresholding of the wavelet packets

In this paper, we propose a new speech enhancement system using the wavelet thresholding algorithm. The basic wavelet thresholding algorithm has some defects including the assumption of white Gaussian noise (WGN), malfunction in unvoiced segments, bad auditory quality, etc. In the proposed system, we introduce a new algorithm which does not require any voiced/unvoiced detection system. Also, in this proposed method adaptive wavelet thresholding and modified thresholding functions are introduced to improve the speech enhancement performance as well as the automatic speech recognition (ASR) accuracy. A new voice activity detector (VAD) was designed to update noise statistics in the proposed speech enhancement system when facing to the colored and non-stationary noises. The proposed method was evaluated on several speakers and under various noise conditions including white Gaussian noise, pink noise, and multi-talker babble noise. The SNR and ASR results show that the new method highly improves the performance of speech enhancement algorithm based on the wavelet thresholding.

A high-order recurrent neuro-fuzzy system with internal dynamics: Application to the adaptive noise cancellation

A high-order recurrent neuro-fuzzy system (HO-RNFS) is suggested in this paper, suitable for modeling highly complex nonlinear temporal processes. Feedback connections are introduced in the network including the context and the feedback nodes that serve as a means to memorize the firing history. The feedback paths in the firing loop are implemented through finite impulse response (FIR) synaptic filters leading to a higher-order network with enhanced temporal capabilities. The inference mechanism of the HO-RNFS is implemented by means of dynamic fuzzy rules where multiple steps-ahead predictions are provided for the internal variables, at the consequent part. Its structure is organized in an on-line fashion using a concurrent structure and parameter algorithm. Structure learning generates dynamically the input and output clusters of the rules, while parameter learning adjusts the network weights. The HO-RNFS is compared to the recurrent self-organizing neural fuzzy inference network (RSONFIN), being a special case of the suggested network. The experimental setup includes a benchmark temporal system and the adaptive noise cancellation problem. Extensive experimentation reveals that HO-RNFS exhibits superior speech enhancement performance as contrasted to RSONFIN, when complex noise passages are considered.

A speech enhancement method using the sliding DFT

In this paper, we propose a novel noise reduction of speech signals using sliding DFT (SDFT) which is based on the spectral subtraction method. The spectral subtraction method reduces noise by subtracting the spectrum of noise estimated from an input signal and is a relatively simple and effective technique. We use SDFT to obtain the frequency spectra of speech signals. As is well known, to obtain the successive N-points output, SDFT requires only N times of complex multiplication, although the Fast Fourier Transform (FFT) requires Nlog2N times that. Thus SDFT is computationally simple as compared with FFT. The frame length, N, is determined based on the pitch period. In this paper, the pitch period is estimated by using an ideal low-pass filter and the analytic signal that is obtained from speech signals by the Hilbert converter. Finally, it is shown that the proposed method has good performance for speech enhancement.

Nonstationary-state hidden Markov model representation of speech signals for speech enhancement

A novel formulation of the nonstationary-state hidden Markov model (NS-HMM), employed as the speech model and serving as the theoretical basis for the construction of a speech enhancement system, is presented in this paper. The NS-HMM is used as a compact, parametric model, generalized from the stationary-state HMM, for describing clean speech statistics in the construction of the minimum mean-square-error (MMSE) speech enhancement system. The feature selection problem associated with the use of the NS-HMM in designing the speech enhancement system is addressed. The MMSE formulation is derived where the NS-HMM is used as the clean speech model and Gaussian-mixture, stationary-state HMM as the additive noise model. Speech enhancement experiments are conducted, demonstrating superiority of the NS-HMM over the stationary-state HMM in the speech enhancement performance for low SNRs. Detailed diagnostic analysis on the speech enhancement system's operation shows that the superiority arises from the ability of the NS-HMM to fit the spectral trajectory of the signal embedded in noise more closely than the stationary-state HMM.