Noisy Speech Recognition Research Articles

Speech recognition under noisy environments using segmental unit input HMM

AbstractIn this paper, we apply a segmental unit input HMM to noisy speech recognition. In this modeling, several successive frames are combined and treated as an input vector. We expect that the segmental unit input HMM will be effective for noisy speech recognition because segmental statistics considering correlation between frames reduce noise effects when the correlation of noise between frames is assumed to be small. In recognition experiments, we compared the segmental unit input HMM with a conventional frame‐based HMM and found the segmental unit input HMM to be superior. We also compared the segmental unit input HMM with dynamic cepstral coefficients, which have both static and dynamic features, and found that the segmental unit input HMM is more effective than the dynamic cepstrum. We also combined the segmental unit input HMM with a spectral subtraction method and confirmed the effectiveness of the method. Additionally, in experiments using acoustic models trained with noisy speech, the segmental unit input HMM outperformed the conventional HMM. From these results, we propose PMC for the segmental unit input HMM. Experimental results showed the PMC for segmental unit input HMM offered better recognition performance than the original PMC. © 2002 Wiley Periodicals, Inc. Syst Comp Jpn, 33(8): 111–120, 2002; Published online in Wiley InterScience (www.interscience.wiley.com). DOI 10.1002/scj.1151

Correction to "An SNR-incremental stochastic matching algorithm for noisy speech recognition"

Correction to "An SNR-incremental stochastic matching algorithm for noisy speech recognition"

A design of adaptive beamformer based on average speech spectrum for noisy speech recognition.

A design of adaptive beamformer based on average speech spectrum for noisy speech recognition.

Adaptation method using expectation-maximisation for noisy speech recognition

An expectation-maximisation-based adaptation method for the mean of noise is proposed for noisy speech recognition. The noisy testing speech is used for the unsupervised adaptation by employing the parallel model combination (PMC). The proposed method has shown improved performances compared with the PMC method in the isolated word recognition experiments.

Noisy speech recognition using de-noised multiresolution analysis acoustic features.

This paper describes a novel application of multiresolution analysis (MRA) in extracting acoustic features that possess de-noising capability for robust speech recognition. The MRA algorithm is used to construct a mel-scaled wavelet packet filter-bank, from which subband powers are computed as the feature parameters for speech recognition. Wiener filtering is applied to a few selected subbands at some intermediate stages of decomposition. For high-frequency bands, Wiener filters are designed based on a reduced fraction of the estimated noise power, making the consonant features much more prominent and contrastive. The proposed method is evaluated in phone recognition experiments with the TIMIT database. In the presence of stationary white noise at 10-dB SNR, the de-noised MRA features attain a phone recognition rate of 32%. There is a noticeable improvement compared with the accuracy of 29% and 20% attained by the commonly used mel-frequency cepstral coefficients (MFCC) with and without cepstral mean normalization (CMN), respectively. The effectiveness of the MRA features is also verified by the fact that they exhibit smaller distortion from clean speech.

A probabilistic union model with automatic order selection for noisy speech recognition.

A critical issue in exploiting the potential of the sub-band-based approach to robust speech recognition is the method of combining the sub-band observations, for selecting the bands unaffected by noise. A new method for this purpose, i.e., the probabilistic union model, was recently introduced. This model has been shown to be capable of dealing with band-limited corruption, requiring no knowledge about the band position and statistical distribution of the noise. A parameter within the model, which we call its order, gives the best results when it equals the number of noisy bands. Since this information may not be available in practice, in this paper we introduce an automatic algorithm for selecting the order, based on the state duration pattern generated by the hidden Markov model (HMM). The algorithm has been tested on the TIDIGITS database corrupted by various types of additive band-limited noise with unknown noisy bands. The results have shown that the union model equipped with the new algorithm can achieve a recognition performance similar to that achieved when the number of noisy bands is known. The results show a very significant improvement over the traditional full-band model, without requiring prior information on either the position or the number of noisy bands. The principle of the algorithm for selecting the order based on state duration may also be applied to other sub-band combination methods.

Union: A new approach for combining sub-band observations for noisy speech recognition

Recent studies have shown that the sub-band based speech recognition approach has the potential of improving upon the conventional, full-band based model against frequency-selective noise. A critical issue towards exploiting this potential is the choice of the method for combining the sub-band observations. This paper introduces a new method, namely, the probabilistic-union model, for this combination. The new model is based on the probability theory for the union of random events, and represents a new method for modeling partially corrupted observations given little knowledge about the corruption. The new model has been incorporated into a hidden Markov model (HMM) and tested for recognizing a speaker-independent E-set, corrupted by various types of additive noise. The results show that the new model offers robustness to partial frequency corruption, requiring little or no knowledge about the noise statistics.

Noisy speech processing by recurrently adaptive fuzzy filters

Two noisy speech processing problems-speech enhancement and noisy speech recognition-are dealt with. The technique we focus on is by using the filtering approach; a novel filter, the recurrently adaptive fuzzy filter (RAFF), is proposed and applied to these two problems. The speech enhancement is based on adaptive noise cancellation with two microphones, where the RAFF is used to eliminate the noise corrupting the desired speech signal in the primary channel. As to the noisy speech recognition, the RAFF is used to filter the noise in the feature domain of speech signals. The RAFF is inherently a recurrent multilayered connectionist network for realizing the basic elements and functions of dynamic fuzzy inference, and may be considered to be constructed from a series of dynamic fuzzy rules. As compared to other existing nonlinear filters, three major advantages of the RAFF are observed: 1) a priori knowledge can be incorporated into the RAFF, which makes the fusion of numerical data and linguistic information possible; 2) owing to the dynamic property of the RAFF, the exact lagged order of the input variables need not be known in advance; 3) no predetermination, like the number of hidden nodes, must be given since the RAFF can find its optimal structure and parameters automatically Several examples on adaptive noise cancellation and noisy speech recognition problems using the RAFF are illustrated to demonstrate the performance of the RAFF.

Recognition of noisy speech by a nonstationary AR HMM with gain adaptation under unknown noise

In this paper, a gain-adapted speech recognition method in unknown noise is developed in the time domain. Noise is assumed to be colored. To cope with the notable nonstationary nature of speech signals such as fricative, glides, liquids, and transition region between phones, the nonstationary autoregressive (NAR) hidden Markov model (HMM) is used for clean speech. The nonstationary AR process is represented by using polynomial functions with a linear combination of M known basis functions. When only noisy signals are available, the estimation problem of unknown noise inevitably arises. By using multiple Kalman filters, the estimation of noise model and gain contour of speech is performed.

Robust features for speech recognition based on admissible wavelet packets

A six-band filter structure derived by using admissible wavelet packet for the extraction of the features for recognition of noisy speech is proposed. A simple compensation for white Gaussian noise is carried out and the recognition performance is compared with the features based on Mel scale cepstral coefficients (MFCC) and 24-band admissible wavelet packet filter structure.

An SNR-incremental stochastic matching algorithm for noisy speech recognition

In this paper, an signal-to-noise ratio (SNR)-incremental stochastic matching (SISM) algorithm is proposed for robust speech recognition in noisy environments. The SISM algorithm is an extension of Sankar and Lee's (1996) stochastic matching (SM) for dealing with the distortion due to additive noise. We address two issues concerning the original maximum likelihood-based SM techniques. One concern is that the initial condition of the expectation-maximization (EM) algorithm has to be set carefully if the mismatch between training and testing is large. The other is that the performance is often limited by the newly adapted model in noise compensation instead of reaching the higher level of accuracy often obtained in clean environments. Our proposed SISM algorithm attempts to improve the initial condition and to relax the performance bound. First, the SISM algorithm provides a good initial condition making use of a set of environment-matched models. The second is a recursive operation, i.e., the reference model in each recursion is changed along the direction of SNR increment in order to push the recognition performance to that obtained at higher SNR levels. Experimental results show that the SISM algorithm provides further improvement after the best environment-matched performance has been reached, and can therefore obtain an additional discriminative power through using the speech models with higher SNR instead of retraining process.

The application of segment-based orthogonal transform in signal bias removal

The iterative technique for signal bias removal (SBR) to minimize the effects of corrupted speech with unknown noisy sources was proposed for robust speech recognition. The SBR method was applied by considering a multiplicative spectral bias or equivalently, an additive cepstral bias. Two problems of this approach can be found. One is that it needs sufficient iterations to attain better results and the other difficulty is the lower accuracy of bias estimation. To overcome these drawbacks, a novel approach of SBR incorporated with orthogonal transform is adopted in this study. Instead of the conventional frame-based process, this method is a segment-by-segment approach. The basic idea of this approach is that the segment constituted by any feature element will be modeled using orthogonal transform coefficients. Owing to the characteristics of orthogonal transform, the high-order coefficients are bias-free and only the zeroth order coefficient is bias-corrupted. The high-order coefficients can therefore be used for bias estimation and the real bias will be deduced from the zeroth-order coefficient. Effectiveness of the method was confirmed by an experiment of multispeaker noisy continuous Mandarin speech recognition. The significant improvement on computation and recognition rate can be achieved by this method.

A fast algorithm for parallel model combination for noisy speech recognition

Based on the log-normal assumption, parallel model combination (PMC) provides an effective method to adapt the cepstral means and variances of speech models for noisy speech recognition. In addition, the log-add method has been derived to adapt the mean by ignoring the cepstral variance during the process of PMC. This method is efficient for speech recognition in a high signal-to-noise ratio (SNR) environment. In this paper, a new interpretation of the log-add method is proposed. This leads to a modified scheme for performing the adaptation procedure in PMC. This modified method is shown to be efficient in improving recognition accuracy in low SNR. Based on this modified PMC method, we derive a direct adaptation procedure for the variance of speech models in the cepstral domain. The proposed method is a fast algorithm because the computation for the transformation of the covariance matrix is no longer required. Three recognition tasks are conducted to evaluate the proposed method. Experimental results show that the proposed technique not only requires lower computational cost but it also outperforms the original PMC technique in noisy environments.

Combining signal limiters with noise-robust techniques by using SNR-dependent adaptive interpolation scheme

An investigation is presented into the feasibility of combining signal limiters with a noise-robust technique for seeking further performance improvements in noisy speech recognition.

GA-based noisy speech recognition using two-dimensional cepstrum

Among various kinds of speech features, the two-dimensional (2-D) cepstrum (TDC) is a special one, which can simultaneously represent several types of information contained in the speech waveform: static and dynamic features, as well as global and fine frequency structures. Analysis results show that the coefficients located at lower indexes portion of the TDC matrix seem to be more significant than others. Hence, to represent an utterance only some TDC coefficients need to be selected to form a feature vector instead of the sequence of feature vectors. It has the advantages of simple computation and less storage space. However, our experiments show that the selection of TDC coefficients is quite sensitive to background noise. In order to solve this problem, we propose the GA-based M-TDC (modified TDC) method in this paper to improve the representativeness and robustness of the selected TDC coefficients in noisy environments. The M-TDC differs from the standard TDC by the use of filters to remove the noise components. Furthermore, in the GA-based M-TDC method, we apply the genetic algorithms (GAs) to find the robust coefficients in the M-TDC matrix. From the experiments with five noise types, we find that the GA-based M-TDC method has better recognition results than the original TDC approaching noisy environments.

Noise-compensated hidden Markov models

The technique of hidden Markov models has been established as one of the most successful methods applied to the problem of speech recognition. However, its performance is considerably degraded when the speech signal is contaminated by noise. This work presents a technique which improves the performance of hidden Markov models when these models are used in different noise conditions during the speech recognition process. The input speech signal enters unchanged to the recognition process, while the models used by the recognition system are compensated according to the affecting noise characteristics, power and spectral shape. Hence, the compensation stage is independent of the recognition stage, allowing the models to be continually adjusted. The models used in this work are from a continuous density hidden Markov algorithm, having cepstral coefficients derived from linear predictive analysis as state parameters. It is used only static features in the models in order to show that, when properly compensated for the noise, these static features contribute significantly to improve noisy speech recognition. It is observed from the results that the parameters kept their capability to discriminate among different classes of signals, indicating that, in the context of speech recognition, the use of autoregressive-derived parameters with noisy signals does not represent an impediment. A matrix-way of converting from autoregressive coefficients to normalized autocorrelation coefficients is presented. The affecting noise is assumed additive and statistically independent of the speech signal. Although the noise dealt with should also be stationary, good performance was achieved for nonstationary noise, such as operations room noise and factory environment noise. The concept of intra-word signal-to-noise ratio is presented and successfully applied. The resulting compensated models are revealed to be less dependent on the training data set when compared to the trained hidden Markov models. Due to the computational simplicity, the time required to adjust a model is significantly shorter than the time to train it.

Smoothing hidden Markov models by using an adaptive signal limiter for noisy speech recognition

When a speech recognition system is deployed in the real world, environmental interference will make noisy speech signals and reference models mismatched and cause serious degradation in recognition accuracy. To deal with the effect of environmental mismatch, a family of signal limiters has been successfully applied to a template-based DTW recognizer to reduce the variability of speech features in noisy conditions. Though simulation results indicate that heavily smoothing can effectively reduce the variability of speech features in low signal-to-noise ratio (SNR), it would also cause the loss of information in speech features. Therefore, we suggest that the smoothing factor of a signal limiter should be related to SNR and adapted on a frame by frame basis. In this paper, an adaptive signal limiter (ASL) is proposed to smooth the instantaneous and dynamic spectral features of reference models and test speech. By smoothing spectral features, the smoothed covariance matrices of reference models can be obtained by means of maximum likelihood (ML) estimation. A speech recognition task for multispeaker isolated Mandarin digits has been conducted to evaluate the effectiveness and robustness of the proposed method. Experimental results indicate that the adaptive signal limiter can achieve significant improvement in noisy conditions and is more robust than the hard limiter over a wider range of SNR values.

Robust features for noisy speech recognition based on temporal trajectory filtering of short-time autocorrelation sequences

This paper introduces a new representation of speech that is invariant to noise. The idea is to filter the temporal trajectories of short-time one-sided autocorrelation sequences of speech such that the noise effect is removed. The filtered sequences are denoted by the relative autocorrelation sequences (RASs), and the mel-scale frequency cepstral coefficients (MFCC) are extracted from RAS instead of the original speech. This new speech feature set is denoted as RAS-MFCC. Experiments were conducted on a task of multispeaker isolated Mandarin digit recognition to demonstrate the effectiveness of RAS-MFCC features in the presence of white noise and colored noise. The proposed features are also shown to be superior to other robust representations and compensation techniques.

Subband-based blind signal separation for noisy speech recognition

A method for directly extracting clean speech features from noisy speech is proposed. This process is based on independent component analysis (ICA) and a new feature analysis technique for reducing the computational complexity of the frequency domain ICA. For noisy speech signals recorded in real environments, this method yielded a considerable performance improvement.

Mean compensation based on projection-based group delay scheme for noisy speech recognition

A mean vector compensation technique based on the projection-based group delay scheme has been combined with a semi-continuous HMM to improve the recognition rate in noisy environments. The proposed approach compensates the mean vector using a projection-based scale factor and the bias estimated from the training and/or testing data to balance the mismatch between different environments. Experiments show that the significant improvement in speaker-dependent, isolated word recognition was achieved by adding the projection-based scale factor and mean vector bias.