Speaker Adaptation In Speech Recognition Research Articles

Eigennoise Speech Recovery in Adverse Environments with Joint Compensation of Additive and Convolutive Noise

The learning-based speech recovery approach using statistical spectral conversion has been used for some kind of distorted speech as alaryngeal speech and body-conducted speech (or bone-conducted speech). This approach attempts to recover clean speech (undistorted speech) from noisy speech (distorted speech) by converting the statistical models of noisy speech into that of clean speech without the prior knowledge on characteristics and distributions of noise source. Presently, this approach has still not attracted many researchers to apply in general noisy speech enhancement because of some major problems: those are the difficulties of noise adaptation and the lack of noise robust synthesizable features in different noisy environments. In this paper, we adopted the methods of state-of-the-art voice conversions and speaker adaptation in speech recognition to the proposed speech recovery approach applied in different kinds of noisy environment, especially in adverse environments with joint compensation of additive and convolutive noises. We proposed to use the decorrelated wavelet packet coefficients as a low-dimensional robust synthesizable feature under noisy environments. We also proposed a noise adaptation for speech recovery with the eigennoise similar to the eigenvoice in voice conversion. The experimental results showed that the proposed approach highly outperformed traditional nonlearning-based approaches.

Fast Adaptation of Deep Neural Network Based on Discriminant Codes for Speech Recognition

Fast adaptation of deep neural networks (DNN) is an important research topic in deep learning. In this paper, we have proposed a general adaptation scheme for DNN based on discriminant condition codes, which are directly fed to various layers of a pre-trained DNN through a new set of connection weights. Moreover, we present several training methods to learn connection weights from training data as well as the corresponding adaptation methods to learn new condition code from adaptation data for each new test condition. In this work, the fast adaptation scheme is applied to supervised speaker adaptation in speech recognition based on either frame-level cross-entropy or sequence-level maximum mutual information training criterion. We have proposed three different ways to apply this adaptation scheme based on the so-called speaker codes: i) Nonlinear feature normalization in feature space; ii) Direct model adaptation of DNN based on speaker codes; iii) Joint speaker adaptive training with speaker codes. We have evaluated the proposed adaptation methods in two standard speech recognition tasks, namely TIMIT phone recognition and large vocabulary speech recognition in the Switchboard task. Experimental results have shown that all three methods are quite effective to adapt large DNN models using only a small amount of adaptation data. For example, the Switchboard results have shown that the proposed speaker-code-based adaptation methods may achieve up to 8-10% relative error reduction using only a few dozens of adaptation utterances per speaker. Finally, we have achieved very good performance in Switchboard (12.1% in WER) after speaker adaptation using sequence training criterion, which is very close to the best performance reported in this task (“Deep convolutional neural networks for LVCSR,” T. N. Sainath , Proc. IEEE Acoust., Speech, Signal Process., 2013).

Speaker-adaptive speech recognition using speaker diarization for improved transcription of large spoken archives

This paper deals with speaker-adaptive speech recognition for large spoken archives. The goal is to improve the recognition accuracy of an automatic speech recognition (ASR) system that is being deployed for transcription of a large archive of Czech radio. This archive represents a significant part of Czech cultural heritage, as it contains recordings covering 90years of broadcasting. A large portion of these documents (100,000h) is to be transcribed and made public for browsing. To improve the transcription results, an efficient speaker-adaptive scheme is proposed. The scheme is based on integration of speaker diarization and adaptation methods and is designed to achieve a low Real-Time Factor (RTF) of the entire adaptation process, because the archive’s size is enormous. It thus employs just two decoding passes, where the first one is carried out using the lexicon with a reduced number of items. Moreover, the transcripts from the first pass serve not only for adaptation, but also as the input to the speaker diarization module, which employs two-stage clustering. The output of diarization is then utilized for a cluster-based unsupervised Speaker Adaptation (SA) approach that also utilizes information based on the gender of each individual speaker. Presented experimental results on various types of programs show that our adaptation scheme yields a significant Word Error Rate (WER) reduction from 22.24% to 18.85% over the Speaker Independent (SI) system while operating at a reasonable RTF.

On the Development of Speech Resources for the Mixtec Language

The Mixtec language is one of the main native languages in Mexico. In general, due to urbanization, discrimination, and limited attempts to promote the culture, the native languages are disappearing. Most of the information available about the Mixtec language is in written form as in dictionaries which, although including examples about how to pronounce the Mixtec words, are not as reliable as listening to the correct pronunciation from a native speaker. Formal acoustic resources, as speech corpora, are almost non-existent for the Mixtec, and no speech technologies are known to have been developed for it. This paper presents the development of the following resources for the Mixtec language: (1) a speech database of traditional narratives of the Mixtec culture spoken by a native speaker (labelled at the phonetic and orthographic levels by means of spectral analysis) and (2) a native speaker-adaptive automatic speech recognition (ASR) system (trained with the speech database) integrated with a Mixtec-to-Spanish/Spanish-to-Mixtec text translator. The speech database, although small and limited to a single variant, was reliable enough to build the multiuser speech application which presented a mean recognition/translation performance up to 94.36% in experiments with non-native speakers (the target users).

Systems and methods for providing online fast speaker adaptation in speech recognition

A system ( 230 ) performs speaker adaptation when performing speech recognition. The system ( 230 ) receives an audio segment and identifies the audio segment as a first audio segment or a subsequent audio segment associated with a speaker turn. The system ( 230 ) then decodes the audio segment to generate a transcription associated with the first audio segment when the audio segment is the first audio segment and estimates a transformation matrix based on the transcription associated with the first audio segment. The system ( 230 ) decodes the audio segment using the transformation matrix to generate a transcription associated with the subsequent audio segment when the audio segment is the subsequent audio segment.

MPE-based discriminative linear transforms for speaker adaptation

In this paper, the use of discriminative linear transforms (DLT) is investigated to construct speaker adaptive speech recognition systems, where a discriminative criterion rather than ML is used for transform parameter estimation. The minimum phone error (MPE) criterion is investigated for DLT estimation, by making use of a so-called weak-sense auxiliary function to derive the estimation formulae. An implementation based on lattices is used for DLT statistics accumulation, where the use of a weakened language model allows more confusion data to be included. To improve DLT estimation for unsupervised adaptation, a method of incorporating word correctness information of the supervision into transform estimation is developed. The confidence scores calculated by confusion network decoding are used to represent the word correctness and weight the numerator statistics during DLT estimation. This makes the DLT estimation less sensitive to errors in the supervision. Experiments on transcription of read newspaper data and on conversational telephone speech transcription have shown the improvements of DLT over MLLR for both supervised and unsupervised adaptation, and the effectiveness of confidence scores for improving both normal and DLT-based MLLR adaptation.

Systematic speaker variation and within-speaker center of gravity correlations in the TIMIT database

The systematicity of speaker variation in consonants was examined by measuring the noise component of consonants in the TIMIT database. Fricatives and stops were compared by measuring at the temporal middle of fricatives and at the release of stops. There is a high correlation between many of the center of gravity measures. For example, if a speaker has a particularly high center of gravity in the sound /t/, she or he will also have a high center of gravity in /d/, /s/, /p/, /k/, and other consonants. The full set of correlations will be described in the paper. The correlations appear to stem from individual differences and not from dialect variations. The implications of the results for rapid speaker adaptation in speech recognition will be explored. [Work supported by San Jose State University Faculty Grant.]

Speaker change detection and speaker clustering using VQ distortion measure

AbstractThis paper addresses the problem of the detection of speaker changes and clustering speakers when no information is available regarding speaker classes or even the total number of classes. This study provides preprocessing for a method which adapts speaker information based on incremental on‐line speaker clustering. We assume that no previous information on speakers is available (no speaker model, no training phase) and that people do not speak simultaneously. The aim is to apply speaker grouping information to speaker adaptation for speech recognition.We use Vector Quantization (VQ) distortion as the criterion. A speaker model is created from successive utterances as a codebook by a VQ algorithm, and the VQ distortion is calculated between the model and an utterance.The results of an experiment on speaker detection and speaker clustering are presented. The speaker change detection experiment was compared with results of the Generalized Likelihood Ratio (GLR) and Bayesian Information Criterion (BIC). By using GLR and BIC, we obtained F‐measures of 75.8 and 81.4, respectively. On the other hand, we obtained an F‐measure of 84.4 by the proposed method. Furthermore, by a combination of the method with an acoustic back‐off method, we obtained a measure of 93.1 and a classification rate of 97.7%. These results show the superiority of our proposed method. Finally, we demonstrate the validity of our proposed method by experiments on speech recognition using on‐line speaker adaptation. © 2003 Wiley Periodicals, Inc. Syst Comp Jpn, 34(13): 25–35, 2003; Published online in Wiley InterScience (www.interscience.wiley.com). DOI 10.1002/scj.10488

Future directions in speech information processing

Speech processing technologies, including speech recognition, synthesis, and coding are expected to play important roles in an advanced multimedia society with user-friendly human–machine interfaces. Speech recognition systems include not only those that recognize messages but also those that recognize the identity of the speaker. This paper predicts future directions in speech processing. It describes the most important research problems and tries to forecast where progress will be made in the near future and what applications will become commonplace as a result of the increased capabilities. The most promising application area is telecommunications. To solve various fundamental problems, a unified approach across speech recognition, synthesis, and coding is indispensable. Handling the common phenomenon of voice individuality from different aspects are: research on speaker adaptation in speech recognition, automatic speaker verification, voice conversion in speech synthesis, and the problems of very low-bit-rate speech coding quality variation from speaker to speaker. The methods that essentially solve such problems should be based on a common mathematical model. Research on the mechanism of speech information processing in our human brains, that is, how meanings of speech are understood and how speech is produced, is also crucial for epoch-making technological development in conversational speech understanding and natural speech synthesis.

N-Best-based unsupervised speaker adaptation for speech recognition

This paper proposes an instantaneous speaker adaptation method that uses N-best decoding for continuous mixture-density hidden-Markov-model-based speech-recognition systems. This method is effective even for speakers whose decoding using speaker-independent (SI) models are error-prone and for whom speaker adaptation techniques are truly needed. In addition, smoothed estimation and utterance verification are introduced into this method. The smoothed estimation is based on the likelihood values for adapted models of word sequences obtained by N-best decoding and improves the performance of error-prone speakers, and the utterance verification technique reduces the amount of calculation required. Performance evaluation using connected-digit (four-digit strings) recognition experiments performed over actual telephone lines showed a reduction of 36·4% in the error rates of speakers whose decoding using SI models are error-prone.

Method of speaker adaptive speech recognition

A method for recognizing spoken words of a speech includes extracting feature vectors from a speech signal which corresponds to a spoken phrase, and segmenting and classifying the successive extracted feature vectors into syllable oriented word subunits by means of a stored supply of word subunits to form a set of hypotheses. The set of hypotheses is used to generate, by three dimensional time dynamic comparision, a set of word hypotheses by comparing the segmented and classified word subunits with standard pronunciations and pronunciation variants of a plurality of words stored in a reference pattern vocabulary. The generated set of word hypotheses are then subjected to syntactic analysis to determine the spoken phrase.

Speaker adaptation using spectral interpolation for speech recognition

AbstractThis paper proposes a technique for supervised speaker adaptation for speech recognition based on a continuous density hidden Markov model (HMM). When the number of utterances for the adaptation words is decreased in order to reduce the burden on the user, the ratio of demisyllables (recognition units) not appearing in the uttered words is increased. In the proposed method, the HMM parameters for the demisyllables contained in the adaptation data are corrected and then the HMM parameters for the demisyllables not contained in the adaptation data are corrected by interpolation in the parameter space (spectral interpolation). To avoid a biased estimation of the parameters that depend on the adaptation data set, a correction is made based on large‐scale uttered data by a large number of speakers. A word recognition experiment, executed for 100 similar words, simulates a 5000‐word large‐vocabulary speech recognition; and the proposed method is then evaluated. When the recognition rate by the speaker‐independent HMM reaches 81.2 percent, the recognition rate is improved up to 85.2 percent by using 50 words for adaptation. Thus, the effectiveness of the proposed method is verified.

On speaker-independent, speaker-dependent, and speaker-adaptive speech recognition

The DARPA Resource Management task is used as a domain for investigating the performance of speaker-independent, speaker-dependent, and speaker-adaptive speech recognition. The error rate of the speaker-independent recognition system, SPHINX, was reduced substantially by incorporating between-word triphone models additional dynamic features, and sex-dependent, semicontinuous hidden Markov models. The error rate for speaker-independent recognition was 4.3%. On speaker-dependent data, the error rate was further reduced to 2.6-1.4% with 600-2400 training sentences for each speaker. Using speaker-independent models, the authors studied speaker-adaptive recognition. Both codebooks and output distributions were considered for adaptation. It was found that speaker-adaptive systems outperform both speaker-independent and speaker-dependent systems, suggesting that the most effective system is one that begins with speaker-independent training and continues to adapt to users.< <ETX xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">&gt;</ETX>

Articulatory representation and speech technology.

In this paper we demonstrate the feasibility and usefulness of articulation-based approaches in two major areas of speech technology: speech recognition and speech synthesis. Our articulatory recognition model estimates probabilities of categories of manner and place of articulation, which establish the articulatory feature vector. The transformation from the articulatory level to the symbolic level is performed by hidden Markov models or multi-layer perceptions. Evaluations show that the articulatory approach is a good basis for speaker-independent and speaker-adaptive speech recognition. We are now working on a more realistic articulatory model for speech recognition. An algorithm based on an analysis by synthesis model maps the acoustic signal to 10 articulatory parameters which describe the position of the articulators. EMA (electro-magnetic articulograph) measurements recorded at the University of Munich provide good initial estimates of tongue coordinates. In order to improve articulatory speech synthesis we investigated an accurate physical model for the generation of the glottal source with the aid of a numerical simulation. This model takes into account nonlinear vortical flow and its interaction with sound-waves. The simulation results can be used to improve the articulatory synthesis model developed by Ishizaka and Flanagan (1972).

Speaker adaptation in speech recognition using linear regression techniques

A new technique of speaker adaptation for use in speaker-independent speech recognition systems is presented. The training data is used to build models (based on linear regression) of sounds. At recognition time, the models are used together with an incomplete set of sounds from a new speaker to estimate values for unheard sounds, which are used to adapt the speaker-independent models.

Speaker-dependent-feature extraction, recognition and processing techniques

This paper discusses recent advances in and perspectives of research on speaker-dependent-feature extraction from speech waves, automatic speaker identification and verification, speaker adaptation in speech recognition, and voice conversion techniques. Speaker-dependent information exists both in the spectral envelope and in the supra-segmental features of speech. This individual information can be further classified into temporal and dynamic features. Speaker identification/verification methods can be divided into text-dependent and tect-independent methods. Although text-dependent speaker verification techniques have almost reached the level suitable for practical implementation, text-independent techniques are still in the fundamental research stage. Both supervised and unsupervised speaker adaptation algorithms for speech recognition have recently been proposed, and remarkable progress has been achieved in this field. Improving synthesized speech quality by adding natural characteristics of voice individuality, and converting synthesized voice individuality from one speaker to another, are as yet little exploited research fields to be studied in the near future. Research on speaker-dependent information is one of the most important future directions for achieving advanced speech information processing systems.