Speaker Normalization Research Articles

Towards an Intelligent Acoustic Front End for Automatic Speech Recognition: Built-in Speaker Normalization

A proven method for achieving effective automatic speech recognition (ASR) due to speaker differences is to perform acoustic feature speaker normalization. More effective speaker normalization methods are needed which require limited computing resources for real-time performance. The most popular speaker normalization technique is vocal-tract length normalization (VTLN), despite the fact that it is computationally expensive. In this study, we propose a novel online VTLN algorithm entitled built-in speaker normalization (BISN), where normalization is performed on-the-fly within a newly proposed PMVDR acoustic front end. The novel algorithm aspect is that in conventional frontend processing with PMVDR and VTLN, two separating warping phases are needed; while in the proposed BISN method only one single speaker dependent warp is used to achieve both the PMVDR perceptual warp and VTLN warp simultaneously. This improved integration unifies the nonlinear warping performed in the front end and reduces simultaneously. This improved integration unifies the nonlinear warping performed in the front end and reduces computational requirements, thereby offering advantages for real-time ASR systems. Evaluations are performed for (i) an in-car extended digit recognition task, where an on-the-fly BISN implementation reduces the relative word error rate (WER) by 24%, and (ii) for a diverse noisy speech task (SPINE 2), where the relative WER improvement was 9%, both relative to the baseline speaker normalization method.

Effects of Native Language and Training on Speaker Normalization on Lexical Tone Perception: A Behavioral and ERP Study

Effects of Native Language and Training on Speaker Normalization on Lexical Tone Perception: A Behavioral and ERP Study

Improving Robustness in Frequency Warping-Based Speaker Normalization

This letter addresses the issue of frequency warping-based speaker normalization in noisy acoustic environments. Techniques are developed for improving the robustness of localized estimates of frequency warping transformations that are applied to individual observation vectors. It is shown that automatic speech recognition (ASR) performance can be improved by using speaker class-dependent distributions characterizing frequency warping transformations associated with individual hidden Markov model states. The effect of these techniques is demonstrated over a range of noise conditions on the Aurora 2 speech corpus.

Speaker normalization using cortical strip maps: A neural model for steady state vowel identification

Auditory signals of speech are speaker-dependent, but representations of language meaning are much more speaker-independent. Such a speaker normalization transformation enables speech to be learned and understood from different speakers. A neural model is presented that performs speaker normalization to generate a pitch-independent representation of speech sounds, while also preserving information about speaker identity. This speaker-invariant representation is categorized into unitized speech items, which input to sequential working memories whose distributed patterns can be categorized, or chunked, into syllable and word representations. The proposed model circuits fit into an emerging theory of auditory streaming and speech categorization in which auditory streaming and speaker normalization both use similar neural designs; namely, multiple cortical strip map representations of auditory signals. This design homology may clarify how speaker normalization circuits evolved from more primitive streaming mechanisms. Simulations with synthesized steady-state vowels from the Peterson and Barney (1952) vowel database achieve accuracy rates similar to those achieved by human listeners. These results are compared to behavioral data and other speaker normalization models. [Work supported in part by NSF and ONR.]

Speaker normalization using cortical strip maps: A neural model for steady-state vowel categorization

Auditory signals of speech are speaker dependent, but representations of language meaning are speaker independent. The transformation from speaker-dependent to speaker-independent language representations enables speech to be learned and understood from different speakers. A neural model is presented that performs speaker normalization to generate a pitch-independent representation of speech sounds, while also preserving information about speaker identity. This speaker-invariant representation is categorized into unitized speech items, which input to sequential working memories whose distributed patterns can be categorized, or chunked, into syllable and word representations. The proposed model fits into an emerging model of auditory streaming and speech categorization. The auditory streaming and speaker normalization parts of the model both use multiple strip representations and asymmetric competitive circuits, thereby suggesting that these two circuits arose from similar neural designs. The normalized speech items are rapidly categorized and stably remembered by adaptive resonance theory circuits. Simulations use synthesized steady-state vowels from the Peterson and Barney [Peterson, G. E., and Barney, H.L., J. Acoust. Soc. Am. 24, 175-184 (1952).] vowel database and achieve accuracy rates similar to those achieved by human listeners. These results are compared to behavioral data and other speaker normalization models.

Speaker Adaptation With Limited Data Using Regression-Tree-Based Spectral Peak Alignment

Spectral mismatch between training and testing utterances can cause significant degradation in the performance of automatic speech recognition (ASR) systems. Speaker adaptation and speaker normalization techniques are usually applied to address this issue. One way to reduce spectral mismatch is to reshape the spectrum by aligning corresponding formant peaks. There are various levels of mismatch in formant structures. In this paper, regression-tree-based phoneme- and state-level spectral peak alignment is proposed for rapid speaker adaptation using linearization of the vocal tract length normalization (VTLN) technique. This method is investigated in a maximum-likelihood linear regression (MLLR)-like framework, taking advantage of both the efficiency of frequency warping (VTLN) and the reliability of statistical estimations (MLLR). Two different regression classes are investigated: one based on phonetic classes (using combined knowledge and data-driven techniques) and the other based on Gaussian mixture classes. Compared to MLLR, VTLN, and global peak alignment, improved performance can be obtained for both supervised and unsupervised adaptations for both medium vocabulary (the RM1 database) and connected digits recognition (the TIDIGITS database) tasks. Performance improvements are largest with limited adaptation data which is often the case for ASR applications, and these improvements are shown to be statistically significant.

A shift-based approach to speaker normalization using non-linear frequency-scaling model

In this work, we present a speaker-normalization method based on the idea that the speaker-dependent scale-factor can be separated out as a fixed translation factor in an alternate domain. We also introduce a non-linear frequency-scaling model motivated by the analysis of speech data. The proposed shift-based normalization approach is implemented using a maximum-likelihood (ML) search for the translation factor in the alternate domain. The advantage of our approach is that we are able to show the relationship between conventional frequency-warping based vocal-tract length normalization (VTLN) methods and the methods based on shifts in psycho-acoustic scale thus providing a unifying frame-work for speaker-normalization. Additionally, in our approach it is simple to show that the shifting required for normalization can be expressed as a linear transformation in the cepstral domain. This is important for computational efficiency since we do not have to recompute the features by re-doing the signal processing for each scale/translation factor as is usually done in conventional normalization. We present recognition results using our proposed approach on a digit recognition task and show that the non-linear scaling model provides relative improvement of 4% for adults and 7.5% for children when compared to the linear-scaling model.

Effects of native language and training on speaker normalization in lexical tone perception

Tone languages, such as Thai, use pitch differences to distinguish lexical meaning. In this study, we investigated the effects of native language and training on speaker normalization in perception of lexical tones in Thai. Three tokens each of the Thai high, mid, and low tones were generated to simulate three female speakers with three relatively different F0 pitch ranges (high, mid, and low). The stimuli were presented to native Mandarin Chinese (NC), American English (NE), and Thai (NT) for categorization before and after a two‐day perceptual training. Results obtained thus far from 32 NC, 30 NE, and 11 NT suggested that, overall: (a) NC outperformed NE both before and after training, (b) NT numerically outperformed NC before training, but not after training, and (c) performance of all three groups of participants improved after training. Interestingly, it was also found that participants’ performance varied from tone to tone, depending on the tonal environment in which it was presented for discrimination. These results suggest that knowledge of a tonal language and perceptual training facilitates the speaker normalization process in lexical tone perception, and that tonal contexts affect discrimination of tones.

Vocal tract normalization equals linear transformation in cepstral space

Vocal tract normalization (VTN) is a widely used speaker normalization technique which reduces the effect of different lengths of the human vocal tract and results in an improved recognition accuracy of automatic speech recognition systems. We show that VTN results in a linear transformation in the cepstral domain, which so far have been considered as independent approaches of speaker normalization. We are now able to compute the Jacobian determinant of the transformation matrix, which allows the normalization of the probability distributions used in speaker-normalization for automatic speech recognition. We show that VTN can be viewed as a special case of Maximum Likelihood Linear Regression (MLLR). Consequently, we can explain previous experimental results that improvements obtained by VTN and subsequent MLLR are not additive in some cases. For three typical warping functions the transformation matrix is calculated analytically and we show that the matrices are diagonal dominant and thus can be approximated by quindiagonal matrices.

화자독립 음성인식을 위한 GMM 기반 화자 정규화

화자독립 음성인식기의 화자 정규화를 위해 GMM(Gaussian mixture model)분포를 이용하는 방법에 대해 실험한다. 이 방법은 벡터 양자화기를 이용한 선행 연구를 개선한 것으로, 정규화된 학습용 특징벡터들의 확률분포를 최적의 클러스터의 수를 갖는 GMM분포로 모델링한 다음, 이 분포를 이용하여 시험용화자의 워핑계수를 추정한다. 이 연구의 목적은 기존의 ML을 이용한 방법의 단점을 개선하는 동시에 벡터 양자화기를 이용한 선행연구와'soft decision'이라 불리는 확률 분포를 이용한 방법의 성능을 비교하는데 있다. TIMIT 코퍼스를 대상으로 한 음소 인식 실험에서 클러스터의 수를 적절한 크기로 설정한 GMM분포를 이용함으로써 벡터 양자화기를 이용한 방법에 비해 약간 나은 인식률을 얻을 수 있었다. For the purpose of speaker normalization in speaker independent speech recognition systems, experiments are conducted on a method based on Gaussian mixture model(GMM). The method, which is an improvement of the previous study based on vector quantizer, consists of modeling the probability distribution of canonical feature vectors by a GMM with an appropriate number of clusters, and of estimating the warp factor of a test speaker by making use of the obtained probabilistic model. The purpose of this study is twofold: improving the existing ML based methods, and comparing the performance of what is called 'soft decision' method with that of the previous study based on vector quantizer. The effectiveness of the proposed method is investigated by recognition experiments on the TIMIT corpus. The experimental results showed that a little improvement could be obtained tv adjusting the number of clusters in GMM appropriately.

Speaker Normalization for Chinese Vowel Recognition in Cochlear Implants

Because of the limited spectra-temporal resolution associated with cochlear implants, implant patients often have greater difficulty with multitalker speech recognition. The present study investigated whether multitalker speech recognition can be improved by applying speaker normalization techniques to cochlear implant speech processing. Multitalker Chinese vowel recognition was tested with normal-hearing Chinese-speaking subjects listening to a 4-channel cochlear implant simulation, with and without speaker normalization. For each subject, speaker normalization was referenced to the speaker that produced the best recognition performance under conditions without speaker normalization. To match the remaining speakers to this "optimal" output pattern, the overall frequency range of the analysis filter bank was adjusted for each speaker according to the ratio of the mean third formant frequency values between the specific speaker and the reference speaker. Results showed that speaker normalization provided a small but significant improvement in subjects' overall recognition performance. After speaker normalization, subjects' patterns of recognition performance across speakers changed, demonstrating the potential for speaker-dependent effects with the proposed normalization technique.

Improved automatic speech recognition through speaker normalization

In this paper, speaker adaptive acoustic modeling is investigated by using a novel method for speaker normalization and a well known vocal tract length normalization method. With the novel normalization method, acoustic observations of training and testing speakers are mapped into a normalized acoustic space through speaker-specific transformations with the aim of reducing inter-speaker acoustic variability. For each speaker, an affine transformation is estimated with the goal of reducing the mismatch between the acoustic data of the speaker and a set of target hidden Markov models. This transformation is estimated through constrained maximum likelihood linear regression and then applied to map the acoustic observations of the speaker into the normalized acoustic space. Recognition experiments made use of two corpora, the first one consisting of adults’ speech, the second one consisting of children’s speech. Performing training and recognition with normalized data resulted in a consistent reduction of the word error rate with respect to the baseline systems trained on unnormalized data. In addition, the novel method always performed better than the reference vocal tract length normalization method adopted in this work. When unsupervised static speaker adaptation was applied in combination with each of the two speaker normalization methods, a different behavior was observed on the two corpora: in one case performance became very similar while in the other case the difference remained significant.

Reliable but weak voice-formant cues to body size in men but not women

Whether voice formants provide reliable cues to adult body size has been contested recently for some animals and humans and the outcome bears critically on theories of social competition and mate choice, language origins, and speaker normalization. We report two experiments to test listeners’ ability to assess speaker body size. In Experiment 1, listeners heard paired comparisons of the same short phrase spoken by two adults of the same sex paired randomly with respect to height and indicated which was larger. Both sexes (M=20; F=22) showed an equal but modest ability to identify the larger male (mean correct=58.5%; T=31.5, P<0.001) that correlated with the magnitude of their height difference but could not pick the larger female (mean correct=52.0%; T=1.05, P=0.305) regardless of the height difference. Experiment 2 used single word comparisons, focused only on male voices, and controlled F0 while manipulating F1−F4 between speakers. When F0 was equal but F1−F4 predicted the height difference between speakers, both sexes (M=12; F=18) correctly chose the taller male (80%). When F1−F4 values of the shorter male were reduced below those of the taller male (or vice versa), subjects shifted to pick the shorter male as being larger.

Effects of speaker normalization based on vocal tract length ratios on word recognition using compound parameters

Effects of speaker normalization based on vocal tract length ratios on word recognition using compound parameters

Effects of Backward Speech and Speaker Variability in Language Discrimination by Rats.

Human infants use prosodic cues present in speech to extract language regularities, and it has been suggested that this capacity is anchored in more general mechanisms that are shared across mammals. This study explores the extent to which rats can generalize prosodic cues that have been extracted from a training corpus to new sentences and how this discrimination process is affected by the normalization of the sentences when multiple speakers are introduced. Conditions 1 and 2 show rats' abilities to use prosodic cues present in speech, allowing them to discriminate between sentences not previously heard. But this discrimination is not possible when sentences are played backward. Conditions 3 and 4 show that language discrimination by rats is also taxed by the process of speaker normalization. These findings have remarkable parallels with data from human adults, human newborns, and cotton-top tamarins. Implications for speech perception by humans are discussed.

Computer generated feedback on vowel production by learners of English as a second language

The acquisition of new vowel sounds is one of the most challenging tasks for the adult learner of a second language. It is also an aspect which requires a great deal of time and individual attention. Learners of English are particularly hampered by the irregular orthographic representation of vowel sounds. This paper reports on research in progress into the development of an application to provide learners with real time feedback on their vowel production. This process includes the calculation and plotting of formants on a graph using the PRAAT programme. The patterns which result are highly reminiscent of the traditional vowel chart, which in turn reflects articulation, i.e. articulatory changes are reflected on the graph. This application can provide feedback on both pure vowels and diphthongs. Some of the difficulties and obstacles which as yet prevent it from being a tool which can be used by the learner autonomously will also be outlined and discussed. Foremost amongst which are: the question of speaker normalisation (how to compensate for the difference in values between male, female and pre-pubertal speakers) which may be resolved by way of statistical elaboration of the formant data; and the methods used to isolate the vowel sound from the rest of the signal.

Robust recognition of children's speech

Developmental changes in speech production introduce age-dependent spectral and temporal variability in the speech signal produced by children. Such variabilities pose challenges for robust automatic recognition of children's speech. Through an analysis of age-related acoustic characteristics of children's speech in the context of automatic speech recognition (ASR), effects such as frequency scaling of spectral envelope parameters are demonstrated. Recognition experiments using acoustic models trained from adult speech and tested against speech from children of various ages clearly show performance degradation with decreasing age. On average, the word error rates are two to five times worse for children speech than for adult speech. Various techniques for improving ASR performance on children's speech are reported. A speaker normalization algorithm that combines frequency warping and model transformation is shown to reduce acoustic variability and significantly improve ASR performance for children speakers (by 25-45% under various model training and testing conditions). The use of age-dependent acoustic models further reduces word error rate by 10%. The potential of using piece-wise linear and phoneme-dependent frequency warping algorithms for reducing the variability in the acoustic feature space of children is also investigated.

Speaker adaptation with all-pass transforms

Modern speech recognition systems are based on the hidden Markov model (HMM) and employ cepstral features to represent input speech. In speaker normalization, the cepstral features of speech from a given speaker are transformed to match the speaker independent HMM. In speaker adaptation, the means of the HMM are transformed to match the input speech. Vocal tract length normalization (VTLN) is a popular normalization scheme wherein the frequency axis of the short-time spectrum is rescaled prior to the extraction of cepstral features. In this work, we develop novel speaker adaptation schemes by exploiting the fact that frequency domain transformations similar to that inherent in VTLN can be accomplished entirely in the cepstral domain through the use of conformal maps. We describe two classes of such maps: rational all-pass transforms (RAPTs) which are well-known in the signal processing literature, and sine-log all-pass transforms (SLAPTs) which are novel in this work. For both classes of maps, we develop the relations necessary to perform maximum likelihood estimation of the relevant transform parameters using enrollment data from a new speaker. We also propose the means by which an HMM may be trained specifically for use with this type of adaptation. Finally, in a set of recognition experiments conducted on conversational speech material from the Switchboard Corpus as well as the English Spontaneous Scheduling Task, we demonstrate the capacity of APT-based speaker adaptation to achieve word error rate reductions superior to those obtained with other popular adaptation techniques, and moreover, reductions that are additive with those provided by VTLN.

VTLN Through Frequency Warping Based on Pitch

This article describes a Vocal Tract Length Nor­ malization (VTLN) procedure through frequency warping based on pitch estimates. This procedure aims to reduce the inter-speaker variability of speech signals in order to obtain a robust automatic speech recognition system. Two additional methods are also described: one for reducing the environment variability and another for compensating the coarticulation effects on connected word pronunciation. En­ vironment variability is compensated by explicitly modeling some frequent noise phenomena. Coarticulation phenomena compensation reduces speech signal variability by modeling events that result from coarticulation between adjacent mod­ els. Inter-speaker variability removal is performed by a traditional speaker normalization method, which consists in expanding or compressing the Mel filterbank bandwidths, in order to normalize the Vocal Tract Length (VTL) of each speaker. Most of the existing methods for VTL estimation are based on formant estimation, but the difficulty of formant estimation is a known performance limitation. The proposed method over­ comes such a problem since it estimates the warping factor through pitch. The recognition results, obtained for a tele­ phone digit recognition task (with phones and sub words as units), prove that this procedure leads to similar improve­ ments to those obtained with traditional methods based on formant estimates, actually outperforming them in some sit­ uations.

Model‐based speaker normalization methods for speech recognition

AbstractA speaker normalization method using a speech generation model is proposed in order to achieve high‐performance speaker adaptation with a small amount of adaptation data. The speaker‐ and phoneme‐dependent vocal tract area function is approximated by the corresponding area function produced by the articulatory model of a standard speaker, combined with phoneme‐independent feature quantities of the vocal‐tract shape of the normalized target speaker as estimated from the formant frequencies of two vowels. The frequency warping functions are determined from the formant frequencies of speech calculated from the vocal‐tract area functions thus obtained, and normalization of the uttered speech is performed by stretching the speech spectrum in the frequency‐axis direction. Continuous phoneme recognition experiments using phoneme connection rules show that the recognition error using a gender‐dependent model is reduced by about 30% in the proposed method and that recognition performance superior to that of vocal‐tract length normalization is obtained. The recognition performance of the proposed method is also equivalent to that of speaker adaptation by moving vector field smoothing (VFS) using 10 phonetically balanced sentences, showing that high‐performance speaker adaptation using a small amount of adaptation data can be achieved by the proposed method. © 2003 Wiley Periodicals, Inc. Electron Comm Jpn Pt 2, 86(2): 45–56, 2003; Published online in Wiley InterScience (www.interscience.wiley.com). DOI 10.1002/ecjb.10119