Speaker Variability Research Articles

Robust Feature Extraction for Continuous Speech Recognition Using the MVDR Spectrum Estimation Method

This paper describes a robust feature extraction technique for continuous speech recognition. Central to the technique is the minimum variance distortionless response (MVDR) method of spectrum estimation. We consider incorporating perceptual information in two ways: 1) after the MVDR power spectrum is computed and 2) directly during the MVDR spectrum estimation. We show that incorporating perceptual information directly into the spectrum estimation improves both robustness and computational efficiency significantly. We analyze the class separability and speaker variability properties of the features using a Fisher linear discriminant measure and show that these features provide better class separability and better suppression of speaker-dependent information than the widely used mel frequency cepstral coefficient (MFCC) features. We evaluate the technique on four different tasks: an in-car speech recognition task, the Aurora-2 matched task, the Wall Street Journal (WSJ) task, and the Switchboard task. The new feature extraction technique gives lower word-error-rates than the MFCC and perceptual linear prediction (PLP) feature extraction techniques in most cases. Statistical significance tests reveal that the improvement is most significant in high noise conditions. The technique thus provides improved robustness to noise without sacrificing performance in clean conditions

Nasal consonant production in Broca’s and Wernicke’s aphasics: Speech deficits and neuroanatomical correlates

The present study investigated the articulatory implementation deficits of Broca’s and Wernicke’s aphasics and their potential neuroanatomical correlates. Five Broca’s aphasics, two Wernicke’s aphasics, and four age-matched normal speakers produced consonant–vowel–(consonant) real word tokens consisting of [m, n] followed by [i, e, a, o, u]. Three acoustic measures were analyzed corresponding to different properties of articulatory implementation: murmur duration (a measure of timing), amplitude of the first harmonic at consonantal release (a measure of articulatory coordination), and murmur amplitude over time (a measure of laryngeal control). Results showed that Broca’s aphasics displayed impairments in all of these parameters, whereas Wernicke’s aphasics only exhibited greater variability in the production of two of the parameters. The lesion extent data showed that damage in either Broca’s area or the insula cortex was not predictive of the severity of the speech output impairment. Instead, lesions in the upper and lower motor face areas and the supplementary motor area resulted in the most severe implementation impairments. For the Wernicke’s aphasics, the posterior areas (superior marginal gyrus, parietal, and sensory) appear to be involved in the retrieval and encoding of lexical forms for speech production, resulting in increased variability in speech production.

Effects of training with sentences at varied speaking rates on native English speakers’ perception of Japanese vowel length

In training non-native speakers to perceive difficult second language (L2) phonetic contrasts, providing variation in speakers and phonetic contexts helps the formation of L2 categories [Pisoni and Lively, in Speech Perception and Linguistic Experience (1995), pp. 433–459. The present study tested the hypothesis that high stimulus variability in speaking rate would have positive effects for L2 learners. Native English speakers’ ability to identify Japanese vowel length was examined with three types of training: slow-only, fast-only, and slow-fast. Their perceptual abilities were measured with a pretest and posttest and compared with a control group that was not trained. Test stimuli were sentences containing Japanese disyllables spoken at three rates (slow, normal, fast). Participants were trained to identify the length of the second vowel of disyllables embedded in a sentence and they received immediate feedback. The pretest to posttest improvement of the slow-only (8.2%) and slow-fast (8.7%) training groups was significantly greater than the control group’s improvement (3.9%). However, the fast-only (7.7%) training group’s improvement did not significantly differ from the control group’s improvement. In addition, slow-fast training, unlike other training, had an advantage in performance on the fast rate stimuli. Implications for the variability theory will be discussed. [ Work supported by NSF Grant No. BCS0418246.]

Semi‐automatic labeling of pitch accents in American English

This study provides a simple classification of several pitch accents that occur in American English, based on acoustic measurements of F0. Audio samples recorded by a female speaker and transcribed in the ToBI framework (Beckman, 1994) were used to develop five rules to recognize the pitch accents H*, L*, L+H*, and L*+H. Since no information about prosodic phrasing was given to the recognizer, some combinations with phrase accents were considered and contours then grouped into equivalent labels (e.g., %HL* equivalent to H*L−). Based on the rules developed, a semi−automatic process of labeling pitch accents was implemented in Praat. Half of the audio samples were used to train the labeler, the other half to test it. The accuracy level achieved was 69%. When effects of microprosody, creaky voice, or incorrect automatic pitch tracking were removed, the accuracy of classification into equivalent labels increased to 90%. Two types of applications are proposed: semi−automatic computer labeling of intonation for larger quantities of data, and a tool to help human transcribers learn American English ToBI using Praat. In addition, the simple classification is the first step towards looking into speaker variability in realization of different contours, and typology of intonational contours in different languages.

Acoustic characteristics of vowels in three regional dialects of American English

Most of the comparative sociophonetic studies of regional dialect variation have focused on individual vowel differences across dialects as well as speaker variables. The present work seeks to define basic acoustic characteristics of entire vowel systems for three different regional variants of American English spoken in southeastern Wisconsin (affected by the Northern Cities Shift), western North Carolina (affected by the Southern Vowel Shift), and central Ohio (not considered to be affected currently by any vowel shift). Three groups of speakers (men and women) aged 20–29 years were recorded from each geographic area defined by two to three counties (creating a highly homogeneous set of speakers). Acoustic measures for the set of 14 monophthongs and diphthongs in /h—d/ context included vowel space area for each speaker, global spectral rate of change for diphthongized vowels (defined over the first three formant slopes), the amount of frequency change for F1 and F2 at two temporal points located close to vowel onset and offset (vector length), and vowel duration. These measures will establish both systemic and vowel inherent characteristics across the three dialects, serving as a basis for future examination of conditioning factors on vowels in chain shifts. Dialectal differences will be discussed. [Work supported by NIH NIDCD R01 DC006871.]

Optimizing the coverage of a speech database through a selection of representative speaker recordings

In the context of the N eologos French speech database creation project, 1 The N eologos project was funded by the French Ministry of Research in the framework of the T echnolangue program. 1 a general methodology was defined for the selection of representative speaker recordings. The selection aims at providing a good coverage in terms of speaker variability while limiting the number of recorded speakers. This is intended to make the resulting database both more adapted to the development of recently proposed multi-model methods and less expensive to collect. The presented methodology proposes a selection process based on the optimization of a quality criterion defined in a variety of speaker similarity modeling frameworks. The selection can be achieved with respect to a unique similarity criterion, using classical clustering methods such as hierarchical or K-medians clustering, or it can combine several speaker similarity criteria, thanks to a newly developed clustering method called focal speakers selection. In this framework, four different speaker similarity criteria are tested, and three different speaker clustering algorithms are compared. Results pertaining to the collection of the N eologos database are also discussed.

Recurrence quantification analysis of speech

A time-series analysis technique known as recurrence quantification analysis (RQA) holds promise as a new method for investigating the dynamical properties of the acoustic speech signal. RQA was created to deal with complex time series generated by high-dimensional, nonlinear dynamical systems. While RQA has most frequently been used for physiological studies and investigations into nonlinear dynamics, it has not been widely applied to the acoustic analysis of speech. Recurrence quantification analysis is free of many key assumptions involved in Fourier-type analyses such as data set size, stationarity, and linearity, which are routinely violated in the speech stream. This study offers some preliminary findings on the dynamic nature of speech and characteristics of speaker variability obtained by using recurrence quantification analysis. Future directions for the use of RQA in speech perception and production studies, including shadowing and imitation studies, are discussed.

Robust Formant Tracking for Continuous Speech With Speaker Variability

Several algorithms have been developed for tracking formant frequency trajectories of speech signals, however most of these algorithms are either not robust in real-life noise environments or are not suitable for real-time implementation. The algorithm presented in this paper obtains formant frequency estimates from voiced segments of continuous speech by using a time-varying adaptive filterbank to track individual formant frequencies. The formant tracker incorporates an adaptive voicing detector and a gender detector for formant extraction from continuous speech, for both male and female speakers. The algorithm has a low signal delay and provides smooth and accurate estimates for the first four formant frequencies at moderate and high signal-to-noise ratios. Thorough testing of the algorithm has shown that it is robust over a wide range of signal-to-noise ratios for various types of background noises.

Perceptual non‐intrusive speech quality assessment using a self‐organizing map

PurposeThis paper seeks to propose a new non‐intrusive method for the assessment of speech quality of voice communication systems and evaluate its performance.Design/methodology/approachThe method is based on measuring perception‐based objective auditory distances between the voiced parts of the output speech to appropriately matching references extracted from a pre‐formulated codebook. The codebook is formed by optimally clustering a large number of parametric speech vectors extracted from a database of clean speech records. The auditory distances are then mapped into equivalent subjective mean opinion scores (MOSs). The required clustering and matching processes are achieved by an efficient data‐mining tool known as the self‐organizing map (SOM). The proposed method was examined using a wide range of distortion including speech compression, wireless channel impairments, VoIP channel impairments, and modifications to the signal from features such as AGC.FindingsThe experimental results reported indicate that the proposed method provides a high level of accuracy in predicting the actual subjective quality of the speech. Specifically, the second version of the method, which is based on the use of bark spectrum (BS) analysis, is more accurate in predicting the MOS scores compared with its first and third versions (which are based on BS analysis and mel frequency cepstrum coefficients (MFCC), respectively), and outperforms the ITU‐T PESQ in a large number of test cases, particularly those related to distortion caused by channel impairments and signal level modifications.Research limitations/implicationsIt is believed that the prototype developed of the proposed objective speech quality measure is sufficiently accurate and robust against speaker, utterance and distortion type variations. Nevertheless, there are still possible directions for further improvements and enhancement. In general there are three areas that could be pursued for further improvements: widening the coverage of speaker variations of the system's codebook; formulating and using a perceptual speech model that provides true speaker‐independent representation of speech; and implementing the proposed measure as a stand‐alone system, preferably for real‐time applications.Practical implicationsBeing an output‐based method, the proposed method can be employed for monitoring and assessing telecommunications networks under both live traffic conditions and off‐line evaluation.Originality/valueThe main contribution of this paper is the introduction of a new output‐based, non‐intrusive method for the assessment of speech quality that is sufficiently accurate and robust. To the best of the author's knowledge, no reliable output‐based objective speech quality assessment method has to date been reported or formally recognised.

SpeechFind: advances in spoken document retrieval for a National Gallery of the Spoken Word

Advances in formulating spoken document retrieval for a new National Gallery of the Spoken Word (NGSW) are addressed. NGSW is the first large-scale repository of its kind, consisting of speeches, news broadcasts, and recordings from the 20th century. After presenting an overview of the audio stream content of the NGSW, with sample audio files from U.S. Presidents from 1893 to the present, an overall system diagram is proposed with a discussion of critical tasks associated with effective audio information retrieval. These include advanced audio segmentation, speech recognition model adaptation for acoustic background noise and speaker variability, and information retrieval using natural language processing for text query requests that include document and query expansion. For segmentation, a new evaluation criterion entitled fused error score (FES) is proposed, followed by application of the CompSeg segmentation scheme on DARPA Hub4 Broadcast News (30.5% relative improvement in FES) and NGSW data. Transcript generation is demonstrated for a six-decade portion of the NGSW corpus. Novel model adaptation using structure maximum likelihood eigenspace mapping shows a relative 21.7% improvement. Issues regarding copyright assessment and metadata construction are also addressed for the purposes of a sustainable audio collection of this magnitude. Advanced parameter-embedded watermarking is proposed with evaluations showing robustness to correlated noise attacks. Our experimental online system entitled SpeechFind is presented, which allows for audio retrieval from a portion of the NGSW corpus. Finally, a number of research challenges such as language modeling and lexicon for changing time periods, speaker trait and identification tracking, as well as new directions, are discussed in order to address the overall task of robust phrase searching in unrestricted audio corpora.

Intra- and inter-speaker variations of formant pattern for lateral syllables in Standard Chinese

Speech variation of speakers is a crucial issue for speaker recognition and identification, especially for forensic practice. Greater intra-speaker variation is one main reason for incorrect speaker identification in real forensic situations. Understanding the stability of acoustic parameters and their variation in speech is therefore significant for the evaluation of effective parameters for speaker identification. In this paper, all vowels in Standard Chinese including five monophthongs, eight diphthongs and four triphthongs were combined with lateral /l/. Finally, 15 lateral syllables with different tones for 10 speakers were selected and acoustically analyzed. Central frequencies of the first four formants for each syllable were measured for quantitative comparison of intra- and inter-speaker variation in order to provide a general idea of speaker variation in Standard Chinese, and finally serving for forensic application. Results show that the overall intra-speaker variation is less than the inter-speaker variation in great extent under laboratory condition though occasionally they are contrary. This supports the basis for forensic speaker identification, that is, intra-speaker variation should be less than inter-speaker variation in many acoustic features, and further validates the probability and reliability of forensic speaker identification.

Robust sound classification through the representation of similarity using response fields derived from stimuli during early experience

Models of auditory processing, particularly of speech, face many difficulties. Included in these are variability among speakers, variability in speech rate, and robustness to moderate distortions such as time compression. We constructed a system based on ensembles of feature detectors derived from fragments of an onset-sensitive sound representation. This method is based on the idea of 'spectro-temporal response fields' and uses convolution to measure the degree of similarity through time between the feature detectors and the stimulus. The output from the ensemble was used to derive segmentation cues and patterns of response, which were used to train an artificial neural network (ANN) classifier. This allowed us to estimate a lower bound for the mutual information between the class of the input and the class of the output. Our results suggest that there is significant information in the output of our system, and that this is robust with respect to the exact choice of feature set, time compression in the stimulus, and speaker variation. In addition, the robustness to time compression in the stimulus has features in common with human psychophysics. Similar experiments using feature detectors derived from fragments of non-speech sounds performed less well. This result is interesting in the light of results showing aberrant cortical development in animals exposed to impoverished auditory environments during the developmental phase.

Difference or Deficit in Speakers of African American English?

Difference or Deficit in Speakers of African American English?

Speaker Variations Influence Speechreading Speed for Dynamic Faces

We investigated the influence of task-irrelevant speaker variations on speechreading performance. In three experiments with video digitised faces presented either in dynamic, static-sequential, or static mode, participants performed speeded classifications on vowel utterances (German vowels /u/ and /i/). A Garner interference paradigm was used, in which speaker identity was task-irrelevant but could be either correlated, constant, or orthogonal to the vowel uttered. Reaction times for facial speech classifications were slowed by task-irrelevant speaker variations for dynamic stimuli. The results are discussed with reference to distributed models of face perception (Haxby et al, 2000 Trends in Cognitive Sciences 4 223-233) and the relevance of both dynamic information and speaker characteristics for speechreading.

Study of effect of speaker variability and driving conditions on the performance of an automatic speech recognition engine inside a vehicle

Spoken dialogue based information retrieval systems are being used inside vehicles. The user satisfaction of using such a system depends on how an ASR engine performs. However, the performance of an ASR is affected by speaker variability, driving conditions, etc. Here, we report the study that we performed to analyze these effects of speaker variability, different driving conditions and the effect of driving task on the ASR performance. This study consists of experimental design, data collection and systematically testing an ASR engine using this data. From the obtained results, it can be observed that (I) the ASR performance exhibits (a) significant speaker variability since the stress of driving task varies from speaker to speaker, (b) significant performance degradation across driving conditions since the noise type and level varies and (c) significant effect of driving task on recognition performance, and (II) the effect of live noise on recognition performance is not same as adding car noise to the pre-recorded speech data. The former observation is important since by just training an ASR engine on lots of speech data will not help and it is essential to include stress factors and cognition load in ASR engines to improve its performance.

Effects of Backward Speech and Speaker Variability in Language Discrimination by Rats.

Human infants use prosodic cues present in speech to extract language regularities, and it has been suggested that this capacity is anchored in more general mechanisms that are shared across mammals. This study explores the extent to which rats can generalize prosodic cues that have been extracted from a training corpus to new sentences and how this discrimination process is affected by the normalization of the sentences when multiple speakers are introduced. Conditions 1 and 2 show rats' abilities to use prosodic cues present in speech, allowing them to discriminate between sentences not previously heard. But this discrimination is not possible when sentences are played backward. Conditions 3 and 4 show that language discrimination by rats is also taxed by the process of speaker normalization. These findings have remarkable parallels with data from human adults, human newborns, and cotton-top tamarins. Implications for speech perception by humans are discussed.

Multiple views of the response of an ensemble of spectro-temporal features support concurrent classification of utterance, prosody, sex and speaker identity

Models of auditory processing, particularly of speech, face many difficulties. These difficulties include variability among speakers, variability in speech rate and robustness to moderate distortions such as time compression. In contrast to the ‘invariance of percept’ (across different speakers, of different sexes, using different intonation, and so on) is the observation that we are sensitive to the identity, sex and intonation of the speaker.In previous work we have reported that a model based on ensembles of spectro-temporal feature detectors, derived from onset sensitive pre-processing of a limited class of stimuli, preserves significant information about the stimulus class. We have also shown that this is robust with respect to the exact choice of feature set, moderate time compression in the stimulus and speaker variation. Here we extend these results to show a) that by using a classifier based on a network of spiking neurons with spike-driven plasticity, the output of the ensemble constitutes an effective rate coding representation of complex sounds; and b) that the same set of spectro-temporal features concurrently preserve information about a range of qualitatively different classes into which the stimulus might fall. We show that it is possible for multiple views of the same pattern of responses to generate different percepts. This is consistent with suggestions that multiple parallel processes exist within the auditory ‘what’ pathway with attentional modulation enhancing the task-relevant classification type. We also show that the responses of the ensemble are sparse in the sense that a small number of features respond for each stimulus type. This has implications for the ensembles' ability to generalise, and to respond differentially to a wide variety of stimulus classes.

Systematic speaker variation and within-speaker center of gravity correlations in the TIMIT database

The systematicity of speaker variation in consonants was examined by measuring the noise component of consonants in the TIMIT database. Fricatives and stops were compared by measuring at the temporal middle of fricatives and at the release of stops. There is a high correlation between many of the center of gravity measures. For example, if a speaker has a particularly high center of gravity in the sound /t/, she or he will also have a high center of gravity in /d/, /s/, /p/, /k/, and other consonants. The full set of correlations will be described in the paper. The correlations appear to stem from individual differences and not from dialect variations. The implications of the results for rapid speaker adaptation in speech recognition will be explored. [Work supported by San Jose State University Faculty Grant.]

Fifty years of progress in speech recognition

Human level speech recognition has proved to be an elusive goal because of the many sources of variability that affect speech: from stationary and dynamic noise, microphone variability, and speaker variability to variability at phonetic, prosodic, and grammatical levels. Over the past 50 years, Jim Flanagan has been a continuous source of encouragement and inspiration to the speech recognition community. While early isolated word systems primarily used acoustic knowledge, systems in the 1970s found mechanisms to represent and utilize syntactic (e.g., information retrieval) and semantic knowledge (e.g., Chess) in speech recognition systems. As vocabularies became larger, leading to greater ambiguity and perplexity, we had to explore the use task specific and context specific knowledge to reduce the branching factors. As the need arose for systems that can be used by open populations using telephone quality speech, we developed learning techniques that use very large data sets and noise adaptation methods. We still have a long way to go before we can satisfactorily handle unrehearsed spontaneous speech, speech from non-native speakers, and dynamic learning of new words, phrases, and grammatical forms.

Ear Voice Span in English into Korean Simultaneous Interpretation

A computer-aided analysis of some 800 sentences taken from English into Korean simultaneous interpretation(SI) revealed that the average ear-voice-span (EVS) is 3 seconds. Statistical analysis showed that speaker variables, including the length of original English sentence, wpm, and pauses affected the length of EVS, and the EVS, in turn, affected the interpreter variables. When the EVS is lengthened, Korean interpreters increased the speed of their target language (TL) delivery to catch up with the speaker. Long EVS had a negative effect, not only on the quality of the sentence being processed, but also on the processing of the following sentence.