Intra-speaker Variability Research Articles

Hypernetworks for Personalizing ASR to Atypical Speech

Abstract Parameter-efficient fine-tuning (PEFT) for personalizing automatic speech recognition (ASR) has recently shown promise for adapting general population models to atypical speech. However, these approaches assume a priori knowledge of the atypical speech disorder being adapted for—the diagnosis of which requires expert knowledge that is not always available. Even given this knowledge, data scarcity and high inter-/intra-speaker variability further limit the effectiveness of traditional fine-tuning. To circumvent these challenges, we first identify the minimal set of model parameters required for ASR adaptation. Our analysis of each individual parameter’s effect on adaptation performance allows us to reduce Word Error Rate (WER) by half while adapting 0.03% of all weights. Alleviating the need for cohort-specific models, we next propose the novel use of a meta-learned hypernetwork to generate highly individualized, utterance-level adaptations on-the-fly for a diverse set of atypical speech characteristics. Evaluating adaptation at the global, cohort, and individual-level, we show that hypernetworks generalize better to out-of-distribution speakers, while maintaining an overall relative WER reduction of 75.2% using 0.1% of the full parameter budget.

Bɪt] by [bɪʔ]: Variation in T-glottaling in Scottish Standard English

Abstract The present study investigates internal and external constraints conditioning variable T-glottaling, the realization of the voiceless alveolar stop /t/ as a glottal stop [ʔ], in supraregional Scottish Standard English. Drawing on phonemically annotated speech data from the Scottish component of the International Corpus of English, a total of 12,162 /t/ tokens produced by 138 speakers were extracted from eight formal speaking categories in the corpus and analyzed auditorily. The results showed that about 28% of the analyzed /t/ tokens were produced as glottal stops, with significant inter- and intra-speaker variability. The realization of T-glottaling is subject to both linguistic (phonetic context and word type) and social factors (age, gender, and speech style). Moreover, patterns of various types of T-glottaling differ from each other and constitute distinctive processes of ongoing sound change in Scotland.

Realisation, elision and intrusion of /h/ in French learners of English

As the phonological systems of English and French diverge both at the segmental level and at the prosodic level, French learners encounter difficulties with English pronunciation. One example is the glottal fricative /h/ since it is absent from the French phonemic inventory–with the exception of the onomatopoeia “haha” to express laughter. Following studies by Exare (2017) and Kamiyama et al. (2011), this paper aims to provide further empirical evidence of the realisation, elision, and intrusion of /h/ in French speakers’ English productions. The learners were university students who did not major in English. The participants were given three tasks, i.e., word reading, sentence reading, and spontaneous speech. Auditory analyses were then carried out by the researcher. Results show four speaker-profiles: (1) /h/ is absent altogether; (2) /h/ is accurately realised; (3) /h/ is accurately realised in addition to regular occurrences of intrusive /h/; (4) only intrusive /h/ occurs. Although the speaking task has an influence on intra-speaker variability, intra-speaker consistency has been observed, with participants belonging to one of the above learner-profiles. Also of interest were the differences between men and women. These results suggest that different teaching strategies should be established to better suit the needs of the different profiles.

Vowel Production in Children and Adults With Down Syndrome: Fundamental and Formant Frequencies of the Corner Vowels.

Atypical vowel production contributes to reduced speech intelligibility in children and adults with Down syndrome (DS). This study compares the acoustic data of the corner vowels /i/, /u/, /æ/, and /ɑ/ from speakers with DS against typically developing/developed (TD) speakers. Measurements of the fundamental frequency (f o) and first four formant frequencies (F1-F4) were obtained from single word recordings containing the target vowels from 81 participants with DS (ages 3-54 years) and 293 TD speakers (ages 4-92 years), all native speakers of English. The data were used to construct developmental trajectories and to determine interspeaker and intraspeaker variability. Trajectories for DS differed from TD based on age and sex, but the groups were similar with the striking change in f o and F1-F4 frequencies around age 10 years. Findings confirm higher f o in DS, and vowel-specific differences between DS and TD in F1 and F2 frequencies, but not F3 and F4. The measure of F2 differences of front-versus-back vowels was more sensitive of compression than reduced vowel space area/centralization across age and sex. Low vowels had more pronounced F2 compression as related to reduced speech intelligibility. Intraspeaker variability was significantly greater for DS than TD for nearly all frequency values across age. Vowel production differences between DS and TD are age- and sex-specific, which helps explain contradictory results in previous studies. Increased intraspeaker variability across age in DS confirms the presence of a persisting motor speech disorder. Atypical vowel production in DS is common and related to dysmorphology, delayed development, and disordered motor control.

Timbre features with MEDIAN values for compensating intra-speaker variability in speaker identification of whispering sound

Timbre features with MEDIAN values for compensating intra-speaker variability in speaker identification of whispering sound

Three Kinds of Rising-Falling Contours in German wh-Questions: Evidence From Form and Function

The intonational realization of utterances is generally characterized by regional as well as inter- and intra-speaker variability in f0. Category boundaries thus remain “fuzzy” and it is non-trivial how the (continuous) acoustic space maps onto (discrete) pitch accent categories. We focus on three types of rising-falling contours, which differ in the alignment of L(ow) and H(igh) tones with respect to the stressed syllable. Most of the intonational systems on German have described two rising accent categories, e.g., L+H*and L*+H in the German ToBI system. L+H*has a high-pitched stressed syllable and a low leading tone aligned in the pre-tonic syllable; L*+H a low-pitched stressed syllable and a high trailing tone in the post-tonic syllable. There are indications for the existence of a third category which lies between these two categories, with both L and H aligned within the stressed syllable, henceforth termed (LH)*. In the present paper, we empirically investigate the distinctiveness of three rising-falling contours [L+H*, (LH)*, and L*+H, all with a subsequent low boundary tone] in Germanwh-questions. We employ an approach that addresses both theformand thefunctionof the contours, also takingregional variationinto account. In Experiment 1 (form), we used a delayed imitation paradigm to test whether Northern and Southern German speakers can imitate the three rising-falling contours inwh-questions as distinct contours. In Experiment 2 (function), we used a free association task to investigate whether listeners interpret the pragmatic meaning of the three contours differently. Imitation results showed that German speakers—both from the North and the South—reproduced the three contours. There was a small but significant effect of regional variety such that contours produced by speakers from the North were slightly more distinct than those by speakers from the South. In the association task, listeners from both varieties attributed distinct meanings to the (LH)*accent as opposed to the two ToBI accents L+H*and L*+H. Combined evidence fromformandfunctionsuggests that three distinct contours can be found in the acoustic and perceptual space of German rising-falling contours.

Picture Description in the Assessment of Connected Speech Intelligibility in Parkinson’s Disease: A Pilot Study

Introduction: Assessment of intelligibility in dysarthria tends to rely on oral reading of sentences or words. However, self-generated utterances are closer to a client's natural speech. This study investigated how transcription of utterances elicited by picture description can be used in the assessment of intelligibility in speakers with Parkinson’s disease. Methods: Speech samples from eleven speakers with Parkinson’s disease and six neurologically healthy persons were audio-recorded. Forty-two naive listeners completed transcriptions of self-generated sentences from a picture description task and orally read sentences from the Swedish Test of Intelligibility, as well as scaled ratings of narrative speech samples. Results: Intelligibility was higher in orally read than self-generated sentences and higher for content words than for the whole sentence in self-generated sentences for most of the speakers, although these within-group differences were not statistically significant at the group level. Adding contextual leads for the listeners increased intelligibility in self-generated utterances significantly but with individual variation. Although correlations between the intelligibility measures were at least moderate or strong, there was a considerable inter- and intra-speaker variability in intelligibility scores between tasks for the speakers with Parkinson’s disease, indicating individual variation of factors that impact intelligibility. Intelligibility scores from neurologically healthy speakers were generally high across tasks with no significant differences between the conditions. Conclusion: Within-speaker variability supports literature recommendations to use multiple methods and tasks when assessing intelligibility. The inclusion of transcription of self-generated utterances elicited by picture description to the intelligibility assessment has the potential to provide additional information to assessment methods based on oral reading of pre-scripted sentences and to inform the planning of interventions.

Theta activity and phase resetting during perception of French homophonous utterances

ABSTRACT Speech perception involves segmenting a continuous stream of speech into its word components. This can be challenging in the case of homophonous utterances only differing in non-contrastive subphonemic features. Yet, the speech perception system seems able to discriminate subphonemic deviation in homophonous utterances, since it has been shown to elicit a mismatch response (MMN). Here, we focused on the oscillatory correlates, namely phase resetting and power, of non-contrastive subphonemic deviation processing in language. An oddball task that considered natural intraspeaker variability was used. Subphonemic deviance elicited intertrial phase coherence (ITC) differences in the theta band at Fz during the time window of the MMN. No differences in power were found. This suggests that the processing of subphonemic deviation in speech signals, reflected by the MMN, might rely on mechanisms of phase resetting. ITC might facilitate the synchronous firing of functional networks involved in the processing of subphonemic deviance.

Automatic speaker verification from affective speech using Gaussian mixture model based estimation of neutral speech characteristics

Intra-speaker variability, caused by emotional speech, is a real threat to the performance of speaker recognition systems. In fact, as human beings, we are constantly changing our emotional state. While many efforts have been made to increase automatic speaker verification (ASV) robustness towards channel effects or spoofing attacks, only a handful of studies have addressed the detrimental consequences of affective speech. In this paper, we propose a new method to minimize the mismatch between neutral and affective speech. To this end, a Gaussian mixture model is used to learn a prior probability distribution of the neutral speech for a given speaker (i.e., characterizing his/her source space). This knowledge is then used to minimize the differences between target (affective) and source (neutral) spaces. The proposed method is validated across four multi-lingual emotional datasets. Experimental results show a consistent improvement in performance across eight emotional states, with significant reductions of equal error rate relative to the baseline.

Compensation Techniques for Speaker Variability in Continuous Emotion Prediction

Continuous time-varying prediction of emotions based on speech in terms of attributes (i.e., arousal) has received considerable attention in the past few years. However, the variability introduced by factors not related to emotion, such as speaker and phonetic variability, which in turn may lead to less reliable models and less accurate emotion predictions, has not been fully explored yet. In particular, even though speaker variability has been shown to be a significant confounding factor in continuous emotion prediction systems, there remains a paucity of analyses about how speaker variability affects continuous emotion prediction systems and which methods can be applied to compensate for this variability. This paper first formulates speaker variability systematically in terms of probability distributions in both feature and model spaces, and quantifies the effect of speaker variability by comparing inter- and intra-speaker variability between speaker-dependent models. Second, two compensation techniques based on partial least squares dimensional reduction and feature mapping are proposed. Finally, the effectiveness of the proposed techniques is validated on three databases, across which they show consistent improvement in arousal, valence and dominance prediction. Additional quantitative analyse reveals that the two proposed techniques compensate for speaker variability in both the feature and model spaces simultaneously.

An Acoustic Analysis of Fluctuations for Inter and Intra Speaker Variability in Speech Sounds

An Acoustic Analysis of Fluctuations for Inter and Intra Speaker Variability in Speech Sounds

An Acoustic Analysis of Fluctuations for Inter and Intra Speaker Variability in Speech Sounds

An Acoustic Analysis of Fluctuations for Inter and Intra Speaker Variability in Speech Sounds

Code-switching alone cannot explain intraspeaker syntactic variability: Evidence from a spoken elicitation experiment

AbstractWe address the question whether speakers activate different grammars when they encounter linguistic input from different registers, here written standardised language and spoken dialect. This question feeds into the larger theoretical and empirical question if variable syntactic patterns should be modelled as switching between different registers/grammars, or as underspecified mappings from form to meaning within one grammar. We analyse 6000 observations from 26 high school students from Tromsø, comprising more than 20 phonological, morphological, lexical and syntactic variables obtained from two elicited production experiments: one using standardised written language and one using spoken dialect as the elicitation source. The results suggest that most participants directly activate morphophonological forms from the local dialect when encountering standardised orthographic forms, suggesting that they do not treat the written and spoken language as different grammars. Furthermore, the syntactic variation does not track the morphophonological variation, which suggests that code/register-switching alone cannot explain syntactic optionality.

Acoustic Variability of Voice Signal as Factor of Information Security for Automatic Speech Recognition Systems with Tuning to User Voice

The phenomenon of the voice signal acoustic variability in automatic speech recognition systems is considered. There are two varieties—intra- and inter-speaker speech variability. The probabilistic cluster model of minimal speech units in the Kullback–Leibler information metric is used for their mathematical description and comparison in magnitude. On its basis, theoretical estimates of the voice signal acoustic variability for each of its varieties are obtained separately. The effect of information security in systems with tuning to the authorized user voice is described and quantitatively characterized. The intra-speaker variability is negligible in comparison with the inter-speaker variability of speech, and therefore does not have a noticeable harmful effect on the effectiveness of automatic speech recognition. The computational experiment is set up to confirm and develop the theoretical research results, where two speech streams from two different speakers are considered. The author’s software is used for its implementation. According to the experimental results we find that the level of inter-speaker speech variability in a number of cases goes beyond the inter-phonemic differences within a homogeneous speech flow. Therefore, in systems with tuning to the speaker voice, the effect of voice signal acoustic variability is not only unambiguously generally positive, namely: it is an information protection from unauthorized access, but also it is significant in terms of probability-theoretic relation. The obtained results are intended for the development of new and modernization of existing systems for automatic speech recognition, designed to work in a standalone mode.

Syllable shape effects on temporal aspects of the Danish stød

Danish phonologically contrasts modal phonation with “stød,” a type of nonmodal phonation related to creaky voice (Fischer-Jørgensen, 1989). Stød (?) is licensed in a syllable by either a long vowel or a vowel + sonorant sequence (CV:?(Obstruent/Sonorant), CVS?). Previous research noting high inter- and intra-speaker variability in stød realization has led researchers to claim that temporal aspects of stød production are random and not under speakers' control (Grønnum & Basbøll, 2007). In contrast, this study provides novel evidence that, overall, stød onset is timed with respect to syllables' rime midpoint and stød offset with respect to sonorant rime offset in monosyllabic words, though differences based on rime content persist across word types. 10 native Danish speakers were recorded reading sentences with embedded monosyllabic, stød-bearing words (CV:?(O/S), CVS?), and stød durations were marked. Comparison of different measures of stød onset (lag from either rime onset, rime midpoint, or vowel midpoint) revealed that lag from rime midpoint minimized variance in stød onset timing across word types. Furthermore, CV:?O/S words had earlier onset/offset times than CVS? words, which had earlier onset times than CV:? words. Together, these results indicate that stød production across different word types is not random but in fact modulated by syllable shape.

Linguistic Diversity in Appalachia

This study investigates how American English speakers from within and outside the Appalachian region interpret negative auxiliary inversion (NAI). Previously observed in Appalachian and other English varieties, NAI has surface syntax similar to yes-no questions but receives a declarative interpretation (e.g., Didn’t everybody watch Superbowl 53, meaning ‘not everybody watched’). Previous work shows that NAI is associated with a reading in which some but not all people participated in an event, as opposed to one in which no one participated. Results from an interpretation task revealed that Appalachian participants tended to obtain the ‘not all’ and not the ‘no one’ reading for NAI. In contrast, non-Appalachian participants’ interpretations exhibited greater inter- and intraspeaker variability. Appalachian participants with more ‘not all’ interpretations reported positive attitudes toward NAI use, and they also distinguished between attested and unattested syntactic subject types (e.g., everybody, many people, *few people) in a naturalness rating task. Appalachian participants with more ‘no one’ interpretations had more negative attitudes toward NAI use and made no distinction between subject types. These results highlight how individuals from Appalachia interpret NAI differently than individuals from outside the region and suggest that language attitudes may impact semantic interpretation within a nonmainstream speaker group.

Lexical frequency and morphological regularity as sources of heritage speaker variability in the acquisition of mood

The present investigation examines intra-speaker variability in heritage speakers (HSs) of Spanish by focusing on the potential effects of two variables in their acquisition of mood: lexical frequency and morphological regularity. To do so, this study tested participants’ interpretation and use of early-acquired mood alternations conveying either assertive or jussive meanings. Results from a truth-value judgment and a picture-based sentence completion task revealed that HSs’ performance was significantly modulated by the lexical frequency of the matrix verb introducing the modal alternation, as well as by the regularity of the embedded verbal form. In particular, frequent matrix verbs and irregular forms yielded higher rates of accuracy across most of the experimental tasks, suggesting that the degree of variability in HSs’ outcomes is determined, to a certain extent, by these two factors. These results favor an account of heritage language acquisition and loss focused on the effects of lexical activation on bilinguals’ grammatical representations (Perez-Cortes, Putnam and Sánchez, 2019; Putnam and Sánchez, 2013).

Automatic vocal tract landmark localization from midsagittal MRI data

The various speech sounds of a language are obtained by varying the shape and position of the articulators surrounding the vocal tract. Analyzing their variations is crucial for understanding speech production, diagnosing speech disorders and planning therapy. Identifying key anatomical landmarks of these structures on medical images is a pre-requisite for any quantitative analysis and the rising amount of data generated in the field calls for an automatic solution. The challenge lies in the high inter- and intra-speaker variability, the mutual interaction between the articulators and the moderate quality of the images. This study addresses this issue for the first time and tackles it by means of Deep Learning. It proposes a dedicated network architecture named Flat-net and its performance are evaluated and compared with eleven state-of-the-art methods from the literature. The dataset contains midsagittal anatomical Magnetic Resonance Images for 9 speakers sustaining 62 articulations with 21 annotated anatomical landmarks per image. Results show that the Flat-net approach outperforms the former methods, leading to an overall Root Mean Square Error of 3.6 pixels/0.36 cm obtained in a leave-one-out procedure over the speakers. The implementation codes are also shared publicly on GitHub.

Corner vowels in males and females ages 4 to 20 years: Fundamental and F1-F4 formant frequencies.

The purpose of this study was to determine the developmental trajectory of the four corner vowels' fundamental frequency (fo) and the first four formant frequencies (F1-F4), and to assess when speaker-sex differences emerge. Five words per vowel, two of which were produced twice, were analyzed for fo and estimates of the first four formants frequencies from 190 (97 female, 93 male) typically developing speakers ages 4-20 years old. Findings revealed developmental trajectories with decreasing values of fo and formant frequencies. Sex differences in fo emerged at age 7. The decrease of fo was larger in males than females with a marked drop during puberty. Sex differences in formant frequencies appeared at the earliest age under study and varied with vowel and formant. Generally, the higher formants (F3-F4) were sensitive to sex differences. Inter- and intra-speaker variability declined with age but had somewhat different patterns, likely reflective of maturing motor control that interacts with the changing anatomy. This study reports a source of developmental normative data on fo and the first four formants in both sexes. The different developmental patterns in the first four formants and vowel-formant interactions in sex differences likely point to anatomic factors, although speech-learning phenomena cannot be discounted.

A real-time magnetic resonance imaging study of cross-speaker variability in the production of /ɹ/

There is a longstanding interest in understanding the production mechanisms of the American English (AE) rhotic, /ɹ/ especially given both the wide intraspeaker variability and interspeaker heterogeneity in its articulation, and the resultant acoustics. A diversity of tongue shapes, from those creating a more anterior tongue-tip (“retroflex”) constriction to those with a more posterior bunched tongue dorsum constriction, has been observed with notable intraspeaker variation as a function of context; more specifically, a tendency for retroflexion preceding low and/or back vowels. The present study aims to analyze AE /ɹ/ production using real-time MRI videos (83 frames/s) where the target sound was produced in several contexts by 32 American English speakers (16 F/16 M). Initial findings in symmetric VCV production reveal that some of these speakers exclusively use a “retroflex” strategy, some a “bunched” strategy, and others both strategies, consistent with the previously observed tendencies. This observed variability, and the fact that multiple types of static and dynamic imaging data are available from the same speakers, makes this dataset ideal for further probing /ɹ/ production strategies in conjunction with speaker-specific characteristics of vocal tract structure, such as palate shape, and function, such as inter-articulator coordination. [Work supported by NIH and NSF.]