F0 Contour Research Articles

Speaker sincerity influences F0-coupled head motion in spontaneous monologues

Movements of the head during speech serve multiple communicative purposes, including perceptual enhancement of prosodic F0 contours. However, it remains uncertain how much of any correlation between head movement and F0 may be due to physiological coupling mechanisms exerting effects on glottal tension. In this work, six native speakers of American English (3F) were videorecorded during production of spontaneous monologues lasting 90 seconds. Each speaker extemporized twice on two topics chosen from a list (e.g., "video games waste time"). One of these they agreed with ("sincere"), and the other they pretended to agree with ("insincere"). Speaker head movements were quantified as displacement/rotation of a six DOF rigid body from the videos using OpenFace (Baltrušaitis, 2018). F0 for each speaker was converted to semitones relative to the median value for that speaker. Results show larger semitone excursion range but smaller head movement range for the sincere condition, with no significant differences in syllable rate across conditions. LME models predicting absolute semitone changes from head displacement show significant positive slopes for main effects of movement and sincerity, and an additive interaction for sincere productions. These differences suggest that, while head movement correlates with F0 overall, speaker sincerity can interact with the relation.

Just-Noticeable Differences of Fundamental Frequency Change in Mandarin-Speaking Children with Cochlear Implants.

Fundamental frequency (F0) provides the primary acoustic cue for lexical tone perception in tonal languages but remains poorly represented in cochlear implant (CI) systems. Currently, there is still a lack of understanding of sensitivity to F0 change in CI users who speak tonal languages. In the present study, just-noticeable differences (JNDs) of F0 contour and F0 level changes in Mandarin-speaking children with CIs were measured and compared with those in their age-matched normal-hearing (NH) peers. Results showed that children with CIs demonstrated significantly larger JND of F0 contour (JND-C) change and F0 level (JND-L) change compared to NH children. Further within-group comparison revealed that the JND-C change was significantly smaller than the JND-L change among children with CIs, whereas the opposite pattern was observed among NH children. No significant correlations were seen between JND-C change/JND-L change and age at implantation /duration of CI use. The contrast between children with CIs and NH children in sensitivity to F0 contour and F0 level change suggests different mechanisms of F0 processing in these two groups as a result of different hearing experiences.

Computational Modelling of Tone Perception Based on Direct Processing of f0 Contours.

It has been widely assumed that in speech perception it is imperative to first detect a set of distinctive properties or features and then use them to recognize phonetic units like consonants, vowels, and tones. Those features can be auditory cues or articulatory gestures, or a combination of both. There have been no clear demonstrations of how exactly such a two-phase process would work in the perception of continuous speech, however. Here we used computational modelling to explore whether it is possible to recognize phonetic categories from syllable-sized continuous acoustic signals of connected speech without intermediate featural representations. We used Support Vector Machine (SVM) and Self-organizing Map (SOM) to simulate tone perception in Mandarin, by either directly processing f0 trajectories, or extracting various tonal features. The results show that direct tone recognition not only yields better performance than any of the feature extraction schemes, but also requires less computational power. These results suggest that prior extraction of features is unlikely the operational mechanism of speech perception.

Acoustic Analysis of Taiwanese Tones in Esophageal Speech and Pneumatic Artificial Laryngeal Speech.

This study examined the acoustic properties of Taiwanese (Southern Min) lexical tones produced in esophageal speech (ES) and pneumatic artificial laryngeal speech (PAL), including onset fundamental frequency (F0), slope of F0 contour, duration, and amplitude (intensity) of the vowel portion of syllables carrying seven Taiwanese tones. Seven ES speakers, seven PAL speakers, and 18 normal laryngeal (NL) speakers (all male) read aloud target syllables carrying the tones in monosyllables, disyllables, and sentences. Acoustic features were measured with Praat. Acoustic analysis revealed (a) no significant effects of linguistic level on acoustic parameters except for duration, which generally significantly decreased from monosyllables to disyllables to sentences for Tones 1 and 5; (b) significant differences across the speaker groups in onset F0 (PAL > ES > NL), duration (PAL > NL and PAL > ES for all tones), and mean amplitude (PAL > NL > ES); and (c) no significant difference in mean slope of F0 contour across the speaker groups. These preliminary data on acoustic properties of Taiwanese tones produced by ES and PAL speakers could facilitate the design of therapeutic strategies for improving the intelligibility of alaryngeal lexical tones. Further studies are recommended to examine the relative perceptual importance of the acoustic parameters.

Checked Syllables, Checked Tones, and Tone Sandhi in Xiapu Min

A “checked” syllable usually refers to one with a short vowel and an oral or glottal coda, which results impressionistically in a “short” and “abrupt” quality. Although common in languages of the world, it is unclear how to characterize checked syllables phonetically. In this study, we investigated the acoustic features of checked syllables in citation and sandhi forms in Xiapu Min, an under-documented language from China. We conducted a production experiment and analyzed the F0, phonatory quality, vowel duration, and vowel quality in checked syllables. The results show that, in citation tones, checked syllables are realized with distinct F0 contours from unchecked syllables, along with glottalization in the end and a shorter duration overall. In sandhi tones, checked syllables lose their distinct F0 contours and become less glottalized. However, the shorter duration of checked syllables is retained in sandhi forms. This study lays out the acoustic properties that tend to be associated with checked syllables and can be used when testing checked syllables in other language varieties. The fact that in Xiapu Min sandhi checked tones become less glottalized but preserve their shorter duration suggests that, when checked syllables become unchecked diachronically, glottalization might be lost prior to duration lengthening.

Quantitative analysis of fundamental frequency in Spanish (L2) and Brazilian Portuguese (L1)

This paper analyzes the intonation of Spanish and Brazilian Portuguese (BP) produced by monolingual speakers of both languages and bilingual BP speakers that lived in Spain on average for 6 years. Bilinguals produced data in both Spanish L2 (BL2) and BP L1 (BL1). Speech materials are sentences in different modalities (declaratives, yes-no and wh-questions) and reading styles (isolated sentences and storytelling). Fundamental frequency (f0) contours were analyzed to assess learning in Spanish L2 and language attrition in the L1 production of bilinguals. Variability in the f0 contours of the four language conditions was gauged by means of three indices (peak rate, peak range and global standard deviation). Dynamic time warping (DTW) distances between pairs of f0 contours were also measured as a way to measure within- and between-language differences in intonation patterns. The main results are: 1) BL2 and BL1 contours are significantly more variable than the monolingual ones both quantitatively and qualitatively; 2) BL2 contours partially converge towards the patterns of Spanish monolinguals, showing that there is learning; 3) there is evidence for language attrition in the form of transfer of Spanish patterns to BP contours produced by bilinguals; 4) Learning and attrition levels are different depending on sentence modality, such that learning is greater in modalities that differ less between BP and Spanish and attrition is greater in modalities that differ the most.

Using tone and break indices labelling to document patterns of prosodic prominence and their acoustic effects in a database of spoken Italian sentences

This study examined how tonal prominence impacts the acoustic cues to features of Italian vowels. It was one aspect of an analysis of the LaMIT (Lexical Access Model for Italian) project [Di Benedetto et al., “Speech recognition of spoken Italian based on detection of landmarks and other acoustic cues to distinctive features,” in 179th ASA Meeting, Chicago, IL, 8–12 December (2020)]. We strove to understand and systematize the quantitative changes to vowel quality provoked by prosodic and lexical stress, using the LaMIT corpus (100 Italian sentences recited twice by four speakers). Prominent vowels were classified with the TOBI (Tones and Break Indices) labelling method. Accents deemed tonally prominent, known as “pitch accents,” were starred and given a label that reflected their F0 contours. Labels were limited to those put forth by Avesani (“ToBIt: Un sistema di trascrizione per l’intonazione italiana,” in Atti 5e Giornate di Studio del Gruppo di Fonetica Sperimentale, Povo, Italy, pp. 85–98) as constitutive of the prosody of standard Italian. Labels also marked silences, elisions, and spaces between words. Results will clarify if prominence in Italian induces enhanced vowel sonority, or instead a “hyperarticulation” of distinctive segments, contributing to an ongoing debate on the subject.

Perception of checked tones in Xiapu Min

Xiapu Min is a variety of Min Chinese spoken in Xiapu, Fujian, China. The language has seven tones, two of which (T53 and T21) are checked tones that only appear in syllables with a glottal stop coda (Chai and Ye, 2019). Compared with their unchecked counterparts, checked tones have distinct f0 contours, glottalization at the end of the syllable, and are of shorter duration. In this study, we investigate whether those acoustic features are used by listeners of Xiapu Min to identify whether a tone is checked or not. We ran a forced-choice identification task with resynthesized audio stimuli. Stimuli consisted of a natural unchecked token resynthesized with five distinct f0 contours. Each vowel was further modified to have a short versus long duration and presence versus absence of glottalization. The results indicate that the listeners are more likely to identify a stimulus as having a checked tone when it has a high-falling f0 contour (similar to checked T53), shorter duration, or glottalization, among which the duration has the largest coefficient. Thus, all three phonetic parameters found in checked tone production also influence checked tone identification. However, duration is more likely to matter for the identification of checkedness than f0 or glottalization.

The f0 distribution of Korean speakers in a spontaneous speech corpus*

The fundamental frequency, or f0, is an important acoustic measure in the prosody of human speech. The current study examined the f0 distribution of a corpus of spontaneous speech in order to provide normative data for Korean speakers. The corpus consists of 40 speakers talking freely about their daily activities and their personal views. Praat scripts were created to collect f0 values, and a majority of obvious errors were corrected manually by watching and listening to the f0 contour on a narrow-band spectrogram. Statistical analyses of the f0 distribution were conducted using R. The results showed that the f0 values of all the Korean speakers were right-skewed, with a pointy distribution. The speakers produced spontaneous speech within a frequency range of 274 Hz (from 65 Hz to 339 Hz), excluding statistical outliers. The mode of the total f0 data was 102 Hz. The female f0 range, with a bimodal distribution, appeared wider than that of the male group. Regression analyses based on age and f0 values yielded negligible R-squared values. As the mode of an individual speaker could be predicted from the median, either the median or mode could serve as a good reference for the individual f0 range. Finally, an analysis of the continuous f0 points of intonational phrases revealed that the initial and final segments of the phrases yielded several f0 measurement errors. From these results, we conclude that an examination of a spontaneous speech corpus can provide linguists with useful measures to generalize acoustic properties of f0 variability in a language by an individual or groups. Further studies would be desirable of the use of statistical measures to secure reliable f0 values of individual speakers.

Mandarin tone recognition training with cochlear implant simulation: Amplitude envelope enhancement and cue weighting.

With limited fundamental frequency (F0) cues, cochlear implant (CI) users recognize Mandarin tones using amplitude envelope. This study investigated whether tone recognition training with amplitude envelope enhancement may improve tone recognition and cue weighting with CIs. Three groups of CI-simulation listeners received training using vowels with amplitude envelope modified to resemble F0 contour (enhanced-amplitude-envelope training), training using natural vowels (natural-amplitude-envelope training), and exposure to natural vowels without training, respectively. Tone recognition with natural and enhanced amplitude envelope cues and cue weighting of amplitude envelope and F0 contour were measured in pre-, post-, and retention-tests. It was found that with similar pre-test performance, both training groups had better tone recognition than the no-training group after training. Only enhanced-amplitude-envelope training increased the benefits of amplitude envelope enhancement in the post- and retention-tests than in the pre-test. Neither training paradigm increased the cue weighting of amplitude envelope and F0 contour more than stimulus exposure. Listeners attending more to amplitude envelope in the pre-test tended to have better tone recognition with enhanced amplitude envelope cues before training and improve more in tone recognition after enhanced-amplitude-envelope training. The results suggest that auditory training and speech enhancement may bring maximum benefits to CI users when combined.

Effects of speaking style on the shape of fundamental frequency distributions

The present study has two main goals. The first is to describe the effects of three speaking styles (spontaneous interview, sentence reading and word list reading) on statistical estimators of fundamental frequency (f0) variability (mean, standard deviation, skewness and kurtosis) in five female and five male speakers of Brazilian Portuguese (BP). Most f0 contours of word reading are bimodal. Analysis of their time-normalized contours suggests this is caused by the time-compressed realization of fast transitions from low to high or high to low tones aligned with stressed syllables. Considering only unimodal distributions, results show that there are no statistically significant effects in the male data for any of the four variability estimators. Effects show up in female data. Spontaneous style has statistically significant higher mean, SD and skewness than read speech. Findings in the previous literature indicate the reverse pattern, though, for languages other than BP. The second goal of the study is to characterize the statistical properties of f0 distributions beyond mean and SD. Results confirm previous observations that most f0 distributions have positive skewness, are left-tailed and have kurtosis values that deviate significantly from the normal because of large deviations from the central or modal value. A distribution fitting procedure tested six distributions. The asymmetric Burr type XII distribution emerges as the one that best fits the data in the corpus. Results show that two of the parameters that determine its shape correlate well with the empirical f0 distribution values of SD and skewness. Important effects of speaking style on f0 seen in female speakers can be reproduced by combinations of the Burr distributions’ parameters.

Contour clustering: A field-data-driven approach for documenting and analysing prototypical f0 contours

ABSTRACTThis paper reports an automatic data-driven analysis for describing prototypical intonation patterns, particularly suitable for initial stages of prosodic research and language description. The approach has several advantages over traditional ways to investigate intonation, such as the applicability to spontaneous speech, language- and domain-independency, and the potential of revealing meaningful functions of intonation. These features make the approach particularly useful for language documentation, where the description of prosody is often lacking. The core of this approach is a cluster analysis on a time-series of f0 measurements and consists of two scripts (Praat and R, available from https://constantijnkaland.github.io/contourclustering/). Graphical user interfaces can be used to perform the analyses on collected data ranging from spontaneous to highly controlled speech. There is limited need for manual annotation prior to analysis and speaker variability can be accounted for. After cluster analysis, Praat textgrids can be generated with the cluster number annotated for each individual contour. Although further confirmatory analysis is still required, the outcomes provide useful and unbiased directions for any investigation of prototypical f0 contours based on their acoustic form.

Consonantal F0 perturbation in American English involves multiple mechanisms.

In this study, we revisit consonantal perturbation of F0 in English, taking into particular consideration the effect of alignment of F0 contours to segments and the F0 extraction method in the acoustic analysis. We recorded words differing in consonant voicing, manner of articulation, and position in syllable, spoken by native speakers of American English in both statements and questions. In the analysis, we compared methods of F0 alignment and found that the highest F0 consistency occurred when F0 contours were time-normalized to the entire syllable. Applying this method, along with using syllables with nasal consonants as the baseline and a fine-detailed F0 extraction procedure, we identified three distinct consonantal effects: a large but brief (10-40 ms) F0 raising at voice onset regardless of consonant voicing, a smaller but longer-lasting F0 raising effect by voiceless consonants throughout a large proportion of the following vowels, and a small lowering effect of around 6 Hz by voiced consonants, which was not found in previous studies. Additionally, a brief anticipatory effect was observed before a coda consonant. These effects are imposed on a continuously changing F0 curve that is either rising-falling or falling-rising, depending on whether the carrier sentence is a statement or a question.

Effects of perceptual abilities and lexical knowledge on the phonetic categorization of second language speech.

This study examined how phonetic categorization in a second language (L2) is jointly affected by perceptual abilities and lexical knowledge. Adult L1 Mandarin Chinese and L1 English-L2 Mandarin learners performed a phonetic categorization task. The stimuli varied the F0 contour along a continuum resulting in four different tonal word/nonword end point combinations. Both L1 and L2 listeners categorized more ambiguous tokens as words than nonwords, thus demonstrating a lexical bias in their behavior, i.e., the Ganong effect. Non-phonetic, linguistic information can thus modify L2 phonetic categorization of lexical tones. This effect, however, can be constrained by the listener's pitch perception abilities.

The Interaction Between Timescale and Pitch Contour at Pre-attentive Processing of Frequency-Modulated Sweeps

Speech comprehension across languages depends on encoding the pitch variations in frequency-modulated (FM) sweeps at different timescales and frequency ranges. While timescale and spectral contour of FM sweeps play important roles in differentiating acoustic speech units, relatively little work has been done to understand the interaction between the two acoustic dimensions at early cortical processing. An auditory oddball paradigm was employed to examine the interaction of timescale and pitch contour at pre-attentive processing of FM sweeps. Event-related potentials to frequency sweeps that vary in linguistically relevant pitch contour (fundamental frequency F0 vs. first formant frequency F1) and timescale (local vs. global) in Mandarin Chinese were recorded. Mismatch negativities (MMNs) were elicited by all types of sweep deviants. For local timescale, FM sweeps with F0 contours yielded larger MMN amplitudes than F1 contours. A reversed MMN amplitude pattern was obtained with respect to F0/F1 contours for global timescale stimuli. An interhemispheric asymmetry of MMN topography was observed corresponding to local and global-timescale contours. Falling but not rising frequency difference waveforms sweep contours elicited right hemispheric dominance. Results showed that timescale and pitch contour interacts with each other in pre-attentive auditory processing of FM sweeps. Findings suggest that FM sweeps, a type of non-speech signal, is processed at an early stage with reference to its linguistic function. That the dynamic interaction between timescale and spectral pattern is processed during early cortical processing of non-speech frequency sweep signal may be critical to facilitate speech encoding at a later stage.

Phonetic accommodation in interaction with a virtual language learning tutor: A Wizard-of-Oz study

We present a Wizard-of-Oz experiment examining phonetic accommodation of human interlocutors in the context of human-computer interaction. Forty-two native speakers of German engaged in dynamic spoken interaction with a simulated virtual tutor for learning the German language called Mirabella. Mirabella was controlled by the experimenter and used either natural or hidden Markov model-based synthetic speech to communicate with the participants. In the course of four tasks, the participants’ accommodating behavior with respect to wh-question realization and allophonic variation in German was tested. The participants converged to Mirabella with respect to modified wh-question intonation, i.e., rising F0 contour and nuclear pitch accent on the interrogative pronoun, and the allophonic contrast [ɪç] vs.[ɪk] occurring in the word ending 〈-ig〉. They did not accommodate to the allophonic contrast [ɛː] vs.[eː] as a realization of the long vowel 〈-ä-〉. The results did not differ between the experimental groups that communicated with either the natural or the synthetic speech version of Mirabella. Testing the influence of the “Big Five” personality traits on the accommodating behavior revealed a tendency for neuroticism to influence the convergence of question intonation. On the level of individual speakers, we found considerable variation with respect to the degree and direction of accommodation. We conclude that phonetic accommodation on the level of local prosody and segmental pronunciation occurs in users of spoken dialog systems, which could be exploited in the context of computer-assisted language learning.

Assessing the prosodic licensing of wh-in-situ in Japanese

The relationship between syntactic structure and prosodic structure has received increased theoretical attention in recent years. Richards (2010) proposes that Japanese allows wh-elements to stay in-situ because of a certain aspect of its prosodic system. Specifically, in contrast to some other languages like English, Japanese can prosodically group wh-elements together with their licensers. This prosodic grouping is phonetically signaled by eradication or reduction of the lexical pitch accents of intervening words. In this theory, a question still remains as to whether each syntactic derivation is checked against its phonetic realization, or what allows Japanese wh-elements to stay in-situ is more abstract phonological prosodic structure, whose phonetic manifestations can potentially be variable. This paper reports an experiment which addressed this question, by testing whether there is eradication or reduction of lexical pitch accents based on the detailed analyses of F0 contours. Our analysis makes use of a computational toolkit that allows us to assess the presence of tonal targets on a token-by-token basis. The results demonstrate that almost all speakers produce some wh-sentences which show reduction or eradication of the lexical pitch accents, as well as some that do not. Those tokens that show reduction or eradication directly support the prediction of Richard’s (2010) theory. The variability observed in the results suggest that the property of Japanese that allows their wh-elements to stay in-situ must be abstract, phonological prosodic structure, whose phonetic realizations can vary within and across speakers. We discuss several possible mechanisms through which such phonetic variation can arise. (Less)

Vocative melodies in Spanish and English

Abstract This study aims to describe the melodic contours used in Spanish and English calling vocatives in order to identify cross-linguistic similarities and differences. Additionally, it also explores how the sociopragmatic factor of formality may condition contour choices in both languages. 18 speakers of Spanish and 18 speakers of English produced a total of 432 one-word vocatives in formal and informal situations. The analyses of the F0 contours revealed that, although Spanish and English share multiple melodies in this speech act (e.g. L+H* L%, L* H% and L+H* !H%), some tones are language-specific (e.g. L+H* HL% for Spanish). In addition, a General Linear Mixed Model confirmed that these contours are not equally attested in all contexts and that the formality of the situation can condition their use.

Contrastive perception of duration and F0 cues for stop categorization.

This study examined the role of linguistic experience in the contrastive perception of suprasegmental cues for stop categorization. Korean-speaking learners of Mandarin and naive listeners labeled word-medial unaspirated stops (e.g., ma.pa) as either fortis [long closure-high F0] or lenis [short closure-low F0]. The results revealed comparable effects of relative duration for both groups: shorter neighboring vowels elicited more fortis responses, arising from longer perceived stop closure. However, F0 contours were processed contrastively only for the learners: stops were perceived as fortis before vowels with lower offset F0, which may have contributed to a higher perceived onset F0.

Stop Voicing and F0 Perturbation in Pahari

The present study has been carried out to investigate the perturbation effect of the voicing of initial stops on the fundamental frequency (F0) of the following vowels in Pahari. Results show that F0 values are significantly higher following voiceless unaspirated stops than voiced stops. F0 contours indicate an initially falling pattern for vowel [a:] after voiced and voiceless unaspirated stops. A rising pattern after voiced stops and a falling pattern after voiceless unaspirated stops is observed after [i:] and [u:]. These results match Umeda (1981) who found that F0 of a vowel following voiceless stops starts high and drops sharply, but when the vowel follows a voiced stop, F0 starts at a relatively low frequency followed by a gradual rise. The present data show no statistically significant difference between the F0 values of vowels with different places of articulation. Place of articulation is thus the least influencing factor.