Voice Quality Variation Research Articles

Mapping acoustic characteristics of emotional prosody in Mandarin disyllabic words: A machine-learning approach

This study conducted an acoustic-prosodic mapping analysis of emotional prosody in Mandarin Chinese. It utilized a validated audiometry corpus with 450 disyllabic words. The spoken words covered five basic emotions produced by a female speaker: angry, sad, happy, fearful, and neutral. A machine-learning approach was adopted to map key acoustic-prosodic features for Mandarin emotional vocalization. The results revealed distinctive acoustic profiles for each emotion, highlighting variations in fundamental frequency, intensity, speaking rate, and voice quality. Emotional utterances consistently exhibited higher mean F0 values than neutral expressions. Fear displayed the highest crest in F0. Angry and happy utterances showed greater vocal intensity and a faster speaking rate compared to fearful and sad expressions. While anger was associated with a creaky voice quality, sadness corresponded with a breathier voice quality. The current findings are limited with the use of the single-speaker corpus. Ongoing efforts aim to expand the corpus with more speakers to test the generalizability and scalability of the analysis approach for subsequent investigations.

The effects of different voice qualities on the perceived personality of a speaker

Although previous studies investigated various aspects of voice quality perception and personality attribution there are no studies, to our knowledge, which simultaneously examine and compare the perception of various voice qualities when produced by the same individual. This work investigates how laryngeal and supralaryngeal voice quality variations of a speaker affect listeners' perceived personality traits (and thus perceived charisma) of that same speaker. Six Canadian English speakers produced paragraphs varying the following voice qualities: modal, creaky, breathy (natural and artificial), (hyper-)nasalization, and smiling (natural and extreme). Listeners of a perception experiment were then tasked to rate 10 statements for each presented audio stimulus. Statements were selected corresponding to a sub-section of the Big 5 personality traits shown to be linked to charisma perception. Results show significantly more positive listener ratings (i.e., higher ratings compared to modal) with medium effects sizes for both smiling variants across all personality traits. In contrast, creaky was perceived significantly more negatively overall for all personality traits, with a medium effect size. Nasal and breathy still achieved statistically significant rating differences compared to the modal baseline. However, the overall effect pattern was more complex, and effect sizes were small or negligible. Additionally, we found consistent differences for some voice qualities when examining listener ratings comparing male vs. female speakers: for both creaky and smiling (but not for other voice qualities), female speakers were rated more negatively when producing creaky for some personality traits, whereas both smiling variants were consistently rated higher for females compared to males.

Bright Voice Quality and Fundamental Frequency Variation in Non-binary Speakers

1) To investigate if vocal variation produced by assigned-female-at-birth (AFAB) non-binary people differed from vocal variation produced by cisgender (cis) participants. Cue values produced by non-binary participants were predicted to differ from those values produced by cisgender participants. 2) To determine if previous subjective assessments of bright voice quality in AFAB non-binary participants were quantifiable, and if so, if non-binary and cisgender participants differed in their voice quality production. A quantitative comparative research design. Phonetic and statistical analyses of continuous speech samples produced by AFAB non-binary and cisgender participants. Vocal cues were mean fundamental frequency (F0) and bright voice quality, measured by cepstral peak prominence-smoothed and spectral slope, with speaker gender as the predictor. At the group level, non-binary participants produced intermediate F0 values - significantly lower than the cis women's and significantly higher than the cis men's. Individually, the majority of non-binary participants produced mean F0 in this intermediate range. Non-binary participants produced significantly less negative spectral slope and higher cepstral peak prominence-smoothed, indicative of a brighter, more resonant voice quality. Individual-level results indicated that vocal training and vocal tract physiology did not fully account for the results found. Participants' agency, particularly their motivation to alter vocal output to avoid being misgendered, has an effect on the AFAB non-binary participants' F0 production and potentially their voice quality. The majority of AFAB non-binary participants uniquely produced the cue combination of intermediate F0 and bright voice quality.

A Moan of Pleasure Should Be Breathy: The Effect of Voice Quality on the Meaning of Human Nonverbal Vocalizations

Prosodic features, such as intonation and voice intensity, have a well-documented role in communicating emotion, but less is known about the role of laryngeal voice quality in speech and particularly in nonverbal vocalizations such as laughs and moans. Potentially, however, variations in voice quality between tense and breathy may convey rich information about the speaker’s physiological and affective state. In this study breathiness was manipulated in synthetic human nonverbal vocalizations by adjusting the relative strength of upper harmonics and aspiration noise. In experiment 1 (28 prototypes × 3 manipulations = 84 sounds), otherwise identical vocalizations with tense versus breathy voice quality were associated with higher arousal (general alertness), higher dominance, and lower valence (unpleasant states). Ratings on discrete emotions in experiment 2 (56 × 3 = 168 sounds) confirmed that breathiness was reliably associated with positive emotions, particularly in ambiguous vocalizations (gasps and moans). The spectral centroid did not fully account for the effect of manipulation, confirming that the perceived change in voice quality was more specific than a general shift in timbral brightness. Breathiness is thus involved in communicating emotion with nonverbal vocalizations, possibly due to changes in low-level auditory salience and perceived vocal effort.

The effect of vocal effort on contrastive voice quality in Shaoxing Wu.

Voice quality varies at different levels of communication functions. In order to better understand the range of voice quality variation in normal speech, it is important to examine the interaction between global functions and local functions. This study investigates the effect of vocal effort on the contrastive voice quality in Shaoxing Wu. Results show that register contrasts are maintained in all vocal effort conditions, suggesting that the controls for global vs local functions are rather independent. However, the contrastivity of the registers is modulated by the vocal effort conditions, and the register contrasts are less well-defined in the loud and soft conditions.

On the relevance of voice quality in contact varieties

AbstractThis paper focuses on the role of voice quality variation in the system of a contact language, Afro-Yungueño Spanish, a restructured variety of Spanish spoken in the Bolivian Yungas valleys. Based on case studies of naturally occurring conversation between multiple speakers, we show that certain non-modal phonation types, in this case falsetto and breathy voice, are used to index expressiveness, intensification, or emphasis. We argue that these practices have discursive meaning that could otherwise also be encoded by means of grammatical and lexical resources, and that they are an integral part of the linguistic system of this variety. We claim that these practices may have resulted from the specific socio-historical context in which this variety evolved. This suggests that voice quality and ecological factors should not be underestimated in order to reach a more complete picture of how meaning is conveyed in apparently simplified contact languages.

Voice quality variation over the course of the English utterance

The presence of atypical voice quality (e.g. breathy and creaky voice) can be used to diagnose voice disorders, but it is also clear that healthy English speakers’ voices vary as a function of phrasing. For example, ends of declarative utterances tend to be produced with creakier voice quality. Yet for speakers who have no diagnosed voice disorder, it is still unclear what factors affect voice variation over an utterance, and how consistent these variations are across speakers. This in turn limits diagnostic utility of perceived non-modal voice. Therefore, the goal of this project is to determine how the voice varies over the course of English utterances. We recorded electroglottographic (EGG) and audio waveforms of 20 male and female speakers of Californian English reading sentences of various lengths that were designed to avoid non-modal voice associated with certain segments. The EGG waveforms were analyzed for contact quotient and f0 at the beginning, middle, and end of the utterance. Discussion will focus on how voice quality differs by position in utterance, and the extent to which it is predictable from f0. We also discuss whether voice quality varies as a function of gender, utterance length, and speakers’ average voice quality.

Voice quality variation and gender

Recent work on American English has established that speakers increasingly use creaky phonation to convey pragmatic information, with young urban women assumed to be the most active users of this phonetic feature. However, no large-scale acoustic or articulatory study has established the actual range and diversity of voice quality variation along gender identities, encompassing different sexual orientations, regional backgrounds, and socioeconomic statuses. The current study does exactly that, through four methods: (1) subjects identifying with a range of gender and other demographic identities were audio recorded while reading wordlists as well as a scripted narrative assuming characters’ voices designed to elicit variation in vowel quality. Simultaneously, (2) electroglottographic readings were taken and analyzed to determine the glottal characteristics of this voice quality variation. (3) Subjects were then asked to rate recordings of other people’s voices to identify the personal characteristics associated with the acoustic reflexes of phonation; in the final task, (4) subjects were explicitly asked about their language ideologies as they relate to gender. Thus, the current study explores the relation between gender identity and phonetic features, measured acoustically, articulatorily, and perceptually. This work is currently underway and preliminary results are being compiled at this time.

Voice quality variation across gender identities

While several studies of the phonetics of American English make reference to general features of “women's speech” and “men's speech,” no large-scale acoustic or articulatory study has established the actual range and diversity of phonetic variation along gender identities, encompassing different sexual orientations, regional backgrounds, and socioeconomic classes. The current study explores the acoustic and articulatory phonetic features related to gender, particularly focusing on voice quality, e.g., creaky and breathy phonation. Subjects identifying with a range of gender and other demographic identities were audio recorded producing the Rainbow Passage as well as wordlists targeting variation in vowel quality. Simultaneously, electroglottographic (EGG) readings were taken and analyzed for voice quality. Subjects were then asked to rate recordings of other people’s voices to identify the personal characteristics associated with the acoustic reflexes of phonation; in the final task, subjects were also explicitly asked about their language ideologies as they relate to gender. Thus, the current study explores the relation between gender identity and phonetic features, measured acoustically, articulatorily, and perceptually. This work is currently underway and preliminary results are being compiled at this time.

A cross-language study of breathing rhythm related to different linguistic structures in Chinese and Korean

This paper investigates correlations between respiration patterns and different linguistic structures in Chinese and Korean.The study includes comparative analysis between the Korean and Chinese scripts of poetry and news with same meaning and different languages, and the subjects included 4 native speakers (2 males, 2 females) for each language. It can be made certain by measuring physiological and acoustic parameters by means of two respiratory sensors for chest and abdomen and electroglottalgraphy. Preliminary results show that due to the poetry has fixed frame, therefore regardless of language and sex, the respiratory rhythm of the poetry is quite similar in two languages. In contrast, for news scripts, the data show that breathing groups of the Chinese quite correspond to intonation units. In contrast,the Korean is more flexable in planning breathing rhythm,generally breathing resets appears not only in intonation phrase but in accentual phrase,and chest respiration is mostly used for the accentual phrase. It suggests that different linguistic structures operate powerfully upon preplanning and respiratory patterns. Furthermore, the results show the correlation of voice quality variation and respiratory rhythm in different languages.

ASM: Adaptive Voice Stream Multicast over Low-Power Wireless Networks

Low-power Wireless Networks (LWNs) have become increasingly available for mission-critical applications such as security surveillance and disaster response. In particular, emerging low-power wireless audio platforms provide an economical solution for ad hoc voice communication in emergency scenarios. In this paper, we develop a system called Adaptive Stream Multicast (ASM) for voice communication over multihop LWNs. ASM is composed of several novel components specially designed to deliver robust voice quality for multiple sinks in dynamic environments: 1) an empirical model to automatically evaluate the voice quality perceived at sinks based on current network condition; 2) a feedback-based Forward Error Correction (FEC) scheme where the source can adapt its coding redundancy ratio dynamically in response to the voice quality variation at sinks; 3) a Tree-based Opportunistic Routing (TOR) protocol that fully exploits the broadcast opportunities on a tree based on novel forwarder selection and coordination rules; and 4) a distributed admission control algorithm that ensures the voice quality guarantees when admitting new voice streams. ASM has been implemented on a low-power hardware platform and extensively evaluated through experiments on a test bed of 18 nodes. The experiment results show that ASM can achieve satisfactory multicast voice quality in dynamic environments while incurring low-communication overhead.

Quantitative voice quality analyses of a soprano singing early music in three different performance styles

This paper considers the measurement of voice quality variations relating to three different performance styles of early music singing by a trained soprano. In particular, the various outputs available following the collection of electrolaryngographic data are presented along with acoustic analyses based on the acoustic pressure waveform to enable a comparison of the three different singing styles. Voice quality analysis is based on the output from the electrolaryngograph, that enables a measurement of fundamental period to 1 μs accuracy to be made, enabling fundamental frequency variations including vibrato to be explored. In addition, the percentage of each cycle for which the vocal folds are in contact is measured and varies between the three voice qualities providing insights into the nature of vocal fold vibration in each case. Acoustic output spectral measures are made that are not only based on traditional wide- and narrow-band but also hearing modelling spectrography. These provide a clear indication as to which portions of the spectrum are most prominent in terms of perception of the three different voice qualities. A number of features resulting from the various analyses distinguish the singing styles, thereby providing evidence that the methods used are appropriate for objective characterisation and differentiation of singing styles in early music.

Effects of native language on perception of voice quality

Little is known about how listeners judge phonemic versus allophonic (or freely varying) versus post-lexical variations in voice quality, or about which acoustic attributes serve as perceptual cues in specific contexts. To address this issue, native speakers of Gujarati, Thai, and English discriminated among pairs of voices that differed only in the relative amplitudes of the first versus second harmonics (H1–H2). Results indicate that speakers of Gujarati (which contrasts H1–H2 phonemically) were more sensitive to changes than are speakers of Thai or English. Further, sensitivity was not affected by the overall source spectral slope for Gujarati speakers, unlike Thai and English speakers, who were most sensitive when the spectrum fell away steeply. In combination with previous findings from Mandarin speakers, these results suggest a continuum of sensitivity to H1–H2. In Gujarati, the independence of sensitivity and spectral context is consistent with use of H1–H2 as a cue to the language's phonemic phonation contrast. Speakers of Mandarin, in which creaky phonation occurs in conjunction with the low-dipping Tone 3, apparently also learn to hear these contrasts, but sensitivity is conditioned by spectral context. Finally, for Thai and English speakers, who vary phonation only post-lexically, sensitivity is both lower and contextually determined, reflecting the smaller role of H1–H2 in these languages.

Voice quality variation with fundamental frequency in English and Mandarin.

Previous research has shown that F0 is positively related to H1*-H2* across male speakers of English [Iseli et al. (2006)] and to H1-H2 (after inverse filtering) within individual male speakers of Dutch [Swerts and Veldhuis (2001)]. That is, males who have overall higher-pitched voices generally have overall higher values of H1*-H2* (cross-speaker relation), and as an individual male’s F0 goes up, H1-H2 generally also goes up (within-speaker relation). The present study investigates both of these relations, cross-speaker and within-speaker, for male and female speakers of English and Mandarin, and extends them to a large set of voice quality measures. The speech samples consist of repeated rising and falling tone sweeps, in which speakers began at a self-selected comfortable pitch, and then swept either up or down in pitch to their highest or lowest comfortable pitch. The beginnings of the sweeps are tested for cross-speaker relations, while the entire sweeps are tested for within-speaker relations. VOICESAUCE, a new program for voice analysis, is used to extract F0, energy, cepstral peak prominence, formants and bandwidths, and a variety of harmonic amplitude measures. Many measures are shown to be strongly related to F0. [Work supported by NSF.]

Semi-supervised speaker identification under covariate shift

In this paper, we propose a novel semi-supervised speaker identification method that can alleviate the influence of non-stationarity such as session dependent variation, the recording environment change, and physical conditions/emotions. We assume that the voice quality variants follow the covariate shift model, where only the voice feature distribution changes in the training and test phases. Our method consists of weighted versions of kernel logistic regression and cross validation and is theoretically shown to have the capability of alleviating the influence of covariate shift. We experimentally show through text-independent/dependent speaker identification simulations that the proposed method is promising in dealing with variations in voice quality.

Cross-language perception of voice and affect

The paper reports on a cross-language study of how voice quality and f0 combine in the signaling of affect. Speakers of Irish-English and Japanese participated in perception tests. The stimuli consisted of a short utterance where f0 and voice source parameters were varied using the LF-model implementation of the KLSyn88a formant synthesizer, and were of three types: (1) VQ only involving voice quality variations and a neutral f0 contour; (2) f0 only, with different affect-related f0 contours and modal voice; (3) VQ+f0 stimuli, where the voice qualities of (1) combine with specific f0 contours from (2). Overall, stimuli involving voice quality variation were consistently associated with affect. In (2) only stimuli with high f0 yielded high affective ratings. Striking differences emerge between the ratings obtained from the two language groups. The results show that not only were some affects consistently perceived by one language group and not the other, but also that specific voice qualities and pitch contours were associated with very different affects across the two groups. The results have important implications for expressive speech synthesis, indicating that language/culture-specific differences need to be considered. [This work is supported by the EU-funded Network of Excellence on Emotion, HUMAINE.]

Effects of spectral slope on perceived breathiness in vowels

Previous studies have indicated that breathiness is correlated with changes in spectral slope as well as aspiration noise [Klatt & Klatt, ‘‘Analysis, synthesis, and perception of voice quality variations among female and male talkers,’’ J. Acoust. Soc. Am. 87, 820–857 (1990)]. In a recent study, Shrivastav and Pinero [Shrivastav & Pinero ‘‘Effect of aspiration noise and spectral slope on perceived breathiness in vowels,’’ J. Acoust. Soc. Am. 117, 2622 (2005)] observed that varying spectral slope resulted in minimal changes on the perceived breathiness for synthetic vowels. However, the stimuli tested in this experiment included a relatively narrow range of spectral slope variation. The goal of the present experiment was to verify the role of spectral slope on the perception of breathiness by testing stimuli that had a wider range of variation in spectral slope. Ten voices (5 male and 5 female) were synthesized using a Klatt synthesizer. Each of these was manipulated to generate two continua varying in their spectral slope from −3 to −30 dB/octave. One continuum had a constant cutoff frequency of 500 Hz, while the other continuum had a cutoff frequency between the second harmonic (H2) and the third harmonic (H3). Ten listeners judged the degree of breathiness using a 7-point rating scale. Preliminary results suggest that spectral slope has minimal effect on the perception of breathiness in synthetic vowels. [Research supported by NIH/R21 DC006690.]

Analysis of modal and creaky voice quality variations

Voice quality, as a major vehicle of information about physical, phonological, and social characteristics of the speaker, has a vital semiotic role to play in spoken interaction [Laver (1968), Laver and Trudgill (1979)]. In the past couple of years, our lab developed an Aperiodicity/Periodicity/Pitch (APP) detector that produces a spectro-temporal profile of the periodic and aperiodic regions of the speech waveform [Deshmukh et al. (in press)]. To do so, the speech signal is passed through a 60-channel gamma tone auditory filterbank. The distribution of the dips occurring in the average magnitude difference function (AMDF) computed from each channel envelope is analyzed to determine periodicity and aperiodicity. Presently, the APP detector classifies both turbulent noise and irregular vocal fold vibration (creakiness) as aperiodic. In this work, we are investigating the detailed characteristics of the AMDF waveform when speech is creaky. This information is presently being used to distinguish aperiodicity due to turbulence from aperiodicity due to creakiness. We will present results from the refined APP detector using various male and female utterances from the TIMIT database.

Voice quality variations in English sentences

This study examines the predictability of changes in voice quality at the sentence level in English. Sentence-level effects can only be isolated once the effects of linguistic factors (e.g., glottalization before a glottalized consonant), social or dialectal, and individual factors have been eliminated. In this study, these effects were controlled by obtaining a baseline value for each measurement for each word of the corpus. Voice quality variations were tracked using quantitative measurements derived from the LF model of the glottal source, and also qualitative descriptions of the waveforms. Preliminary results indicate that there are consistent voice quality differences at the sentence level and that pitch contours and sentence accent also produce predictable effects on voice quality.

Utterance–final voice quality variations: Their perceptual structure and acoustic correlates

Voice quality variations can be used to mark prosodic and discourse boundaries in running speech. In order to quantify acoustic events in the signal during the production of these voice qualities, it is necessary to first discover their perceptual structure. The perceptual structure can then inform the acoustic analysis and help to find perceptually salient acoustic correlates. In this study, a perceptual experiment was run in which the stimuli were single words spoken by multiple talkers, which had all been extracted from the ends of utterances. Pairs of stimuli were given similarity ratings by listeners, and the similarity ratings were used in a multidimensional scaling analysis. The dimensions found in the multidimensional scaling analysis were interpreted using acoustic analysis. The voice qualities of the female voices in this study were found to have a primary perceptual split which was interpretable acoustically as laryngealized versus nonlaryngealized, but the voice qualities found in the male voices did not group into categories, and the perceptual dimensions were not as clearly interpretable in acoustic terms.