Voice Pitch Research Articles

The role of valence, dominance, and pitch in perceptions of artificial intelligence (AI) conversational agents’ voices

There is growing concern that artificial intelligence conversational agents (e.g., Siri, Alexa) reinforce voice-based social stereotypes. Because little is known about social perceptions of conversational agents’ voices, we investigated (1) the dimensions that underpin perceptions of these synthetic voices and (2) the role that acoustic parameters play in these perceptions. Study 1 (N = 504) found that perceptions of synthetic voices are underpinned by Valence and Dominance components similar to those previously reported for natural human stimuli and that the Dominance component was strongly and negatively related to voice pitch. Study 2 (N = 160) found that experimentally manipulating pitch in synthetic voices directly influenced dominance-related, but not valence-related, perceptions. Collectively, these results suggest that greater consideration of the role that voice pitch plays in dominance-related perceptions when designing conversational agents may be an effective method for controlling stereotypic perceptions of their voices and the downstream consequences of those perceptions.

Supplementary Materials for article entitled 'The temporal alignment of speech-accompanying eyebrow movement and voice pitch. A study based on Late Night Show interviews'

Supplementary Materials for article entitled 'The temporal alignment of speech-accompanying eyebrow movement and voice pitch. A study based on Late Night Show interviews'

A Novel Approach for Emotion Detection and Sentiment Analysis for Low Resource Urdu Language Based on CNN-LSTM

Emotion detection (ED) and sentiment analysis (SA) play a vital role in identifying an individual’s level of interest in any given field. Humans use facial expressions, voice pitch, gestures, and words to convey their emotions. Emotion detection and sentiment analysis in English and Chinese have received much attention in the last decade. Still, poor-resource languages such as Urdu have been mostly disregarded, which is the primary focus of this research. Roman Urdu should also be investigated like other languages because social media platforms are frequently used for communication. Roman Urdu faces a significant challenge in the absence of corpus for emotion detection and sentiment analysis because linguistic resources are vital for natural language processing. In this study, we create a corpus of 1021 sentences for emotion detection and 20,251 sentences for sentiment analysis, both obtained from various areas, and annotate it with the aid of human annotators from six and three classes, respectively. In order to train large-scale unlabeled data, the bag-of-word, term frequency-inverse document frequency, and Skip-gram models are employed, and the learned word vector is then fed into the CNN-LSTM model. In addition to our proposed approach, we also use other fundamental algorithms, including a convolutional neural network, long short-term memory, artificial neural networks, and recurrent neural networks for comparison. The result indicates that the CNN-LSTM proposed method paired with Word2Vec is more effective than other approaches regarding emotion detection and evaluating sentiment analysis in Roman Urdu. Furthermore, we compare our based model with some previous work. Both emotion detection and sentiment analysis have seen significant improvements, jumping from an accuracy of 85% to 95% and from 89% to 93.3%, respectively.

Phonological effects on the perceptual weighting of voice cues for voice gender categorization.

Voice perception and speaker identification interact with linguistic processing. This study investigated whether lexicality and/or phonological effects alter the perceptual weighting of voice pitch (F0) and vocal-tract length (VTL) cues for perceived voice gender categorization. F0 and VTL of forward words and nonwords (for lexicality effect), and time-reversed nonwords (for phonological effect through phonetic alterations) were manipulated. Participants provided binary "man"/"woman" judgements of the different voice conditions. Cue weights for time-reversed nonwords were significantly lower than cue weights for both forward words and nonwords, but there was no significant difference between forward words and nonwords. Hence, voice cue utilization for voice gender judgements seems to be affected by phonological, rather than lexicality effects.

When faces and voices come together: Face width‐to‐height ratio and voice pitch contribute independently to social perception

AbstractPrior research has shown that the face's width‐to‐height ratio (fWHR) and the voice's pitch influence social perception. Yet, the relative contribution of either cue has been largely unexplored. We examined the simultaneous effects of fWHR and pitch on social evaluations. Experiment 1 (N = 102) tested how such cues shaped global impressions. Experiment 2 (N = 121) tested fWHR and pitch's effect on behavioural affiliative intentions, framing social interaction as a physical or an intellectual competition. Experiment 3 (N = 57) assessed whether variations in fWHR and pitch could influence trait attribution (i.e., physical formidability and intelligence). Individuals with large faces or low‐pitched voices elicited negative impressions, positive behavioural intentions in a physical competition, and the attribution of stronger formidability but lower intelligence. Across the studies, cues exerted independent effects. The implications of these findings for research on cross‐modal social perception are discussed.

Employing Energy and Statistical Features for Automatic Diagnosis of Voice Disorders.

The presence of laryngeal disease affects vocal fold(s) dynamics and thus causes changes in pitch, loudness, and other characteristics of the human voice. Many frameworks based on the acoustic analysis of speech signals have been created in recent years; however, they are evaluated on just one or two corpora and are not independent to voice illnesses and human bias. In this article, a unified wavelet-based paradigm for evaluating voice diseases is presented. This approach is independent of voice diseases, human bias, or dialect. The vocal folds' dynamics are impacted by the voice disorder, and this further modifies the sound source. Therefore, inverse filtering is used to capture the modified voice source. Furthermore, the fundamental frequency independent statistical and energy metrics are derived from each spectral sub-band to characterize the retrieved voice source. Speech recordings of the sustained vowel /a/ were collected from four different datasets in German, Spanish, English, and Arabic to run the several intra and inter-dataset experiments. The classifiers' achieved performance indicators show that energy and statistical features uncover vital information on a variety of clinical voices, and therefore the suggested approach can be used as a complementary means for the automatic medical assessment of voice diseases.

Mobile Phone Applications Voice Tools and Voice Pitch Analyzer Validated With LingWAVES to Measure Voice Frequency

Voice frequency can be measured to assess the voice change in transgender men and women during treatment. There are many applications that can analyze voice frequency. This validation study aimed to compare the ability to measure voice frequency of the mobile phone applications "Voice Tools" and "Voice Pitch Analyzer" with the registration program LingWAVES as the gold standard. Prospective validation study. A total of 45 participants of whom 20 transgender individuals were included. They were asked to read "The North Wind and the Sun" twice. The first measurement compared voice frequency measured by Voice Tools with LingWAVES while the second measurement compared Voice Pitch Analyzer with LingWAVES. The two applications that were being compared simultaneously measured the voice frequency. Pearson's regression correlations were performed to test for correlation between the mobile phone applications and LingWAVES. Significant correlations were demonstrated between the measurements of Voice Tools and LingWAVES (P < 0.001) and between Voice Pitch Analyzer and LingWAVES (P < 0.001). Voice Tools overestimated voice frequency with a median deviation of 2Hz (range -4 to 20). The overestimation was more pronounced in the high ranges. Voice Pitch Analyzer showed underestimation of voice frequency in high ranges. Median deviation was -2Hz (range -16 to 14). This validation study shows that voice frequency can be reliably measured with the mobile phone applications "Voice Tools" and "Voice Pitch Analyzer". Combined with the ease of use, these applications can be used to measure voice frequency in clinical practice and for research purposes.

Paying Lip Service?

Literature has mainly analyzed the technical attributes of voice for enterprise technical applications, such as voice signature and speech recognition. This paper aims at identifying voice and speech attributes that reflect sentiment and affect (positively or negatively) customer satisfaction levels in voice interactions. The paper used method triangulation that utilizes multiple data sources to gain comprehensive understanding of the domain, including auditory observations, focus group interviews, customer survey, and a review of recent academic studies. The findings indicate that customer experience and satisfaction are influenced by the relative values of voice pitch between the CSR (customer service representative) and the customer. The major conclusion of the study suggests that voice fundamental pitch, speech rate, voice amplitude, and other communication parameters might deploy hidden power and affect service interaction's results. The study opens a new venue for research on social interaction adjustment from the CSR's perspective and from the customer's standpoint.

Task-Dependent Modulation of Auditory Feedback Control of Vocal Intensity

Auditory feedback control of fundamental frequency (fo) is modulated in a task-dependent manner. When voice pitch auditory feedback perturbations are applied in sentence versus sustained-vowel production, larger and faster vocal fo responses are measured in sentence production. This task-dependency reflects the scaling of auditory targets for pitch for the precision required in each speech task. When the range for the pitch auditory target is scaled down for precision (as in the sentence-production task), a greater degree of mismatch is detected from the feedback perturbation and a larger vocal response is measured. The purpose of this study was to determine whether auditory feedback control of vocal intensity is also modulated in a task-dependent manner similar to the control of vocal pitch. Twenty-five English speakers produced repetitions of a sentence and a sustained vowel while hearing their voice auditory feedback briefly perturbed in loudness (+/- 3 or 6 dB SPL, 200 ms duration). The resulting vocal intensity responses were measured, and response magnitudes were robustly larger in the sentence (mean: 1.96 dB) than vowel production (mean: 0.89 dB). Additionally, response magnitudes increased as a function of perturbation magnitude only in sentence production for downward perturbations but decreased in magnitude by perturbation magnitude for upward perturbations. Peak response latencies were robustly shorter in sentence (mean: 184.94 ms) than in vowel production (mean: 214.92 ms). Overall, these results support the hypothesis that auditory feedback control of pitch and loudness are modulated by task and that both pitch and loudness auditory targets are scaled for the precision required for the speaking task.

Heritability of Sound Processing Deficits in Autism: Neural Insights

Heritability of Sound Processing Deficits in Autism: Neural Insights

AN ANALYSIS OF INTONATION PATTERN IN THE PRE SERVICE ENGLISH TEACHER’S TALKS

This study examined the intonation pattern errors occurring in pre-service English teachers’ talks and investigated the causes of the errors. This research used three instruments to gain the data: documentation, observation and interview. Then, the collected data were analyzed using Roach, Wells Haycraft theories and PRAAT software analysis. The participants of this study were 8th semester students in English Language Education Study Program at one of the private universities in Yogyakarta. Three recordings of pre-service English teachers’ talks were transcribed by unitizing schemes, sampling plans, recording/coding, reducing data, abductively inferring contextual phenomena, and narrating the result information of the data collected. The results showed that two kinds of intonation patterns occurred in pre-service English teachers, falling and rising. The speakers’ dominant error was final rising in WH-question sentence type. The errors were caused by the physical impediment, inability in manipulating the pitch voice, and inability to keep the level tone stable. In addition, it was also due to the pre-service English teachers L1 mother tongue. This study gave information of pre-service English teachers’ ability, obstacle, and the causes of errors in producing intonation tone. It can be a reference for pre-service English teachers to look for pre-service teachers’ ability in mastering pronunciation especially intonation.

Behavioral Treatment Approaches to Lowering Pitch in the Female Voice

To assess the outcomes of three voice therapy treatment approaches with an emphasis on lowering speaking pitch. Transmasculine and cisgender individuals may desire to lower their speaking pitch, yet there has not been a method described in the literature to do this effectively using only behavioral techniques. To investigate these approaches, we enrolled 32 adult cisgender females and randomly assigned them to one of four treatment groups: vocal function exercises (VFE), resonant voice therapy (RVT), lip-rounding therapy (LRT), and a control group. Participants received individual instruction and feedback on the given exercise program, and they continued to practice daily for 4 weeks. Acoustic recordings were collected before treatment, immediately after the first session, and after 4 weeks of treatment. Results showed a lower minimum pitch in the physiological range, lower speaking fundamental frequency (SFF) in reading, and lower SFF in spontaneous speech-with treatment groups performing better than the control group. Additionally, participants' self-rating of the vocal effort expended to speak in a low pitch decreased over the treatment period. Each treatment approach (VFE, RVT, and LRT) was successful in lowering the speaking pitch of cisgender females. These methods would likely be useful for clients seeking to speak in a lower pitch. Future research may expand results to include clinical populations, such as transmasculine individuals.

The effect of hearing protection worn by talker and/or target listener on speech production in quiet and noise.

Speech production while wearing hearing protectors poses significant challenges due to their occlusion effect and disruption of the Lombard effect. An experiment was conducted with 24 individuals as they read a list of 12 sentences in open ears and while wearing an earmuff in quiet and in four different noises [pink, International Female Fluctuating Masker (IFFM), speech-spectrum noise (SSnoise), and helicopter] at two levels (70 and 85 dBA). An acoustic manikin, fitted or not with an identical protector, served as the target listener. In noise, speech levels decreased when the talkers wore the earmuff but increased when the target listener was fitted with the earmuff. When the earmuff was used by both the talkers and target listener, speech levels were lower by 3-6 dB at the higher noise level compared to when they were both open ears. Speech levels were typically lower, but extended speech intelligibility index estimates were consistently higher, in fluctuating (IFFM, helicopter) than in continuous noises (pink, SSnoise). Talkers' pitch frequency and voice spectrum measurements followed very closely the changes in speech levels, showing no evidence of compensatory voice modifications. Implications of the lower talker speech levels when wearing hearing protectors are discussed in terms of protector selection, training, and individuals with hearing loss.

Femme/Butch/Androgyne Identity and Preferences for Femininity Across Face, Voice, and Personality Traits in Chinese Lesbian and Bisexual Women.

Studies indicate that individuals preferring masculinity/femininity in one domain also prefer it in other domains. Heterosexual men and women and gay men have reported consistent preferences for masculinity/femininity across the faces and voices of their preferred sex. This study explored the femininity preferences of 417 Chinese lesbian and bisexual women in terms of face, voice pitch, vocal tract length, and personality traits and explored the effect of sexual self-labels (femme, butch, and androgyne) on these preferences. We found that lesbian and bisexual women showed a stronger preference for feminized faces, voice pitch, vocal tract length, and personality traits than masculinized versions, and these preferences were highly consistent across the four domains. Moreover, femininity preference was moderated by sexual self-labels, with butches preferring more feminine voice pitch, vocal tract length, and personality traits than femmes and androgynes. However, no significant difference was found for facial femininity preferences among different sexual self-labels. These findings present evidence of consistent femininity preference across visual, auditory, and personality traits and suggest that, regardless of sexual orientation, multiple cues may be used together when determining the attractiveness of individuals. Furthermore, these results support the hypothesis that the partner preference of lesbian and bisexual women mirrors that of heterosexual men.

Cortical tracking of voice pitch in the presence of multiple speakers depends on selective attention.

Voice pitch carries linguistic and non-linguistic information. Previous studies have described cortical tracking of voice pitch in clean speech, with responses reflecting both pitch strength and pitch value. However, pitch is also a powerful cue for auditory stream segregation, especially when competing streams have pitch differing in fundamental frequency, as is the case when multiple speakers talk simultaneously. We therefore investigated how cortical speech pitch tracking is affected in the presence of a second, task-irrelevant speaker. We analyzed human magnetoencephalography (MEG) responses to continuous narrative speech, presented either as a single talker in a quiet background or as a two-talker mixture of a male and a female speaker. In clean speech, voice pitch was associated with a right-dominant response, peaking at a latency of around 100 ms, consistent with previous electroencephalography and electrocorticography results. The response tracked both the presence of pitch and the relative value of the speaker’s fundamental frequency. In the two-talker mixture, the pitch of the attended speaker was tracked bilaterally, regardless of whether or not there was simultaneously present pitch in the speech of the irrelevant speaker. Pitch tracking for the irrelevant speaker was reduced: only the right hemisphere still significantly tracked pitch of the unattended speaker, and only during intervals in which no pitch was present in the attended talker’s speech. Taken together, these results suggest that pitch-based segregation of multiple speakers, at least as measured by macroscopic cortical tracking, is not entirely automatic but strongly dependent on selective attention.

Stimulation Rate and Voice Pitch Perception in Cochlear Implants.

The stimulation rate in cochlear implant (CI) sound coding, or the "carrier" rate in pulses per second (pps), is known to influence pitch perception, as well as loudness perception and sound quality. Our main objective was to investigate the effects of reduced carrier rate on the loudness and pitch of coded speech samples. We describe two experiments with 16 Nucleus® CI users, where we controlled modulation characteristics and carrier rate using Spectral and Temporal Enhanced Processing (STEP), a novel experimental multichannel sound coder. We used a fixed set of threshold and comfortable stimulation levels for each subject, obtained from clinical MAPs. In the first experiment, we determined equivalence for voice pitch ranking and voice gender categorization between the Advanced Combination Encoder (ACE), a widely used clinical strategy in Nucleus® recipients, and STEP for fundamental frequencies (F0) 120-250Hz. In the second experiment, loudness was determined as a function of the input amplitude of speech samples for carrier rates of 1000, 500, and 250 pps per channel. Then, using equally loud sound coder programs, we evaluated the effect of carrier rate on voice pitch perception. Although nearly all subjects could categorize voice gender significantly above chance, pitch ranking varied across subjects. Overall, carrier rate did not substantially affect voice pitch ranking or voice gender categorization: as long as the carrier rate was at least twice the fundamental frequency, or when stimulation pulses for the lowest, 250 pps carrier were aligned to F0 peaks. These results indicate that carrier rates as low as 250 pps per channel are sufficient to support functional voice pitch perception for those CI users sensitive to temporal pitch cues; at least when temporal modulations and pulse timings in the coder output are well controlled by novel strategies such as STEP.

Think Leader, Think Deep Voice? CEO Voice Pitch and Gender

Several studies in evolutionary psychology, political science, and some in management have established that followers prefer their leaders to have lower-pitched voices. While most studies have explored male leader voices, the follower preference for lower-pitched leader voices has been found to be robust across a number of decision domains ranging from selecting aboriginal tribal leaders to paying CEOs. Drawing upon the follower-centric theories of leadership, we argue that gender will moderate this relationship with the effects of pitch becoming weaker, first, (1) when leaders are female, and second, (2) when followers are female. In a forced-choice lab study where followers (i.e., respondents listening to male and female CEOs’ voices) must select the most trustworthy-sounding CEO among three choices, we replicate the main finding that followers perceive lower-pitched leader voices as being more trustworthy. However, this relationship is moderated when assessing female top executive voices with the effect of voice pitch being much lower than when assessing male CEOs. This finding suggests that followers may have a different leader prototype of what makes a female business leader trustworthy.

Gender and Context-Specific Effects of Vocal Dominance and Trustworthiness on Leadership Decisions

ObjectiveThe evolutionary-contingency hypothesis, which suggests that preferences for leaders are context-dependent, has found relatively consistent support from research investigating leadership decisions based on facial pictures. Here, we test whether these results transfer to leadership decisions based on voice recordings. We examined how dominance and trustworthiness perceptions relate to leadership decisions in wartime and peacetime contexts and whether effects differ by a speaker’s gender. Further, we investigate two cues that might be related to leadership decisions, as well as dominance and trustworthiness perceptions: voice pitch and strength of regional accent.MethodsWe conducted a preregistered online study with 125 raters and recordings of 120 speakers (61 men, 59 women) from different parts in Germany. Raters were randomly distributed into four rating conditions: dominance, trustworthiness, hypothetical vote (wartime) and hypothetical vote (peacetime).ResultsWe find that dominant speakers were more likely to be voted for in a wartime context while trustworthy speakers were more likely to be voted for in a peacetime context. Voice pitch functions as a main cue for dominance perceptions, while strength of regional accent functions as a main cue for trustworthiness perceptions.ConclusionsThis study adds to a stream of research that suggests that (a) people’s voices contain important information based on which we form social impressions and (b) we prefer different types of leaders across different contexts. Future research should disentangle effects of gender bias in leadership decisions and investigate underlying mechanisms that influence how people’s voices contribute to achieving social status.

Maturation of Speech-in-Speech Recognition for Whispered and Voiced Speech.

Some speech recognition data suggest that children rely less on voice pitch and harmonicity to support auditory scene analysis than adults. Two experiments evaluated development of speech-in-speech recognition using voiced speech and whispered speech, which lacks the harmonic structure of voiced speech. Listeners were 5- to 7-year-olds and adults with normal hearing. Targets were monosyllabic words organized into three-word sets that differ in vowel content. Maskers were two-talker or one-talker streams of speech. Targets and maskers were recorded by different female talkers in both voiced and whispered speaking styles. For each masker, speech reception thresholds (SRTs) were measured in all four combinations of target and masker speech, including matched and mismatched speaking styles for the target and masker. Children performed more poorly than adults overall. For the two-talker masker, this age effect was smaller for the whispered target and masker than for the other three conditions. Children's SRTs in this condition were predominantly positive, suggesting that they may have relied on a wholistic listening strategy rather than segregating the target from the masker. For the one-talker masker, age effects were consistent across the four conditions. Reduced informational masking for the one-talker masker could be responsible for differences in age effects for the two maskers. A benefit of mismatching the target and masker speaking style was observed for both target styles in the two-talker masker and for the voiced targets in the one-talker masker. These results provide no compelling evidence that young school-age children and adults are differentially sensitive to the cues present in voiced and whispered speech. Both groups benefit from mismatches in speaking style under some conditions. These benefits could be due to a combination of reduced perceptual similarity, harmonic cancelation, and differences in energetic masking.

The Effects of Pitch Manipulation on Male Ratings of Female Speakers and Their Voices.

Vocal and facial cues typically co-occur in natural settings, and multisensory processing of voice and face relies on their synchronous presentation. Psychological research has examined various facial and vocal cues to attractiveness as well as to judgements of sexual dimorphism, health, and age. However, few studies have investigated the interaction of vocal and facial cues in attractiveness judgments under naturalistic conditions using dynamic, ecologically valid stimuli. Here, we used short videos or audio tracks of females speaking full sentences and used a manipulation of voice pitch to investigate cross-modal interactions of voice pitch on facial attractiveness and related ratings. Male participants had to rate attractiveness, femininity, age, and health of synchronized audio-video recordings or voices only, with either original or modified voice pitch. We expected audio stimuli with increased voice pitch to be rated as more attractive, more feminine, healthier, and younger. If auditory judgements cross-modally influence judgements of facial attributes, we additionally expected the voice pitch manipulation to affect ratings of audiovisual stimulus material. We tested 106 male participants in a within-subject design in two sessions. Analyses revealed that voice recordings with increased voice pitch were perceived to be more feminine and younger, but not more attractive or healthier. When coupled with video recordings, increased pitch lowered perceived age of faces, but did not significantly influence perceived attractiveness, femininity, or health. Our results suggest that our manipulation of voice pitch has a measurable impact on judgements of femininity and age, but does not measurably influence vocal and facial attractiveness in naturalistic conditions.