Voice Cues Research Articles

Analysis of Voice Cues in Recognition of Sarcasm

Background: The voice recognition system is about cognizing the signals, by feature extraction and identification of related parameters. The entire process is referred to as voice analytics. Objective: The paper aims at analyzing and synthesizing the phonetics of voice. The work focuses on the facts of voice analytics i.e. basic blocks of ‘Glottal signature’. The glottal signature and unique voice cues are evaluated to derive the relationship for utterance of emotional words which leads to sentimental expression cues. An effort is made to map further to understand sarcasm behavior in the sounds made by human speech. Methods: The basic blocks of unique features identified in the work are Intensity, Pitch, Formants related to speak, read, interactive and declarative sentences solely on voice cues not on linguistic theory. It is also tested to identify derived features that maps to fine-grained details of voice cues to drill up usage in sarcasm detection. Results: Different unique features identified in the work are, intensity, pitch, formants related to read, speak, interactive and declarative sentences and derived parameters. Conclusion: The work carried out in the paper also supports the analysis of voice segmentation labelling, analyzes the unique features of voice cues, understanding physics of voice, the process is further carried out to recognize sarcasm.

Sarcasm Detection using Cognitive Features of Visual Data by Learning Model

The objective of the paper is to detect sarcasm in human communication. The methodology uses basic cognitive features of human utterances by capturing three modes of data viz., voice, text, and temporal facial features. The captured data is unstructured as it consists of parameters of feelings and emotions to generate sarcasm which affects expressions through glottal and facial organs. The data capturing method is equally challenging as compared to the method of data processing to acquire features. The significant work is aligned to make natural decisions in the prediction processes using cognitive information in the data lineage. Sarcasm detection in natural human communication is a challenging process. The Linguistic features of natural language processing (NLP) methods help identify sentiment as negative and positive sentences based on polarity using the pre-labelled samples. The multiclass neural network model is used as a soft cognition method for the detection of sarcasm under cloud resources. Identified cognitive features have information like voice cues and eye movements, they tend to influence the decision of detecting sarcasm. The visual data are found to be quite interesting and can establish a strong platform in the area of NLP for further research work.

Auditory Attention Deployment in Young Adults with Autism Spectrum Disorder.

Difficulty listening in noisy environments is a common complaint of individuals with autism spectrum disorder (ASD). However, the mechanisms underlying such auditory processing challenges are unknown. This preliminary study investigated auditory attention deployment in adults with ASD. Participants were instructed to maintain or switch attention between two simultaneous speech streams in three conditions: location (co-located versus ± 30° separation), voice (same voice versus male-female contrast), and both cues together. Results showed that individuals with ASD can selectively direct attention using location or voice cues, but performance was best when both cues were present. In comparison to neurotypical adults, overall performance was less accurate across all conditions. These findings warrant further investigation into auditory attention deployment differences in individuals with ASD.

Anxiety biases audiovisual processing of social signals

In everyday life, information from multiple senses is integrated for a holistic understanding of emotion. Despite evidence of atypical multisensory perception in populations with socio-emotional difficulties (e.g., autistic individuals), little research to date has examined how anxiety impacts on multisensory emotion perception. Here we examined whether the level of trait anxiety in a sample of 56 healthy adults affected audiovisual processing of emotion for three types of stimuli: dynamic faces and voices, body motion and dialogues of two interacting agents, and circles and tones. Participants judged emotion from four types of displays – audio-only, visual-only, audiovisual congruent (e.g., angry face and angry voice) and audiovisual incongruent (e.g., angry face and happy voice) – as happy or angry, as quickly as possible. In one task, participants based their emotional judgements on information in one modality while ignoring information in the other, and in a second task they based their judgements on their overall impressions of the stimuli. The results showed that the higher trait anxiety group prioritized the processing of angry cues when combining faces and voices that portrayed conflicting emotions. Individuals in this group were also more likely to benefit from combining congruent face and voice cues when recognizing anger. The multisensory effects of anxiety were found to be independent of the effects of autistic traits. The observed effects of trait anxiety on multisensory processing of emotion may serve to maintain anxiety by increasing sensitivity to social-threat and thus contributing to interpersonal difficulties.

Artificial intelligence enabled preliminary diagnosis for COVID-19 from voice cues and questionnaires.

The COVID-19 outbreak was announced as a global pandemic by the World Health Organization in March 2020 and has affected a growing number of people in the past few months. In this context, advanced artificial intelligence techniques are brought to the forefront as a response to the ongoing fight toward reducing the impact of this global health crisis. In this study, potential use-cases of intelligent speech analysis for COVID-19 identification are being developed. By analyzing speech recordings from COVID-19 positive and negative patients, we constructed audio- and symptomatic-based models to automatically categorize the health state of patients, whether they are COVID-19 positive or not. For this purpose, many acoustic features were established, and various machine learning algorithms are being utilized. Experiments show that an average accuracy of 80% was obtained estimating COVID-19 positive or negative, derived from multiple cough and vowel /a/ recordings, and an average accuracy of 83% was obtained estimating COVID-19 positive or negative patients by evaluating six symptomatic questions. We hope that this study can foster an extremely fast, low-cost, and convenient way to automatically detect the COVID-19 disease.

Development of discrimination and categorization of voice gender cues in school-age children

Children's sensitivity to differences in speakers' voices continues to develop throughout childhood, but it remains unclear how it develops. In a cross-sectional design, we investigated how school-age children's discrimination, a lower-level perceptual task, and categorization, a higher-level cognitive task, of voice gender cues, fundamental frequency (F0) and vocal-tract length (VTL), develop and how these abilities are related. We used an adaptive 3AFC procedure to measure children's F0 and VTL discrimination thresholds and a categorization task to examine their weighting of F0 and VTL for voice gender categorization. Here we show that the acquisition of adult-like performance for discrimination and categorization differs across voice gender cues. Children's discrimination thresholds were adult-like around 8 years of age for VTL but still differed from adults' at 12 years of age for F0. Contrarily, children's cue weights were adult-like around 6 years of age for F0 but around 10 years of age for VTL. Therefore, the two abilities seem to develop at different rates for the different voice cues and do not seem to be closely related. Data from cochlear implant children will also be presented, as for them F0 and VTL discrimination and gender categorization depend on both cognitive development and perceptual limitations.

The effects of word status and vocoding on voice cue perception

The effects of word status and vocoding on voice cue perception

Effect of Spectral Contrast Enhancement on Speech-on-Speech Intelligibility and Voice Cue Sensitivity in Cochlear Implant Users.

Speech intelligibility in the presence of a competing talker (speech-on-speech; SoS) presents more difficulties for cochlear implant (CI) users compared with normal-hearing listeners. A recent study implied that these difficulties may be related to CI users' low sensitivity to two fundamental voice cues, namely, the fundamental frequency (F0) and the vocal tract length (VTL) of the speaker. Because of the limited spectral resolution in the implant, important spectral cues carrying F0 and VTL information are expected to be distorted. This study aims to address two questions: (1) whether spectral contrast enhancement (SCE), previously shown to enhance CI users' speech intelligibility in the presence of steady state background noise, could also improve CI users' SoS intelligibility, and (2) whether such improvements in SoS from SCE processing are due to enhancements in CI users' sensitivity to F0 and VTL differences between the competing talkers. The effect of SCE on SoS intelligibility and comprehension was measured in two separate tasks in a sample of 14 CI users with Cochlear devices. In the first task, the CI users were asked to repeat the sentence spoken by the target speaker in the presence of a single competing talker. The competing talker was the same target speaker whose F0 and VTL were parametrically manipulated to obtain the different experimental conditions. SoS intelligibility, in terms of the percentage of correctly repeated words from the target sentence, was assessed using the standard advanced combination encoder (ACE) strategy and SCE for each voice condition. In the second task, SoS comprehension accuracy and response times were measured using the same experimental setup as in the first task, but with a different corpus. In the final task, CI users' sensitivity to F0 and VTL differences were measured for the ACE and SCE strategies. The benefit in F0 and VTL discrimination from SCE processing was evaluated with respect to the improvement in SoS perception from SCE. While SCE demonstrated the potential of improving SoS intelligibility in CI users, this effect appeared to stem from SCE improving the overall signal to noise ratio in SoS rather than improving the sensitivity to the underlying F0 and VTL differences. A second key finding of this study was that, contrary to what has been observed in a previous study for childlike voice manipulations, F0 and VTL manipulations of a reference female speaker (target speaker) toward male-like voices provided a small but significant release from masking for the CI users tested. The present findings, together with those previously reported in the literature, indicate that SCE could serve as a possible background-noise-reduction strategy in commercial CI speech processors that could enhance speech intelligibility especially in the presence of background talkers that have longer VTLs compared with the target speaker.

Beyond Correlation: Acoustic Transformation Methods for the Experimental Study of Emotional Voice and Speech

While acoustic analysis methods have become a commodity in voice emotion research, experiments that attempt not only to describe but to computationally manipulate expressive cues in emotional voice and speech have remained relatively rare. We give here a nontechnical overview of voice-transformation techniques from the audio signal-processing community that we believe are ripe for adoption in this context. We provide sound examples of what they can achieve, examples of experimental questions for which they can be used, and links to open-source implementations. We point at a number of methodological properties of these algorithms, such as being specific, parametric, exhaustive, and real-time, and describe the new possibilities that these open for the experimental study of the emotional voice.

Speeded reasoning moderates the inverse relationship between autistic traits and emotion recognition.

Autistic people typically have difficulty recognizing other people's emotions and to process nonverbal cues in an automatic, intuitive fashion. This usually also applies to people who-regardless of an official diagnosis of autism-achieve high values in autism questionnaires. However, some autistic people do not seem to have any problems with emotion recognition. One explanation may be that these individuals are able to compensate for their lack of intuitive or automatic processing through a quick conscious and deliberate analysis of the emotional cues in faces, voices, and body movements. On these grounds, we assumed that the higher autistic people's ability to reason quickly (i.e. to make quick logical inferences), the fewer problems they should have with determining other people's emotions. In our study, we asked workers on the crowdsourcing marketplace MTurk to complete a questionnaire about their autistic traits, to perform emotion recognition tests, and to complete a test of the ability to reason under time constraints. In our sample of 217 people, we found the expected pattern. Overall, those who had higher values in the autism questionnaire scored lower in the emotion recognition tests. However, when reasoning ability was taken into account, a more nuanced picture emerged: participants with high values both on the autism questionnaire and on the reasoning test recognized emotions as well as individuals with low autistic traits. Our results suggest that fast analytic information processing may help autistic people to compensate problems in recognizing others' emotions.

Brainstem and early cortical auditory activities associated with language differences in acoustic cue weighting for voicing perception

The purpose of the present study was to identify neural activities reflecting language differences in acoustic cue weighting for phonetic perception. We examined whether brainstem and early cortical auditory responses to voice-onset time (VOT) and onset frequency of fundamental frequency (onset F0) were different between two groups of listeners, native Japanese speakers and Korean second-language learners of Japanese, who use a VOT and onset F0, respectively, as the primary perceptual cue for voicing. In the experiment, we measured auditory brainstem responses (ABRs) and cortical N1 response of the Japanese and Korean participants while listening to stop-consonant stimuli varying in VOT and onset F0 (10 ms and 80 Hz, 30 ms and 80 Hz, or 30 ms and 120 Hz). We found that the N1 response was much more sensitive to VOT distinction in the Japanese than Korean participants, although the distinction of onset F0 was not reflected in early cortical responses in either language groups. There was no obvious difference in the ABRs between the Japanese and Korean participants. These results suggest that early cortical auditory activity is related to the processing of acoustic cue weighting for phonetic perception, while brainstem auditory activity is stimulus-dependent.

Five members of a mixed-sex group of bottlenose dolphins share a stereotyped whistle contour in addition to maintaining their individually distinctive signature whistles.

Most commonly, animal communication systems are driven by shared call repertoires, with some individual distinctiveness encoded as a byproduct of voice cues. We provide evidence that bottlenose dolphins produce both individually distinctive whistles, and a shared whistle type. A stereotyped whistle contour (termed the group whistle) is shared by five bottlenose dolphins that have lived, worked, and traveled together for at least 21 years. These five dolphins are members of a group of eight dolphins that work as a specialized team for the Navy Marine Mammal Program. Each dolphin is routinely recorded during periods when an individual is isolated from the others in above ground pools as part of their routine training. Each of the eight dolphins has an individually distinctive signature whistle. In addition, at least five of these dolphins share a distinct non-signature whistle type. This shared whistle contour was produced an average of 22.4% +/- 9.0% of the time during periods in which individuals were isolated. During these isolations the signature whistle was produced an average of 42.9% +/- 11.9% of the time. This is consistent with decades of signature whistle research. A group of 10 naïve observers rated the similarity of the different whistle contours. The observers rated the group whistle contour produced by all five dolphins as highly similar (P < 0.01). Their ratings further showed that the signature whistles of the five dolphins were very different (P < 0.01). These findings were further supported by discriminant function analyses. That said, the shared whistle contours still exhibited individual differences which may allow conspecifics to identify the producer even when a whistle contour is shared among multiple dolphins. This is the first in-depth analysis of a non-signature whistle type shared among multiple conspecifics.

Does speaker’s voice enthusiasm affect social cue, cognitive load and transfer in multimedia learning?

PurposeThis study aims to examine the effects of voice enthusiasm (enthusiastic voice vs calm voice) on social ratings of the speaker, cognitive load and transfer performance in multimedia learning.Design/methodology/approachTwo laboratory experiments were conducted in which learners learned from a multimedia presentation about computer algorithm that was narrated by either an enthusiastic human voice or a calm human voice.FindingsResults from Experiment 1 revealed that the enthusiastic voice narration led to higher social ratings of the speaker and transfer performance when compared to the calm voice narration. Experiment 2 demonstrated that the enthusiastic voice led to higher affective social ratings (human-like and engaging) and transfer performance as compared to the calm voice. Moreover, it was shown that a calm voice prompted a higher germane load than an enthusiastic voice, which conforms to the argument that prosodic cues in voice can influence processing in multimedia learning among non-native speakers.Originality/valueThis study extends from prior studies that examined voice effects related to mechanization, accent, dialect, and slang in multimedia learning to examining the effects of voice enthusiasm in multimedia learning.

Effects of speech cues in French-speaking children with dysarthria.

Articulatory excursion and vocal intensity are reduced in many children with dysarthria due to cerebral palsy (CP), contributing to the children's intelligibility deficits and negatively affecting their social participation. However, the effects of speech-treatment strategies for improving intelligibility in this population are understudied, especially for children who speak languages other than English. In a cueing study on English-speaking children with dysarthria, acoustic variables and intelligibility improved when the children were provided with cues aimed to increase articulatory excursion and vocal intensity. While French is among the top 20 most spoken languages in the world, dysarthria and its management in French-speaking children are virtually unexplored areas of research. Information gleaned from such research is critical for providing an evidence base on which to provide treatment. To examine acoustic and perceptual changes in the speech of French-speaking children with dysarthria, who are provided with speech cues targeting greater articulatory excursion (French translation of 'speak with your big mouth') and vocal intensity (French translation of 'speak with your strong voice'). This study investigated whether, in response to the cues, the children would make acoustic changes and listeners would perceive the children's speech as more intelligible. Eleven children with dysarthria due to CP (six girls, five boys; ages 4;11-17;0 years; eight with spastic CP, three with dyskinetic CP) repeated pre-recorded speech stimuli across three speaking conditions (habitual, 'big mouth' and 'strong voice'). Stimuli were sentences and contrastive words in phrases. Acoustic analyses were conducted. A total of 66 Belgian-French listeners transcribed the children's utterances orthographically and rated their ease of understanding on a visual analogue scale at sentence and word levels. Acoustic analyses revealed significantly longer duration in response to the big mouth cue at sentence level and in response to both the big mouth and strong voice cues at word level. Significantly higher vocal sound-pressure levels were found following both cues at sentence and word levels. Both cues elicited significantly higher first-formant vowel frequencies and listeners' greater ease-of-understanding ratings at word level. Increases in the percentage of words transcribed correctly and in sentence ease-of-understanding ratings, however, did not reach statistical significance. Considerable variability between children was observed. Speech cues targeting greater articulatory excursion and vocal intensity yield significant acoustic changes in French-speaking children with dysarthria. However, the changes may only aid listeners' ease of understanding at word level. The significant findings and great inter-speaker variability are generally consistent with studies on English-speaking children with dysarthria, although changes appear more constrained in these French-speaking children. What this paper adds What is already known on the subject According to the only study comparing effects of speech-cueing strategies on English-speaking children with dysarthria, intelligibility increases when the children are provided with cues aimed to increase articulatory excursion and vocal intensity. Little is known about speech characteristics in French-speaking children with dysarthria and no published research has explored effects of cueing strategies in this population. What this paper adds to existing knowledge This paper is the first study to examine the effects of speech cues on the acoustics and intelligibility of French-speaking children with CP. It provides evidence that the children can make use of cues to modify their speech, although the changes may only aid listeners' ease of understanding at word level. What are the potential or actual clinical implications of this work? For clinicians, the findings suggest that speech cues emphasizing increasing articulatory excursion and vocal intensity show promise for improving the ease of understanding of words produced by francophone children with dysarthria, although improvements may be modest. The variability in the responses also suggests that this population may benefit from a combination of such cues to produce words that are easier to understand.

Individual differences in human voice pitch are preserved from speech to screams, roars and pain cries.

Fundamental frequency (F0, perceived as voice pitch) predicts sex and age, hormonal status, mating success and a range of social traits, and thus functions as an important biosocial marker in modal speech. Yet, the role of F0 in human nonverbal vocalizations remains unclear, and given considerable variability in F0 across call types, it is not known whether F0 cues to vocalizer attributes are shared across speech and nonverbal vocalizations. Here, using a corpus of vocal sounds from 51 men and women, we examined whether individual differences in F0 are retained across neutral speech, valenced speech and nonverbal vocalizations (screams, roars and pain cries). Acoustic analyses revealed substantial variability in F0 across vocal types, with mean F0 increasing as much as 10-fold in screams compared to speech in the same individual. Despite these extreme pitch differences, sexual dimorphism was preserved within call types and, critically, inter-individual differences in F0 correlated across vocal types (r = 0.36–0.80) with stronger relationships between vocal types of the same valence (e.g. 38% of the variance in roar F0 was predicted by aggressive speech F0). Our results indicate that biologically and socially relevant indexical cues in the human voice are preserved in simulated valenced speech and vocalizations, including vocalizations characterized by extreme F0 modulation, suggesting that voice pitch may function as a reliable individual and biosocial marker across disparate communication contexts.

Analysing picking errors in vision picking systems

Abstract Vision picking empowers users with access to real-time digital order information, while freeing them from handheld radio frequency devices. The smart glasses, as an example of vision picking enabler, provide visual and voice cues to guide order pickers. The glasses mostly also have installed navigation features that can sense the order picker’s position in the warehouse. This paper explores picking errors in vision systems with literature review and experimental work in laboratory environment. The results show the effectiveness of applying vision picking systems for the purposes of active error prevention, when they are compared to established methods, such as paper-picking and using cart mounted displays. A serious competitor to vision picking systems are pick-to-light systems. The strong advantage of vision picking system is that most of the errors are detected early in the process and not at the customer’s site. The cost of fixing the error is thus minimal. Most errors consequently directly influence order picker’s productivity in negative sense. Nonetheless, the distinctive feature of the system is extremely efficient error detection.

Voice fundamental frequency differences and speech recognition with noise and speech maskers in cochlear implant recipients.

Cochlear implant (CI) recipients are limited in their perception of voice cues, such as the fundamental frequency (F0). This has important consequences for speech recognition when several talkers speak simultaneously. This examination considered the comparison of clear speech and noise-vocoded sentences as maskers. For the speech maskers it could be shown that good CI performers are able to benefit from F0 differences between target and masker. This was due to the fact that a F0 difference of 80 Hz significantly reduced target-masker confusions, an effect that was slightly more pronounced in bimodal than in bilateral users.

The Role of Face and Voice Cues in Predicting the Outcome of Student Representative Elections.

First impressions formed after seeing someone's face or hearing their voice can affect many social decisions, including voting in political elections. Despite the many studies investigating the independent contribution of face and voice cues to electoral success, their integration is still not well understood. Here, we examine a novel electoral context, student representative ballots, allowing us to test the generalizability of previous studies. We also examine the independent contributions of visual, auditory, and audiovisual information to social judgments of the candidates, and their relationship to election outcomes. Results showed that perceived trustworthiness was the only trait significantly related to election success. These findings contrast with previous reports on the importance of perceived competence using audio or visual cues only in the context of national political elections. The present study highlights the role of real-world context and emphasizes the importance of using ecologically valid stimulus presentation in understanding real-life social judgment.

Digital Communication in Psychoanalysis: An Oxymoron?

Texts and e-mails can create treatment dilemmas. This is especially the case when impulsively employed by patient and/or therapist, constituting an enactment. The opportunities for miscommunication are rife because of the lack of opportunity to converse in person and because tone of voice, body language, and other nonverbal cues are missing. They can also have a depersonalizing effect on sender and receiver. This article addresses how these problems occur and ways to deal with them.

Self-rated Attractiveness and Sociosexual Behavior Predict Gay Men's Preferences for Masculine Cues in Male Voices in China.

Theory and some evidence suggest that masculine characteristics (e.g., lower pitched voice) will predict indices of men's long-term health. In a related finding, researchers have demonstrated that the attractiveness of individuals predicts variation in their mate preferences. As self-perceived attractiveness may predict sociosexuality (i.e., the willingness of individuals to engage in restricted or unrestricted sexual relationships), we investigated how self-rated attractiveness and sociosexuality influence preferences regarding male voices among 338 gay men across different cities in China. Each participant was randomly presented with six pairs of male voices and gave preferences using a forced-choice method; each pair of male voices consisted of a masculine (lower pitched) and feminine (higher pitched) version of the same original voice. Our findings suggested that gay men who perceived themselves as more attractive showed stronger preference for lower pitched voices compared with self-perceived less attractive individuals. In addition, we found that gay men's sociosexuality score was positively correlated with their preference for masculine cues in male voices, indicating that gay men who were less sociosexually restricted preferred lower pitched voices over higher pitched versions compared with men who were more restricted. Our study presents evidence contributing to a better understanding of condition-dependent strategies of partner choice in gay men.