Vocal Expressions Research Articles

A Cortical Site that Encodes Vocal Expression and Reception.

Socially effective vocal communication requires brain regions that encode expressive and receptive aspects of vocal communication in a social context-dependent manner. Here, we combined a novel behavioral assay with microendoscopy to interrogate neuronal activity in the posterior insula (pIns) in socially interacting mice as they switched rapidly between states of vocal expression and reception. We found that distinct but spatially intermingled subsets of pIns neurons were active during vocal expression and reception. Notably, pIns activity during vocal expression increased prior to vocal onset and was also detected in congenitally deaf mice, pointing to a motor signal. Furthermore, receptive pIns activity depended strongly on social cues, including female odorants. Lastly, tracing experiments reveal that deep layer neurons in the pIns directly bridge the auditory thalamus to a midbrain vocal gating region. Therefore, the pIns is a site that encodes vocal expression and reception in a manner that depends on social context.

Young sanctuary-living chimpanzees produce more communicative expressions with artificial objects than with natural objects

In humans, interactions with objects are often embedded in communicative exchanges. Objects offer unique affordances to explore, carry functions and hold cultural relevance, which can shape children’s interactions and communication. Research indicates that the use of artificial objects, such as certain toys, helps promote pre-linguistic communication, consequently impacting language development. Given that chimpanzees use objects extensively compared to other great apes, and considering the differences between chimpanzees and bonobos in intrinsic motivation for tool use and the extended developmental period during which they learn to use objects, it is reasonable to expect that objects may influence chimpanzees’ communication. Here, we examined interactions of 31 immature sanctuary-living chimpanzees with non-novel artificial and natural objects and tested their vocal and facial expressions, applying methods previously designed for children. Our results showed an increase in these expressions associated with artificial objects. These findings provide the first empirical evidence that chimpanzee communicative expressions may be influenced by inherent properties of objects, potentially promoting varied communication, comparable to the impact distinctive objects have on pre-linguistic children. By exploring this connection between object-centric interactions and communication, this study reveals deep phylogenetic roots where objects may have shaped great ape communication and possibly evolutionary foundations of language.

The Arduous and the Ordinary: Embodying the Conversational Voice in the Long Nineteenth Century

This article aims to chart out a cultural history of the voice in conversation in the nineteenth century. Often understood as a period of decline of the ‘art’ of conversation, as it was defined in the eighteenth century (and studied by historians of the period), the nineteenth century was witness to a shift away from conscious cultivation. Conversation, in the nineteenth century, became the counterpoint to more formalized modes of speech like oratory and debate, and was defined as ‘natural’. As such, it also became an object of study for experts specializing in vocal health, who saw this ‘natural’ form of vocal expression as a place to see the voice in action at its least artificial. Based on the work of these experts – scientific treatises, early forays in laryngoscopy, educational work and self-help manuals – I propose an analysis of their largely normative texts that focuses on sound and vocal practice, rather than the development of scientific discourse and its disciplinary character. I borrow central concepts from sound studies, like that of the audile technique and aurality in order to do so, and place the vocalizing body at the center of this analysis.

How to Express Boredom via Intonation in Kurdish with Special Reference to English

The present study examines the contribution of intonation to the vocal expressions of boredom in utterances. It begins by providing a theoretical overview of the role intonation plays in conveying emotions or attitudes. Further, it outlines the tonetic aspects, pitch characteristics, and other prosodic factors that participate in conveying boredom in English depending on the literature available. The study then turns to analysing sixteen tone groups taken from four Kurdish films and recorded materials of everyday spoken Kurdish. It tests and underscores the tonetic aspects and pitch characteristics of these chosen tone groups used for the expression of boredom, relying on six parameters, namely the syntactic category, the height of the pre-head, the height of the head, the tone used, the pitch range, and the key. The role of all these intonation factors is investigated. Special reference is also made to other factors such as loudness, speed of speaking, and paralinguistic features. The work shows that the cues used to convey boredom vary, although pitch and pitch variation become dominant and central. Among all the intonation cues used to express boredom, pitch range, and tone are the most relevant and useful cues.

Physiological and communicative emotional disconcordance in children on the autism spectrum

BackgroundIndividuals on the autism spectrum commonly have differences from non-autistic people in expressing their emotions using communicative behaviors, such as facial expressions. However, it is not yet clear if this reduced expressivity stems from reduced physiological reactivity in emotional contexts or if individuals react internally, but do not show these reactions externally to others. We hypothesized that autism is characterized by a discordance between in-the-moment internal psychophysiological arousal and external communicative expressions of emotion.MethodsForty-one children on the autism spectrum and 39 non-autistic, typically developing (TD) children of two age groups (2–4 and 8–12 years) participated in a low-level stress task whilst wearing a wireless electrocardiogram. Children’s negative emotional expressions (facial, vocal, bodily) were coded following standardized protocols. Alexithymia traits were assessed using the Children’s Alexithymia Measure with school-aged children only. Data analyses involved ANOVAs, correlations, and sensitivity analyses.ResultsThere were no group differences in physiological arousal (heart rate) or in communicative expressions of stress to the stress task. For TD preschoolers, physiological arousal during the stress task was associated with vocal expressions and for TD school-aged children, they were associated with facial and bodily expressions. By contrast, for children on the autism spectrum, physiological arousal during the stress tasks was not associated with communicative expressions across age groups.ConclusionsOur findings suggest that children on the autism spectrum might experience emotional disconcordance, in that their physiological arousal does not align with their communicative expressions. Therefore, the internally experienced stress of children on the autism spectrum may be inadvertently missed by teachers and caregivers and, consequently, learning opportunities for teaching emotional communication and regulation may be also missed. Our results support the use of wearable biosensors to facilitate such interventions in children on the autism spectrum.

RF‐GCN: Residual fused‐graph convolutional network using multimodalities for facial emotion recognition

AbstractBackgroundThe emotional state of individuals is difficult to identify and it is developing now a days because of vast interest in recognition. Many technologies have been developed to identify this emotional expression based on facial expressions, vocal expressions, physiological signals, and body expressions. Among these, facial emotion is very expressive for recognition using multimodalities. Understanding facial emotions has applications in mental well‐being, decision‐making, and even social change, as emotions play a crucial role in our lives. This recognition is complicated by the high dimensionality of data and non‐linear interactions across modalities. Moreover, the way emotion is expressed by people varies and these feature identification remains challenging, where these limitations are overcome by Deep learning models.MethodsThis research work aims at facial emotion recognition through the utilization of a deep learning model, named the proposed Residual Fused‐Graph Convolution Network (RF‐GCN). Here, multimodal data included is video as well as an Electroencephalogram (EEG) signal. Also, the Non‐Local Means (NLM) filter is used for pre‐processing input video frames. Here, the feature selection process is carried out using chi‐square, after feature extraction, which is done in both pre‐processed video frames and input EEG signals. Finally, facial emotion recognition and its types are determined by RF‐GCN, which is a combination of both the Deep Residual Network (DRN) and Graph Convolutional Network (GCN).ResultsFurther, RF‐GCN is evaluated for performance by metrics such as accuracy, recall, and precision, with superior values of 91.6%, 96.5%, and 94.7%.ConclusionsRF‐GCN captures the nuanced relationships between different emotional states and improves recognition accuracy. The model is trained and evaluated on the dataset and reflects real‐world conditions.

Multimodal analysis of temporal affective variability within treatment for depression.

Affective flexibility, the capacity to respond to life's varying environmental changes in a dynamic and adaptive manner, is considered a central aspect of psychological health in many psychotherapeutic approaches. The present study examined whether affective two-dimensional (i.e., arousal and valence) temporal variability extracted from voice and facial expressions would be associated with positive changes over the course of psychotherapy, at the session, client, and treatment levels. A total of 22,741 mean vocal arousal and facial expression valence observations were extracted from 137 therapy sessions in a sample of 30 clients treated for major depressive disorder by nine therapists. Before and after each session, the clients self-reported their level of well-being on the outcome rating scale. Session-level affective temporal variability was assessed as the mean square of successive differences between consecutive two-dimensional affective measures. Session outcome was positively associated with temporal variability at the session level (i.e., within clients, between sessions) and at the client level (i.e., between clients). Importantly, these associations held when controlling for average session- and client-level valence scores. In addition, the expansion of temporal variability throughout treatment was associated with steeper positive session outcome trajectories over the course of treatment. The continuous assessment of both vocal and facial affective expressions and the ability to extract measures of affective temporal variability from within-session data may enable therapists to better respond and modulate clients' affective flexibility; however, further research is necessary to determine whether there is a causal link between affective temporal variability and psychotherapy outcomes. (PsycInfo Database Record (c) 2024 APA, all rights reserved).

Romances by Alexei Stârcea on the lyrics of Anatol Gujel: imaginary structure, musical language, interpretative tasks

Romances of Alexei Stârcea written on the lyrics of Anatol Gujel, are analyzed from the point of view of poetic content, the interaction of the music with the literary text, musical language, and vocal performance. The theme of the romances Noaptea pe lac, Iluzii and Te aștept is associated with love lyrics, which in the poetry of A. Gujel is closely related to images of nature. The romance Struguri de Moldova glorifies the beauty of the native land. All the four romances were written by A. Stârcea in the second period of his activity (1957–1968) and reflect some features specific to this decade, such as the interest in the work of Moldovan poets, the in-depth development of national themes, the reflection of folklore elements in the musical language, the emergence of a genre symbiosis of romances and songs, the further researches in the field of vocal expressiveness, the dialogic interaction between the vocal and piano parts.

Encoding Emotion

The paper Encoding Emotion details an experiment conducted on early 18th-century French recitative to determine how composers relate harmony to sung text, if at all. Although the existence of a relationship between harmony and vocal expression is generally agreed upon, it is rarely properly taught to baroque singers, even though harmony is fundamental to the concept of basso continuo. It is also rarely discussed in continuo or singing treatises, either historical or modern, and, if so, never explicitly. The present experiment aims to fill that gap by examining harmony and text in actual written music. To do so, a harmonic analysis was performed on excerpts of secco recitative from French operas and cantatas written by Campra, Clérambault, Jacquet de la Guerre, and Rameau; the significant chords and harmonic passages were then logged in a spreadsheet with their corresponding text. Over two hundred lines of results were then analysed to determine whether patterns could be found associating certain words to chords or harmonic progressions. In the end, the dataset was too small for any general conclusions to be made with certainty, but patterns still emerged. Most notably, the plagal cadence correlated to feelings of love and tenderness, particularly in a sacred or noble context. It was also noted that, though general patterns could not always be identified, some chords or harmonic progressions appeared to have specific meaning for a certain composer. The V of IV, for example, was strongly associated with pleasure in Clérambault's works. Although there is only limited use for these results outside of this study, the identified patterns show that a more extensive dataset would yield enlightening results and give singers access to a remarkably rich tool with which to inform their interpretation.

Persistent vocal learning in an aging open-ended learner reflected in neural FoxP2 expression

BackgroundMost vocal learning species exhibit an early critical period during which their vocal control neural circuitry facilitates the acquisition of new vocalizations. Some taxa, most notably humans and parrots, retain some degree of neurobehavioral plasticity throughout adulthood, but both the extent of this plasticity and the neurogenetic mechanisms underlying it remain unclear. Differential expression of the transcription factor FoxP2 in both songbird and parrot vocal control nuclei has been identified previously as a key pattern facilitating vocal learning. We hypothesize that the resilience of vocal learning to cognitive decline in open-ended learners will be reflected in an absence of age-related changes in neural FoxP2 expression. We tested this hypothesis in the budgerigar (Melopsittacus undulatus), a small gregarious parrot in which adults converge on shared call types in response to shifts in group membership. We formed novel flocks of 4 previously unfamiliar males belonging to the same age class, either “young adult” (6 mo − 1 year) or “older adult” (≥ 3 year), and then collected audio-recordings over a 20-day learning period to assess vocal learning ability. Following behavioral recording, immunohistochemistry was performed on collected neural tissue to measure FoxP2 protein expression in a parrot vocal learning center, the magnocellular nucleus of the medial striatum (MMSt), and its adjacent striatum.ResultsAlthough older adults show lower vocal diversity (i.e. repertoire size) and higher absolute levels of FoxP2 in the MMSt than young adults, we find similarly persistent downregulation of FoxP2 and equivalent vocal plasticity and vocal convergence in the two age cohorts. No relationship between individual variation in vocal learning measures and FoxP2 expression was detected.ConclusionsWe find neural evidence to support persistent vocal learning in the budgerigar, suggesting resilience to aging in the open-ended learning program of this species. The lack of a significant relationship between FoxP2 expression and individual variability in vocal learning performance suggests that other neurogenetic mechanisms could also regulate this complex behavior.

Principal dimensions of voice production and their role in vocal expression.

How we produce and perceive voice is constrained by laryngeal physiology and biomechanics. Such constraints may present themselves as principal dimensions in the voice outcome space that are shared among speakers. This study attempts to identify such principal dimensions in the voice outcome space and the underlying laryngeal control mechanisms in a three-dimensional computational model of voice production. A large-scale voice simulation was performed with parametric variations in vocal fold geometry and stiffness, glottal gap, vocal tract shape, and subglottal pressure. Principal component analysis was applied to data combining both the physiological control parameters and voice outcome measures. The results showed three dominant dimensions accounting for at least 50% of the total variance. The first two dimensions describe respiratory-laryngeal coordination in controlling the energy balance between low- and high-frequency harmonics in the produced voice, and the third dimension describes control of the fundamental frequency. The dominance of these three dimensions suggests that voice changes along these principal dimensions are likely to be more consistently produced and perceived by most speakers than other voice changes, and thus are more likely to have emerged during evolution and be used to convey important personal information, such as emotion and larynx size.

A lightweight and privacy preserved federated learning ecosystem for analyzing verbal communication emotions in identical and non-identical databases

The lack of vocal emotional expression is a major deficit in social communication disorders. The current scenario of artificial intelligence focuses on collaborative training of deep learning models without losing data privacy. The primary objective of this paper is to propose a federated learning-based classification model to identify and analyze the emotional capabilities of individuals with vocal emotion deficits. The methodology has developed a collaborative and privacy-preserved approach using federated learning for training the deep learning models. The proposed methodology utilizes Mel-frequency Cepstral Coefficients (MFCC) to preprocess audio recordings. The four datasets (RAVDESS, CREMA, TESS, SAVEE) including emotion-based classified audio recordings were collected from open sources. The collected audio recordings are 3 s each and the total data set has 668376 audio files with happy - 175119 files, sad – 172611 files, angry – 176346 files, and normal - 144300 files. Further, the input audio was pre-processed to generate MFCC features. The study began with extracting features from multiple pre-trained DL models as its base model. Then, the performance of the federated learning (FL) model was tested on independent and identically distributed (IID) and non-IID data. Further, this paper presents a federated deep learning-based multimodal system for verbal communication emotions classification that uses audio datasets to meet data privacy requirements by DL on the FL ecosystem. As per the findings, the federated learning trained model provides nearly similar parametric results in comparison to base model training. For IID data, the model had 99.71 % validation accuracy, precision (99.73 %), recall (99.69 %), and validation loss (0.01). The FL architecture with non-IID data outperformed these measures with validation accuracy (99.97 %), precision (99.97 %), recall (99.97 %), and least loss (0). Hence the acquired results support the utilization of federated learning ecosystem-based trained models with identically and non-identically distributed audio features from emotion identification without losing parametric results. In conclusion, the proposed techniques could be applied to identify verbal emotional deficits in individuals and could support developing emerging technological interventions for their well-being.

The evolution of musicality and cross-domain co-evolutionary interactions

The evolution of human musicality has often been linked to the evolution of the faculty of language since the development of musical and linguistic abilities seems to share a common phase in their ontogenesis. Apart from that, both singing and speaking are, on the one hand, universal forms of human vocal expression and, on the other hand, consist of culturally specific elements. Such a probable co-occurrence of the predisposition to speak and sing, with the cultural variability of both these forms of communication, has prompted researchers to indicate gene–culture co-evolution as the probable mechanism responsible for the emergence of human musicality and the faculty of language. However, in most evolutionary scenarios proposed so far, the evolutionary paths of music and language followed independently after divergence from a common precursor. This article, based on observations of contemporary interactions between language and music, presents a different view in which musical and language-like forms of proto-communication interacted leading to the repurposing of some of their neural mechanisms. In this process, the Baldwinian interplay between plasticity and canalization has been proposed as the most probable evolutionary mechanism that shaped our musicality. The premises that support the presence of cross-domain co-evolutionary interactions in the contemporary communicative niche of Homo sapiens are indicated.

Relación entre meditación, regulación emocional y expresión vocal

Introduction: The voice is an indicator of emotional states, influenced by factors such as vagal tone, breathing and heart rate variability. This study explores these factors and their relationship with emotional regulation and meditative practice as a self-regulation technique. Purpose: To investigate the difference in vocal characteristics and heart rate variability in experienced (EM) and novice (NM) meditators before and after a meditation practice and in non-meditators - control group (CG), before and after a control test. Methods: 3 x 2 quasi-factorial study. Three groups were evaluated (experienced meditators EM; novice meditators NM; and control group CG, non-meditators) at two points in the experimental manipulation - before and after a meditation session for meditators, and before and after a word search task for the control group. The fundamental frequency, jitter, shimmer, harmonic-to-noise ratio and the first (F1), second (F2) and third (F3) formants of the vowel [a]; heart rate variation (SDNN, RMSSD, LF/HF, SD1 and SD2); anxiety state and vocal self-perception, were investigated, before and after the intervention. Results: The EM group achieved optimal vocal tract relaxation. The NM and CG groups showed changes in F1. Long-term meditative practice was associated with a large difference in F3, SDNN and SD2 in heart rate variation. Conclusion: The results suggest that meditation practice influences vocal expression and emotional reaction, and that experience in meditation practice favors this relationship.

Text and Sound-Based Feature Extraction and Speech Emotion Classification for Korean

Embracing the complexities of human emotions conveyed through speech, this study ventures into Speech Emotion Recognition (SER) within the human-computer interaction domain, leveraging cutting-edge artificial intelligence technologies. Focusing on the auditory attributes of speech, such as tone, pitch, and rhythm, the research introduces an innovative approach that amalgamates deep learning techniques with the A Learnable Frontend for Audio Classification (LEAF) algorithm and wav2vec 2.0 pre-trained on a large corpus, specifically targeting Korean voice samples. This methodology underlines the capacity of these technologies to process and decipher complex vocal expressions, aiming to elevate emotion classification precision notably. The exploration extends the horizons of SER by accentuating auditory emotion cues and aspires to enrich machine interactions to be more intuitive and empathetic across various applications like healthcare and customer service. The outcomes underscore the efficacy of transformer-based models, particularly wav2vec 2.0 and LEAF, in capturing the subtle emotional states expressed in speech, thereby affirming the importance of auditory cues over conventional visual and textual indicators. The study's implications for further research herald a promising trajectory for evolving AI systems adept at nuanced emotion detection, thereby forging pathways toward more natural and human-centric interactions between individuals and machines. This advancement is crucial for developing empathetic AI that can seamlessly integrate into our daily lives, understanding and reacting to human emotions in a way that mirrors human understanding and compassion.

Voice Lessons

Abstract This paper reflects on the national referendum for an Indigenous Voice to Parliament that took place in Australia in mid-October 2023. At the time of writing, the aftershocks from the failure of the referendum to gain the necessary majorities were still being felt keenly by many of the Voice advocates and supporters. The hurt and grief of many First Nations people were shared by millions of non-Indigenous “Yes” voters, while much reckoning continued in the subsequent weeks and months. The author here explores what might have been gained if more attention had been given to what an Indigenous Voice to Parliament might “sound like,” instead of the excessive focus on, and public discourse around what it might “look like.” Resources from the philosophies and physiology of voice, communication ethics, cultural studies, critical anthropology, Australian Indigenous writing and scholarship, and psychoanalytic politics are utilised to explore the connections between the human voice, vocal expression, hearing and listening, silence and song.

Depression recognition over fusion of visual and vocal expression using artificial intelligence

<div class="WordSection1"><p>Depression is a mental illness that usually goes untreated in people and can have catastrophic consequences, including suicidal thoughts. Counselling services are widely available, but because depression is a stigmatized illness, many people who are depressed decide not to seek help. Therefore, it is essential to develop an automated system that can recognize depression in individuals before it worsens. In this study, a novel approach is proposed for identifying depression using a combination of visual and vocal emotions. Long short-term memory (LSTM) is used to assess verbal expressions and convolutional neural networks (CNN) to analyze facial expressions. The proposed system is trained using features of depression from the distress analysis interview corpus (DAIC) dataset and tested on videos of college students with frontal faces. The proposed approach is effective in detecting depression in individuals, with high accuracy and reliability.</p></div>

ARTIFICIAL INTELLIGENT SYSTEM FOR AUTOMATIC DEPRESSION LEVEL ANALYSIS THROUGH VISUAL AND VOCAL EXPRESSIONS

Depression is a major mental health disease of human which is rapidly affecting lives worldwide. Early detection and intervention are crucial for effective treatment and management. Depression is regularly identified with thoughts of suicide. Significant depression can bring about a determination of social and physical side effects. It could remember changes in rest, craving, vitality level, concentration, day by day conduct or confidence. In recent years, deep-learned applications concentrated on neural networks have shown superior performance at hand-crafted apps in various areas. This system presents an innovative artificial intelligence (AI) system designed for automatic depression level analysis by analyzing visual and vocal expressions. Leveraging advances in computer vision and natural language processing, the proposed system extracts relevant features from facial expressions and speech patterns to assess depression severity levels accurately. Deep-learned apps that settle the above issues that may precisely assess the degree of voice and face depression. In the proposed method, Convolutionary Neural Networks (CNN) is developed for learning deep-learned features and descriptive raw waveforms for visual expressions. Second. Keywords: Artificial Intelligence, Depression, Vocal, Facial Expressions, Deep Learning

Maternal childhood emotional abuse increases cardiovascular responses to children's emotional facial expressions.

Parents with a history of childhood maltreatment may be more likely to respond inadequately to their child's emotional cues, such as crying or screaming, due to previous exposure to prolonged stress. While studies have investigated parents' physiological reactions to their children's vocal expressions of emotions, less attention has been given to their responses when perceiving children's facial expressions of emotions. The present study aimed to determine if viewing facial expressions of emotions in children induces cardiovascular changes in mothers (hypo- or hyper-arousal) and whether these differ as a function of childhood maltreatment. A total of 104 mothers took part in this study. Their experiences of childhood maltreatment were measured using the Childhood Trauma Questionnaire (CTQ). Participants' electrocardiogram signals were recorded during a task in which they viewed a landscape video (baseline) and images of children's faces expressing different intensities of emotion. Heart rate variability (HRV) was extracted from the recordings as an indicator of parasympathetic reactivity. Participants presented two profiles: one group of mothers had a decreased HRV when presented with images of children's facial expressions of emotions, while the other group's HRV increased. However, HRV change was not significantly different between the two groups. The interaction between HRV groups and the severity of maltreatment experienced was marginal. Results suggested that experiences of childhood emotional abuse were more common in mothers whose HRV increased during the task. Therefore, more severe childhood experiences of emotional abuse could be associated with mothers' cardiovascular hyperreactivity. Maladaptive cardiovascular responses could have a ripple effect, influencing how mothers react to their children's facial expressions of emotions. That reaction could affect the quality of their interaction with their child. Providing interventions that help parents regulate their physiological and behavioral responses to stress might be helpful, especially if they have experienced childhood maltreatment.

An Interactive Feeder to Induce and Assess Emotions from Vocalisations of Chickens.

Understanding the emotional states of animals is a long-standing research endeavour that has clear applications in animal welfare. Vocalisations are emerging as a promising way to assess both positive and negative emotional states. However, the vocal expression of emotions in birds is a relatively unexplored research area. The goal of this study was to develop an interactive feeding system that would elicit positive and negative emotional states, and collect recordings of the vocal expression of these emotions without human interference. In this paper, the mechatronic design and development of the feeder is described. Design choices were motivated by the desire for the hens to voluntarily interact with the feeder and experience the different stimuli that were designed to induce (1) positive low-arousal, (2) positive high-arousal, (3) negative low-arousal, and (4) negative high-arousal states. The results showed that hens were motivated to engage with the feeder despite the risk of receiving negative stimuli and that this motivation was sustained for at least 1 week. The potential of using the interactive feeder to analyse chicken vocalisations related to emotional valence and arousal is being explored, offering a novel proof of concept in animal welfare research. Preliminary findings suggest that hens vocalised in response to all four stimulus types, with the number of vocalisations, but not the probability of vocalising, distinguishing between low- and high-arousal states. Thus, the proposed animal-computer interaction design has potential to be used as an enrichment device and for future experiments on vocal emotions in birds.