Vocal Characteristics Research Articles

Evaluation of an AI-Based Voice Biomarker Tool to Detect Signals Consistent With Moderate to Severe Depression.

Mental health screening is recommended by the US Preventive Services Task Force for all patients in areas where treatment options are available. Still, it is estimated that only 4% of primary care patients are screened for depression. The goal of this study was to evaluate the efficacy of machine learning technology (Kintsugi Voice, v1, Kintsugi Mindful Wellness, Inc) to detect and analyze voice biomarkers consistent with moderate to severe depression, potentially allowing for greater compliance with this critical primary care public health need. We performed a cross-sectional study from February 1, 2021 to July 31, 2022 to examine ≥25 seconds of free-form speech content from English-speaking samples captured from 14,898 unique adults in the United States and Canada. Participants were recruited via social media, provided informed consent, and their voice biomarker results were compared with a self-reported Patient Health Questionnaire-9 (PHQ-9) at a cut-off score of 10 (moderate to severe depression). From as few as 25 seconds of free-form speech, machine learning technology was able to detect vocal characteristics consistent with an increased PHQ-9 ≥10, with a sensitivity of 71.3 (95% CI, 69.0-73.5) and a specificity of 73.5 (95% CI, 71.5-75.5). Machine learning has potential utility in helping clinicians screen patients for moderate to severe depression. Further research is needed to measure the effectiveness of machine learning vocal detection and analysis technology in clinical deployment.

Motivating tones to enhance education: The effects of vocal awareness on teachers' voices.

Effective classroom communication is key to shaping the learning environment and inspiring student engagement. And, it's not just what is said, but how it's said, that influences students. Yet, few (current or future) teachers receive education on vocal pedagogy. This study examined the impact of raising vocal awareness in teachers on their voice production through delivering a voice training program. Specifically, we explored how primary school teacher trainees produced motivational (either soft, warm, and encouraging, or harsh, pressuring, and controlling) and neutral communications before and after the delivery of a voice education program that concentrated on raising voice awareness, vocal anatomy, exercise techniques (e.g. breath control, voice modulation), and voice care. We hypothesised that trainees' voice production would change over the course of the program and lead to more 'prototypical' displays of motivational prosody (e.g. softly spoken encouraging intentions vs. harshly spoken controlling intentions). Results indicated a noticeable difference when communicating motivational intentions between pre- and post-training voice samples: post training, trainees spoke more slowly and with reduced vocal effort irrespective of motivational intention, suggesting that raising vocal awareness can alter classroom communications. The results underscore the importance of vocal awareness training to create a supportive and autonomy-enhancing learning environment.

Comparative Analysis of Vocal Communication and Atypicality in Mandarin-Speaking Children with Autism Spectrum Disorder, Developmental Delay, and Typical Development

PurposeThe current study aimed to examine the similarities and differences in vocal characteristics between Mandarin-speaking 36–72-month-old children with autism spectrum disorder (ASD), children with developmental delay (DD), and typically developing (TD) children. MethodsWe analyzed vocal characteristics during parent–child free play. Video and audio samples were collected from 21 children with ASD, 18 children with DD matched groupwise for developmental level, and 15 TD children matched groupwise for chronological age. ResultsCompared to children in the DD and TD groups, children with ASD exhibited a significantly higher proportion of nonspeech vocalizations (NSV, p<0.01), atypical vocalizations (ATY, p<0.001), noncommunicative vocalizations (NCV, p<0.001), and no response to others (NR, p<0.001). These four metrics were significantly correlated with children’s performance on the developmental evaluation and intelligence test. Receiver operating characteristic analyses indicated that the area under the curve (AUC) ranged from 0.871–0.978, with moderate accuracy, when using these four metrics to differentiate between children with ASD and TD children; this value improved to 1.000 with high accuracy when combining the four metrics. The AUC ranged from 0.747–0.820 when using the four metrics to differentiate children with ASD from those with DD and improved to 0.857 when combining the four metrics. ConclusionThese preliminary data suggest that vocal communication and vocal atypicality may be unique features of children with minimally verbal ASD and that the four metrics—NSV, ATY, NCV, and NR—have the potential to be ASD behavioral markers in screening and diagnosis.

Acoustic speech features are associated with late-life depression and apathy symptoms: Preliminary findings.

Late-life depression (LLD) is a heterogenous disorder related to cognitive decline and neurodegenerative processes, raising a need for the development of novel biomarkers. We sought to provide preliminary evidence for acoustic speech signatures sensitive to LLD and their relationship to depressive dimensions. Forty patients (24 female, aged 65-82 years) were assessed with the Geriatric Depression Scale (GDS). Vocal features were extracted from speech samples (reading a pre-written text) and tested as classifiers of LLD using random forest and XGBoost models. Post hoc analyses examined the relationship between these acoustic features and specific depressive dimensions. The classification models demonstrated moderate discriminative ability for LLD with receiver operating characteristic=0.78 for random forest and 0.84 for XGBoost in an out-of-sample testing set. The top classifying features were most strongly associated with the apathy dimension (R 2=0.43). Acoustic vocal features that may support the diagnosis of LLD are preferentially associated with apathy. The depressive dimensions in late-life depression (LLD) have different cognitive correlates, with apathy characterized by more pronounced cognitive impairment.Acoustic speech features can predict LLD. Using acoustic features, we were able to train a random forest model to predict LLD in a held-out sample.Acoustic speech features that predict LLD are preferentially associated with apathy. These results indicate a predominance of apathy in the vocal signatures of LLD, and suggest that the clinical heterogeneity of LLD should be considered in development of acoustic markers.

Effect of Vocal Loading in Female Bharatanatyam Dance Teachers After an Hour-Long Class.

To compare certain acoustic, aerodynamic, and perceptual parameters before and after an hour-long class to analyze vocal loading characteristics in female Bharatanatyam dance teachers. Prospective study. The study included 52 female Bharatanatyam dance teachers aged 19 to 40years. A brief case history was taken to recruit the participants and document the essential details. Multiparametric voice assessment was done before and after an hour-long dance class for the parameters Mean Intensity, Mean Pitch, Pitch Standard Deviation, Harmonics to Noise Ratio, Jitter percentage, Shimmer, Cepstral Peak Prominence, Acoustic Voice Quality Index, s/z ratio, Maximum Phonation Time (MPT), GRBAS, and Voice-Related Quality of Life assessment. Acoustic Voice Quality Index values above 2.22 in 65.38% of participants before class, indicative of dysphonia. The mean Voice-Related Quality of Life score was 1.98±1.67. The Wilcoxon signed-rank test showed significant differences in mean intensity (P=0.00) and jitter percentage (P=0.012). The paired t test revealed significant differences in mean pitch (P=0.00), Cepstral Peak Prominence (P=0.027), and MPT (P=0.00). No significant difference was found in the GRBAS grade evaluation (P=0.439) before and after the class. In the present study, significant changes in several acoustic parameters, MPT, and Acoustic Voice Quality Index indicating dysphonia, highlight the impact of vocal loading on the voice of female Bharatanatyam dance teachers.

Niche-related processes explain phylogenetic structure of acoustic bird communities in Mexico.

Acoustic communities are acoustically active species aggregations within a habitat, where vocal interactions between species can interfere their communication. The acoustic adaptation hypothesis (AAH) explains how the habitat favors the transmission of acoustic signals. To understand how bird acoustic communities are structured, we tested the effect of habitat structure on the phylogenetic structure, and on the phylogenetic and vocal diversity of acoustic communities in a semi-arid zone of Mexico. From autonomous recordings in three types of vegetation (crop fields, tetecheras, and mesquiteras), which differ in terms of complexity and canopy openness, we evaluated sound attenuation, and estimated metrics of phylogenetic structure and diversity as well as acoustic diversity with the use of two indices. Mesquiteras showed greater vegetation density, more attenuation, more vocal diversity, as well as a phylogenetic structure that tended towards overdispersion, in contrast to crop fields that showed less vegetation density, less attenuation, less vocal diversity and more phylogenetic relatedness, while tetecheras showed intermediate patterns. Phylogenetic structure was explained by vegetation density and excess attenuation. The higher vocal diversity, phylogenetic structure tended towards overdispersion. These results suggest a role for environmental filters in the crop fields, where more closely related species with similar vocal characteristics coexist (supporting AAH), and probably competitive exclusion in the mesquiteras, where more distantly related species coexist, promoting vocal diversity. This study offers information about the influence of habitat on the acoustic community structure, which could inform our understanding of the distribution of species from acoustic perspective.

Sentiment Predictor for Stress Detection Using Voice Stress Analysis

Abstract: This project proposes a machine learning basedapproach to predict the sentiment i.e Stressed, Unstressed orNeutral from voice of a human being and also presents a comparative analysis among various machine learning models. This technology seeks to distinguish between stressed and non-stressed outputs in response to stimuli. The dataset used fortraining in this project comprises audio recordings based onfour different situations, each recorded in various emotionalstates. Given the increased demand for communication betweenintelligent systems and human, automatic stress detection isbecoming an interesting research topic. Even while the level ofspecific hormones, such cortisol, can be precisely measured to indicate stress, this approach is not feasible for diagnosing stress in interactions between humans and machines. The Sentiment Predictor for Stress Detection Using Voice Stress Analysis utilizesmachine learning algorithms such as SVM, Random Forest , Logistic Regression etc. to analyze voice patterns and predict an individual’s emotional state based on the changes in their vocal characteristics. The technology can identify changes in pitch, tone, and rhythm, among other factors, to determinean individual’s stress level. Overall, the Sentiment Predictorfor Stress Detection Using Voice Stress Analysis is a promising technology that can provide valuable insights into an individual’s emotional state and support their well-being in various settings

IMPROVING PERFORMANCE OF VIETNAMESE SPEAKER RECOGNITION USING TRANSFER LEARNING AND ENSEMBLE EMBEDDING

Speaker recognition technology is crucial for identifying or verifying individuals based on their unique vocal characteristics, such as pitch, tone, and speaking style. This technology is widely used to enhance security, improve customer service, support law enforcement, and personalize interactions with smart devices. In recent years, thanks to the application of deep learning techniques, speaker recognition has made significant progress. However, Vietnamese speaker recognition still faces many challenges. This paper proposes new strategies that combine transfer learning and ensemble learning to improve the accuracy of Vietnamese speaker recognition. Experimental results on Vietnamese datasets show significant improvements in recognition accuracy. These findings highlight the potential of tailored approaches to advance speaker recognition technology for Vietnamese speakers and expand its applications in this field.

The rhythm of horse gaits.

What makes animal gaits so audibly rhythmic? To answer this question, we recorded the footfall sound of 19 horses and quantified the rhythmic differences in the temporal structure of three natural gaits: walk, trot, and canter. Our analyses show that each gait displays a strikingly specific rhythmic pattern and that all gaits are organized according to small-integer ratios, those found when adjacent temporal intervals are related by a mathematically simple relationship of integer numbers. Walk and trot exhibit an isochronous structure (1:1)-similar to a ticking clock-while canter is characterized by three small-integer ratios (1:1, 1:2, 2:1). While walk and trot both show isochrony, trot has a slower tempo and is more precise and accurate, like a metronome. Our results quantitatively discriminate horse gaits based on rhythm, revealing striking commonalities with human music and some animal communicative signals. Gait and vocal rhythmicity share key features, and the former likely predates the latter; we suggest this supports gait-based hypotheses for the evolution of rhythm. Specifically, the perception of locomotor rhythmicity may have evolved in different species under pressure for predator recognition and mate selection; it may have been later exapted for rhythmic vocal communication.

Real-time speech emotion recognition using deep learning and data augmentation

In human–human interactions, detecting emotions is often easy as it can be perceived through facial expressions, body gestures, or speech. However, in human–machine interactions, detecting human emotion can be a challenge. To improve this interaction, Speech Emotion Recognition (SER) has emerged, with the goal of recognizing emotions solely through vocal intonation. In this work, we propose a SER system based on deep learning approaches and two efficient data augmentation techniques such as noise addition and spectrogram shifting. To evaluate the proposed system, we used three different datasets: TESS, EmoDB, and RAVDESS. We employe several algorithms such as Mel Frequency Cepstral Coefficients (MFCC), Zero Crossing Rate (ZCR), Mel spectrograms, Root Mean Square Value (RMS), and chroma to select the most appropriate vocal features that represent speech emotions. Three different deep learning models were imployed, including MultiLayer Perceptron (MLP), Convolutional Neural Network (CNN), and a hybrid model that combines CNN with Bidirectional Long-Short Term Memory (Bi-LSTM). By exploring these different approaches, we were able to identify the most effective model for accurately identifying emotional states from speech signals in real-time situation. Overall, our work demonstrates the effectiveness of the proposed deep learning model, specifically based on CNN+BiLSTM enhanced with data augmentation for the proposed real-time speech emotion recognition.

Ecological Validity of Self-Perceived Voice Quality and Acoustic Measures During Voice Assessments: An Observational Study on Faculty Teachers.

This study investigated the ecological validity of conventional voice assessments by comparing the self-perceived voice quality and acoustic characteristics of voice production during these assessments to those in a simulated environment with varying distracting conditions and noise levels. Forty-two university professors (26 women) participated in the study, where they were asked to produce loud connected speech by reading a 100-word text under four different conditions: a conventional assessment and three virtual classroom simulations created with 360° videos, each with different noise levels, played through a virtual reality headset and headphones. The first video depicted students paying attention in class (40 dB classroom noise); the second showed some students talking, generating moderate conversational noise (60 dB); and the third depicted students talking loudly and not paying attention (70 dB). The entire experiment was conducted in a sound-treated room, and the voice of each participant was recorded for acoustic analysis. In each condition, self-perception of voice quality (vocal effort and vocal ease), SPL, fundamental frequency, long-term average spectrum (L1-L0 ratio, alpha ratio, and the 1/5-5/8 ratio), and smooth cepstral peak prominence were measured. Visual distraction and noise level significantly impacted both subjective and acoustic measures of voice production, as shown by numerous statistically significant differences across almost all conditions and variables examined. Specifically, all measures increased with higher levels of distraction and noise, except for the 1/5-5/8 ratio, which showed a decreasing trend. These findings indicate that visual distraction and noise level significantly influence self-perceived and acoustic vocal characteristics and suggest that conventional assessments, typically conducted in silence and without visual distractors, may not accurately represent real-world performance, thus limiting their ecological validity.

Prosodic changes with age: a longitudinal study with three public figures in European Portuguese

Purpose The analysis of acoustic parameters contributes to the characterisation of human communication development throughout the lifetime. The present paper intends to analyse suprasegmental features of European Portuguese in longitudinal conversational speech samples of three male public figures in uncontrolled environments across different ages, approximately 30 years apart. Participants and methods Twenty prosodic features concerning intonation, intensity, rhythm, and pause measures were extracted semi-automatically from 360 speech intervals (3-4 interviews from each speaker x 30 speech intervals x 3 speakers) lasting between 3 to 6 s. Results Twelve prosodic parameters presented significant age effects at least in one speaker. Group mean comparisons revealed significant differences between the youngest (i.e. 50 years) and the oldest age groups (i.e. 80 years) in seven parameters. The results from the analysis point to a lower and less variable fo, higher fo minimum, wider fo peaks, more vocal effort and more variable global intensity, slower speech and articulation rate, and also more frequent and longer pauses in older ages. Conclusion This longitudinal study has the potential to contribute to the characterization of the normal aging process, proving to be significant in the domains of human-machine communication, speech recognition systems, applied linguistics, or the implementation of strategies in communicative contexts with older adults

Changes in Voice Quality after a Pure Tone Stimulation (PTS) Program

Introduction. Auditory feedback allows individuals to monitor their vocal characteristics and adjust to maintain optimal voice quality. One type of auditory stimulation for conversational voice training/therapy is pure tone stimulation. This technique presents binaural auditory stimuli consisting of pure tones separated by half-step intervals to modify the fundamental frequency of the speaking voice and expand the vocal range. Objectives. This study aimed to characterize detectable changes in voice production following the application of pure tone stimulation (PTS) among speakers with and without voice disorders. Methods. Data from thirty-nine participants (28 individuals with voice disorders and 11 individuals with normal voices) were analyzed for this study. All participants engaged in binaural PTS exercises. Participants recorded a sustained vowel /a/ before and after the PTS exercises. Multiple acoustic voice parameters were extracted from the sustained vowel samples (fundamental frequency, pitch strength, harmonics-to-noise ratio, and smoothed cepstral peak prominence). Additionally, a visual analogue scale (VAS) interpretation of narrowband (NB) spectrograms was conducted to assess voice quality. Results. Statistically significant increases in fundamental frequency were found after the PTS exercises, except for males in the normal voices group. Pitch strength increased after the PTS, regardless of gender. Participants also demonstrated an increase in the harmonics-to-noise ratio. VAS ratings of NB spectrograms indicated improvement in voice quality following the program. Conclusions. Voice changes after performing PTS suggest voice quality improvement, as measured by acoustic analysis of vowel production and qualitative assessment of NB spectrograms among participants with and without voice disorders.

Using a mixed-reality headset to elicit and track clinically relevant movement in the clinic.

21st century neurology will require scalable and quantitative tools that can improve neurologic evaluations over telehealth and expand access to care. Commercially available mixed-reality headsets allow for simultaneous presentation of stimuli via holograms projected into the real world and objective and quantitative measurement of hand movement, eye movement, and phonation. We created 6 tasks designed to mimic standard neurologic assessments and administered them to a single participant via the Microsoft HoloLens 2 mixed-reality headset. The tasks assessed postural hand tremor, finger tapping, pronation and supination of hands, hand and eye tracking of a center-out task, hand and eye tracking of a random motion task, and vocal assessment. We show the utility of the HoloLens for commonly used neurological exams. First, we demonstrate that headset-derived holograms can project hand movements and objects in 3D space, providing a method to accurately and reproducibly present test stimuli to reduce test-test variability. Second, we found that participant hand movements closely matched holographic stimuli using a variety of metrics calculated on recorded movement data. Third, we showed that the HoloLens can record and playback exam tasks for visual inspection, sharing with other medical providers, and future analysis. Fourth, we showed that vocal recordings and analysis could be used to profile vocal characteristics over time. Together, this demonstrates the versatility of mixed reality headsets and possible applications for neurological assessment. Administering components of the neurologic exam via a self-contained and commercially available mixed-reality headset has numerous benefits including detailed kinematic quantification, reproducible stimuli presentation from test to test, and can be self-administered expanding access to neurological care and saving hospital time and money.

Insights into Corncrake (Crex crex) movements in a fragmented agricultural landscape using bioacoustics with implications for nature conservation strategies

AbstractMovement ecology data are often collected by catching animals and fitting GPS tags. This technique is expensive, biased and comes at an extra cost for the caught individuals. New conservation technologies allow noninvasive data collection, providing ecological knowledge to aid the conservation of endangered species. Using bioacoustics, we investigated intraseason and between‐year movements of the red‐listed corncrake (Crex crex) and their relationships with landscape structure. Our goals were to track corncrake movements with minimal disturbance to the species and determine how landscape features affect these movements. We recorded males over two consecutive breeding seasons in 2022 and 2023 in an extensively farmed agricultural landscape in the West of Ireland. Individual identity was obtained through vocal characteristics of the calling males which were grouped into clusters. The areas of seminatural grassland and margin features, presence of bird refuge strips, and habitat connectivity were extracted from buffers between locations of two matching recordings (same calling male). These landscape features were used as model predictors to determine the distance that the same male covered. We found that, in a season, males moved 415 m on average (min = 10 m; max = 1805 m) between calling locations, which was more than previously thought in the Western Europe. Males also reoccurred on average 375 m (min = 23 m; max = 1231 m) away from the previous year's location. Landscape connectivity (p = 0.01, N = 42) and seminatural grassland area (p = 0.02, N = 42) were positive predictors of intraseason movements, whereas the margin area (p = 0.01, N = 35) predicted further between‐year movements. Considering our noninvasively sampled results in the Irish context, we suggest increasing the protection area around a calling male to 500 m. In this buffer, we also suggest improving the availability of seminatural grassland to 30% and presence of margin features to 3%. Further land management recommendations are provided.

Comparative Analysis of Compositions for Virtual Singers and Human Singers

This article discusses the similarities and differences between traditional vocal works and virtual singer works. It introduces the vocal characteristics of virtual singers, explores the differences between their performances and those of real singers, and analyzes the considerations that composers have for harmony when composing for virtual singers and real vocals, such as the way in which traditional functional harmonies and modern twelve-tone sequences are harmonic, and what the differences are, and composers' ideas for using different harmonies for different groups when composing. This article focuses on virtual singers Kafu and Hatsune Miku, as well as some representative works by the two virtual singers. Several songs written for real vocals will be analyzed, such as the Mass, Rossini's opera, and the vocal parts of Boulez's Ownerless Mallet. The impact of different vocal characteristics on the uniqueness of musical works, along with the technical differences and similarities between compositions for virtual and real singers, will be discussed.

Human Voice: Legal Protection Challenges

Focused on the phenomenon of human voice, the paper purports to develop approaches to legal protection of ordinary people's voices and those of artists as the technologies for sound synthesis, music and vocal performance are becoming more sophisticated. It was demonstrated the problem of legal protection of human voice has become especially pressing one in the context of artificial neural networks such as vocaloid designed for sound synthesis and voice cloning. The study provides a systemic arrangement of legally meaningful knowledge of human voice useful for the development of legal provisions to protect this personal good. The legal substance of vocal impersonation as a way to simulate and manipulate a synthesized voice was explored. Theoretically applicable legal constructs for protection of voice and computerized cloning technologies were analyzed. A trend to use copyright rather than patent for protecting vocaloid-like generative neural networks and other technological solutions for vocal synthesis was identified. The concept of voice was critically analyzed to propose a viable legal provision for its protection. The primary and auxiliary features of the concept of voice were parametrized for possible use in disputes on the legitimacy of voice cloning or vocal impersonation. The concept of vocal identity of ordinary people and artists was proposed for legal protection of this personal good as a set of performative sonic features including the basic parameters of singing voice as well as acoustic-phonetic and articulatory features of vocalization. A comprehensive legal and technological methodology for the protection of vocal identity was proposed.

Three romances by Alexei Stârcea on the poems of Gheorghe Vodă: figurative structure, composition and vocal features

The article is dedicated to the romances signed by Alexei Stârcea on the lyrics of GheorgheVodă – poet, journalist, screen writer and director, founder of the poetic cinema of Moldova. All the three romances – „Winter Watercolor”, „Spring Watercolor” and „Windows” – were written in 1966, during the period of maturity of A. Stârcea who was attracted to the creation of Gh. Vodă by its philosophical depth, refinement of the poetic style, the theme related to the love for the native country, the parental home, the relationship between man and the world around him, the problems of communication between generations. „The magic crystal” of Gh. Vodă’s poetry was embodied in three romances of great artistic value, each being a special miniature with a vivid artistic image and original musical language, attractive both for performers and for the public.

Building Consistent Characters through Open-Source Generative AI

This paper presents a pipeline for creating consistent characters using open-source generative AI tools, aimed at enhancing educational content. Through the use of XTTS, Fooocus, and FaceFusion, a talking avatar of the marine biologist Raffaele Issel was developed, accurately replicating his facial and vocal features. This study demonstrates how these tools can generate high-quality, engaging educational materials with minimal technical expertise and resources. The methodology facilitates the creation of interactive avatars for use in graphic novels and comic books, enriching museum and educational experiences.

Female song elicits increased vocal display effort in both sexes of Alston's singing mouse.

Acoustic displays are conspicuous behaviors common across diverse animal taxa. They have long been studied in behavioral ecology, evolutionary biology, and neuroscience. Most of these investigations, however, have focused on male display. For species in which both sexes display, correcting this bias will lead to a fuller understanding of the evolution and function of these behaviors. In this study, we investigated the role of vocal advertisement display in female Alston's singing mice (Scotinomys teguina). Singing mice are small muroid rodents of the family Cricetidae, native to high altitude cloud forests of Central America. Females and males both produce elaborate advertisement songs used in mate attraction and intra-sexual competition. Prior studies have largely focused on male behavior, though we recently found that both sexes dramatically increase their rate of singing and song duration in response to playback of conspecific male song. Here, we tested how mice of both sexes adjust their song effort in response to female song playback. Our results show that, like male song, female song elicits robust increases in song effort from both female and male mice. This study reveals additional social contexts that promote high song effort in this species, while raising additional questions about the role of song in communication within and between the sexes.