Speech Analysis Research Articles

Prediction of dysphagia aspiration through machine learning-based analysis of patients’ postprandial voices

BackgroundConventional diagnostic methods for dysphagia have limitations such as long wait times, radiation risks, and restricted evaluation. Therefore, voice-based diagnostic and monitoring technologies are required to overcome these limitations. Based on our hypothesis regarding the impact of weakened muscle strength and the presence of aspiration on vocal characteristics, this single-center, prospective study aimed to develop a machine-learning algorithm for predicting dysphagia status (normal, and aspiration) by analyzing postprandial voice limiting intake to 3 cc.MethodsConducted from September 2021 to February 2023 at Seoul National University Bundang Hospital, this single center, prospective cohort study included 198 participants aged 40 or older, with 128 without suspected dysphagia and 70 with dysphagia-aspiration. Voice data from participants were collected and used to develop dysphagia prediction models using the Multi-Layer Perceptron (MLP) with MobileNet V3. Male-only, female-only, and combined models were constructed using 10-fold cross-validation. Through the inference process, we established a model capable of probabilistically categorizing a new patient's voice as either normal or indicating the possibility of aspiration.ResultsThe pre-trained models (mn40_as and mn30_as) exhibited superior performance compared to the non-pre-trained models (mn4.0 and mn3.0). Overall, the best-performing model, mn30_as, which is a pre-trained model, demonstrated an average AUC across 10 folds as follows: combined model 0.8361 (95% CI 0.7667–0.9056; max 0.9541), male model 0.8010 (95% CI 0.6589–0.9432; max 1.000), and female model 0.7572 (95% CI 0.6578–0.8567; max 0.9779). However, for the female model, a slightly higher result was observed with the mn4.0, which scored 0.7679 (95% CI 0.6426–0.8931; max 0.9722). Additionally, the other models (pre-trained; mn40_as, non-pre-trained; mn4.0 and mn3.0) also achieved performance above 0.7 in most cases, and the highest fold-level performance for most models was approximately around 0.9. The ‘mn’ in model names refers to MobileNet and the following number indicates the ‘width_mult’ parameter.ConclusionsIn this study, we used mel-spectrogram analysis and a MobileNetV3 model for predicting dysphagia aspiration. Our research highlights voice analysis potential in dysphagia screening, diagnosis, and monitoring, aiming for non-invasive safer, and more effective interventions.Trial registration: This study was approved by the IRB (No. B-2109-707-303) and registered on clinicaltrials.gov (ID: NCT05149976).

Decoding paradoxical identities: The discourse construction of left-behind children in news reports

This paper presents an empirical corpus-based study on the identity of left-behind children in China, employing the discourse-historical approach (DHA) framework. It is the first study to combine Latent Dirichlet Allocation (LDA) topic modeling and reported speech analysis to examine how China Daily constructs the identity of left-behind children and explore the cultural and rhetorical factors influencing this identity construction. By integrating computational techniques and qualitative analysis methods, this study provides a comprehensive understanding of the intricate nature of identity construction for left-behind children. This interdisciplinary approach strengthens the theoretical foundations of media analysis and offers fresh insights into the dynamics of media discourse. The findings reveal that China Daily portrays left-behind children in a multifaceted and diverse manner, encompassing both positive and negative aspects, while placing emphasis on their vulnerability and passivity. Furthermore, the media employs communication techniques such as identification by sympathy, antithesis, and inaccuracy to establish emotional resonance and foster audience identification, ultimately influencing the audience’s perspectives on these children.

Psycholinguistic analysis of speech processing under pathophysiological (traumatic) stress factor

The article presents the analysis results of spontaneous speech under pathophysiological traumatic stress factor. Traumatic stress is an acute reaction to pathophysiological factor affecting the psychoemotional sphere and characterised by changes in behaviour patterns including speech. The purpose of this study is the auditory analysis of speech, which demonstrates changes of formal characteristics at the tempo level, rhythm and mistakes under emotional intensity. The subject of the study is the formal speech characteristics of patients with various injuries under natural conditions of pathophysiological acute stress. Systemic principles of the speech analysis are based on an interdisciplinary approach and includes determination of the personality psychological type and the mental status of each subject at the time of patient examination to diagnose the state of emotional intensity. Interdisciplinary psycholinguistic analysis allows assessing emotional and psychophysiological state of patients and is a basis for psychocorrectional work with patients in a state of pathophysiological stress and for treatment taking into account their mental status.

Analysis of Human Voice for Speaker Recognition: Concepts and Advancement

Human voice or speech is a contactless, non-invasive biometric trait for human recognition, easy to use with minimal computer complexity and inexpensive to implement. Speaker recognition (SR) has turned out to be a magnificent approach using speech as the central premise since decades. Its broad range of usages, like forensic speech verification to identify culprits by law enforcement authorities and access control to mobile banking, mobile shopping, etc., has made it a lucrative area of research. Also, the ease of use and dependability of SR will significantly assist people with disabilities in securely accessing and reaping the benefits of digital-era services. Additionally, the emergence of numerous deep learning methods for feature extraction and classification, has helped SR to achieve tremendous progress. This paper presents a comprehensive study on the progression of SR for decades till the present, including integration with Blockchain and challenges. It covers most of the factors that influence SR performance such as fundamentals and structure of SR, different speech pre-processing techniques, various speech features, feature extraction techniques, traditional and neural network-based classification techniques and deep learning-based SR toolkits. As a consequence, in this digital Blockchain era, it will help to design robust and reliable recognition-based services for mankind.

Использование технологий, основанных на искусственном интеллекте, в преподавании иностранного языка в образовательной организации среднего профессионального образования

The article considers artificial intelligence technology together with deep learning algorithm for modernization of foreign language teaching in the educational organization of secondary vocational education. Foreign lan-guage processing is one of the main methods of artificial intelligence and is widely used in speech analysis and recognition. In order to increase the proportion of qualitatively learned material by young professionals, different teaching methods are implemented according to the different training levels of students, and system-atic simulation comparisons are made according to their degree of training. The work emphasizes the im-portance of a balanced approach to the application of artificial intelligence technologies in the process of teaching foreign language as part of the system of secondary vocational education, taking into account the ad-vantages and disadvantages of software products.

An acoustic study of Jonglish Communiy: Javanese-accented speech

People who speak English with a Javanese accent are referred to as Jonglish (Jowo-English) speakers. They end their sentences in a high tone, therefore, the aims of this study were to compare the intensity and pitch of Javanese accent speakers (specifically from Pekalongan, Semarang, and Pati) with native speakers. The focus was on investigating whether the high tone at the end of sentences affects the engaging nature of the conversation. It employed descriptive qualitative design and included data collection, acoustic analysis, perception analysis, linguistic analysis, and interpretation. The data were collected by audio recording in the form of sentences. Praat, a voice analysis, was utilized to analyze the data. The findings demonstrated that Pekalongan and Semarang speakers had higher pitch than native speaker (232.25 Hz and 206.27 Hz). Meanwhile, the pitch score for Pati was lower than native; 192.04 Hz. On the other hand, the intensity from those regions were lower than native speaker; 69.58 dB, 61.51 dB, and 68.203 dB, while the of native speaker was 71.306 dB. The fact that the Jonglish includes a tendency to end sentences with a high tone suggests a particular intonation pattern. Despite these variations, effective communication often relies on contextual cues and the listeners ability to adapt to different speech patterns. The study has the potential to make meaningful contributions to English pronunciation instruction, cultural awareness, and effective cross-cultural communication. The outcomes of the research can inform pedagogical practices and support the broader goal of promoting linguistic diversity in English language education.

Classification research of TCM pulse conditions based on multi-label voice analysis

ObjectiveTo explore the feasibility of remotely obtaining complex information on traditional Chinese medicine (TCM) pulse conditions through voice signals. MethodsWe used multi-label pulse conditions as the entry point and modeled and analyzed TCM pulse diagnosis by combining voice analysis and machine learning. Audio features were extracted from voice recordings in the TCM pulse condition dataset. The obtained features were combined with information from tongue and facial diagnoses. A multi-label pulse condition voice classification DNN model was built using 10-fold cross-validation, and the modeling methods were validated using publicly available datasets. ResultsThe analysis showed that the proposed method achieved an accuracy of 92.59% on the public dataset. The accuracies of the three single-label pulse manifestation models in the test set were 94.27%, 96.35%, and 95.39%. The absolute accuracy of the multi-label model was 92.74%. ConclusionVoice data analysis may serve as a remote adjunct to the TCM diagnostic method for pulse condition assessment.

Illocutionary Act Analysis of Melati and Isabel Wijsen Speech at United Nations

Illocutionary Act Analysis of Melati and Isabel Wijsen Speech at United Nations

Linguistic changes in neurodegenerative diseases relate to clinical symptoms.

The detection and characterization of speech changes may help in the identification and monitoring of neurodegenerative diseases. However, there is limited research validating the relationship between speech changes and clinical symptoms across a wide range of neurodegenerative diseases. We analyzed speech recordings from 109 patients who were diagnosed with various neurodegenerative diseases, including Alzheimer's disease, Frontotemporal Dementia, and Vascular Cognitive Impairment, in a cognitive neurology memory clinic. Speech recordings of an open-ended picture description task were processed using the Winterlight speech analysis platform which generates >500 speech features, including the acoustics of speech and linguistic properties of spoken language. We investigated the relationship between the speech features and clinical assessments including the Mini Mental State Examination (MMSE), Mattis Dementia Rating Scale (DRS), Western Aphasia Battery (WAB), and Boston Naming Task (BNT) in a heterogeneous patient population. Linguistic features including lexical and syntactic features were significantly correlated with clinical assessments in patients, across diagnoses. Lower MMSE and DRS scores were associated with the use of shorter words and fewer prepositional phrases. Increased impairment on WAB and BNT was correlated with the use of fewer nouns but more pronouns. Patients also differed from healthy adults as their speech duration was significantly shorter with more pauses. Linguistic changes such as the use of simpler vocabularies and syntax were detectable in patients with different neurodegenerative diseases and correlated with cognitive decline. Speech has the potential to be a sensitive measure for detecting cognitive impairments across various neurodegenerative diseases.

Predictive Modelling for Parkinson's Disease Diagnosis using Biomedical Voice Measurements

INTRODUCTION: Parkinson's Disease (PD), a progressively debilitating neurological disorder impacting a substantial global population, stands as a significant challenge in modern healthcare. The gradual onset of motor and non-motor symptoms underscores the criticality of early detection for optimal treatment outcomes. In response to this urgency, novel avenues for early diagnosis are being explored, where the amalgamation of biomedical voice analysis and advanced machine learning techniques holds immense promise. Individuals afflicted by PD experience a nuanced deterioration of bodily functions, necessitating interventions that are most effective when initiated at an early stage. The potential of biomedical voice measurements to encode subtle health indicators presents an enticing opportunity. The human voice, an intricate interplay of frequencies and patterns, might offer insights into the underlying health condition.
 OBJECTIVES: This research embarks on a comprehensive journey to delve into the intricate connections between voice attributes and the presence of PD, with the aim of expediting its detection and treatment.
 METHODS: At the heart of this exploration is the Support Vector Machine (SVM) model, a versatile machine learning tool [1-2]. Functioning as a virtual detective, the SVM model learns from historical data to decipher the intricate patterns that differentiate healthy individuals from those with PD [3-4].
 RESULTS: Through the power of pattern recognition, the SVM becomes a predictive instrument, a potential catalyst in unravelling the latent manifestations of PD using the unique patterns harbored within the human voice. Embedded within this research are the practical demonstrations showcased through code snippets [5-7]. By synergizing the intricate voice measurements with the SVM model, we envision the emergence of a diagnostic paradigm where early PD detection becomes both accessible and efficient. This study not only epitomizes the synergy of voice and machine interactions but also attests to the transformative potential of technology within the domain of healthcare. .
 CONCLUSION: Ultimately, this research strives to harness the intricate layers of voice data, as exemplified through the provided model code [8-11], to contribute to the evolution of an advanced tool for PD prediction. By amalgamating the principles of machine learning and biomedical analysis, we aspire to expedite early PD diagnosis, thereby catalyzing more efficacious treatment strategies. In traversing this multidimensional exploration, we aspire to pave the path toward a future where technology plays an instrumental role in enhancing healthcare outcomes for individuals navigating the challenges of PD, ultimately advancing the pursuit of early diagnosis and intervention.

Methodological Foundations of Film Speech Analysis Using Corpora: Technical, Social, and Cultural-National Aspects

The aim of this study is to describe the possibilities and limitations of the corpus approach to the analysis of film subtitles, taking into account their technical, social, and culturalnational features. It offers a review of existing film content corpora accompanied by their critical interpretation, firstly from the perspective of subtitle quality as empirical data for linguistic research, and secondly from the perspective of corpus quality as an environment for quantitative analysis of empirical material. Furthermore, based on established subtitling practices, the author proposes her own viewpoint on subtitles as material for studying film speech, considering technical (line length, screen size adaptation), social (subtitles for the deaf and hard of hearing), and cultural-national (subtitling countries vs dubbing countries) aspects of subtitling in different languages and countries. In the course of the work, it was found that the largest number of subtitles available in open repositories are translated equivalents of cinema speech. It is noted that subtitling is preferred in the USA, UK, India, China, and Japan. The features of subtitling in countries such as Serbia, Finland, and Russia are described. The author concludes that these features are important for the quality of linguistic research on the basis of translated movie speech represented by subtitles.

A Genre-Based Analysis of Political Speeches of Selected Pakistani Prime Ministers in The UN General Assembly

This study consists of gender-based exploration of two political speeches that have been delivered by Nawaz Sharif and Imran Khan at the UN General Assembly. This research aims to investigate the cognitive structure of political speeches and uncover the typical strategies employed by selected politicians to achieve their communicative goals. Bhatia's (1993) genre analysis model is taken as theoretical framework. This research is qualitative in nature and it argues that political addresses at the UN Assembly are followed by certain protocols, which distinguish them as a political subcategory. The findings demonstrate that selected prime ministers conform to this genre's structure to advance their political agendas. Additionally, it reveals that both chosen speakers employ similar strategies for the argumentative aspect of their speeches, choosing strategies based on their respective communicative objectives. It is anticipated that this research will pave a road for upcoming scholars by providing novel insights about usefulness of genre analysis in multiple contexts and it also helps them to get a lucid understanding of the political goals and the projected perception of Pakistan on the global stage.

A Triplet Multimodel Transfer Learning Network for Speech Disorder Screening of Parkinson’s Disease

Deterioration in the quality of a person’s voice and speech is an early sign of Parkinson’s disease (PD). Although a number of computer-based methods have been invested to use patients’ speech for early diagnosis of Parkinson’s disease, they only focus on a fixed pronunciation test, such as the subjects’ monosyllabic pronunciation is analyzed to determine whether they have potential possibility of PD. Moreover, only using traditional speech analysis methods to extract single-view speech features cannot provide a comprehensive feature representation. This paper is dedicated to the study of various pronunciation tests for patients with PD, including the pronunciation of five monosyllabic vowels and a spontaneous dialogue. A triplet multimodel transfer learning network is designed and proposed for identifying subjects with PD in these two groups of tests. First, multisource data extract mel frequency cepstrum coefficient (MFCC) features of speech for preprocessing. Subsequently, a pretrained triplet model represents features from three dimensions as the upstream task of the transfer learning framework. Finally, the pretrained model is reconstructed as a novel model that integrates the triplet model, temporal model, and auxiliary layer as the downstream task, and weights are updated through fine-tuning to identify abnormal speech. Experimental results show that the highest PD detection rates in the two groups of tests are 99% and 90% , respectively, which outperform a large number of internationally popular pattern recognition algorithms and serve as a baseline for other academic researchers in this field.

‘Following the Straight Path’: Egypt Political Regime’s Quest for Political Legitimacy

The study aims to investigate the sources of the Egyptian regime’s legitimacy after the removal of Mohammed Morsi from office in 2013. The ruling elite, in particular President alSisi, has recently shown a growing inclination towards adopting Islam in the political discourse. As a result, there is a need to examine the factors that have contributed to this trend. The central hypothesis of the research is that the government, lacking institutional legitimacy, is compelled to reinforce the Islamic values in political communication to overcome internal political divisions and establish a sense of social connection between the authorities and citizens. To test this hypothesis, the study employs a qualitative content analysis of speeches made by President alSisi to the nation between 2014 and 2023. The analysis identifies four techniques for legitimation (mobilization, national identity consolidation, construction of a common past, and construction of a common future) and examines the functional role of Islam in each of them. The findings of the study suggest that Islamic norms and values are most actively appealed to in segments of texts devoted to national identity consolidation and constructing a common future. Both approaches involve a similar strategy of invoking Islam by associating and transferring attributes within pairs of words (religious context - secular context): nation - the Ummah; state - Islamic world; citizen - a Muslim. By using this rhetorical strategy, the government is shaping a narrative that places the contemporary Egyptian state within a religious and historical framework, as depicted in the Qur’an. This approach is aimed at establishing a set of values that can unite the nation and foster loyalty among citizens towards the government’s vision for Egypt’s future. However, there are almost no semantic sections dedicated to mobilizing citizens in the texts. This can be attributed to a pragmatic effort to steer clear of politicizing the electorate.

A Comprehensive Study on Mental Illness Through Speech and EEG Using Artificial Intelligence


 A typical mental ailment is depression that considerably harms an individual's everyday activities as well as their mental health. In light of the fact that mental health is one of the biggest problems facing society, researchers have been looking into several strategies for efficiently identifying depression. Mental illness can now be identified through speech analysis thanks to modern artificial intelligence. The speech aids in classifying a patient's mental health status, which could benefit their new study. For the purpose of identifying depression or any other emotion or mood in an individual, a number of past studies based on machine learning and artificial intelligence are being studied. The study also examines the effectiveness of facial expression, photos, emotional chatbots, and texts in identifying a person's emotions. Naive-Bayes, Support Vector Machines (SVM), Linear Support Vectors, Logistic Regression, etc. are ML approaches from text processing. Artificial Neural Network (ANN) is a sort of artificial intelligence method used to extract information from photos and classify them in order to recognise emotions from facial expressions.

Comparative Analysis of "A Thousand Splendid Suns" and a Similar Work in Uzbek Literature

This article presents a comparative analysis of Khaled Hosseini's renowned novel "A Thousand Splendid Suns" and comparable works in Uzbek literature. The analysis explores the similarities and differences between the two literary traditions, examining themes, analytical methods, and cultural influences. Both the novel and Uzbek literature delve into the complexities of women's experiences, the impact of war and political unrest, and the pursuit of personal freedom. The narrative styles in both works employ vivid imagery, emotional depth, and multi-layered storytelling techniques. The comparative analysis highlights the resilience of individuals in the face of hardships, the role of women in traditional societies, and the unique narrative approaches used by the authors. The article concludes by emphasizing the valuable insights offered by these literary works and the need for further exploration and analysis of diverse voices in literature.

Parkinsons disease diagnosis through electroencephalographic signal processing and neural network classification

Parkinsons disease diagnosis through electroencephalographic signal processing and neural network classification

Tempo-rhythmic characteristics of spontaneous speech of speakers of certain types of mental accentuations of personality in states of psychoemotional tension

The subject of psycholinguistic research is the formal signs of spontaneous sounding speech of the speaker in conditions of emotional tension. In our opinion, indicators of emotional tension of speech in perception are, first of all, formal means of sounding speech at the level of rhythm, tempo, speech failures, which are in relation to complementarity with the semantic content of the phrase and signal the degree of tension of speech. Thus, the state of emotional tension is transmitted by a complex of formal signs of sounding speech. As an example of the psychological factor of emotional tension, we considered court hearings in criminal cases. The object was audio recordings of spontaneous sounding speech of native speakers of the Russian language in a situation that determines the state of emotional tension of the speaker. The novelty of the research lies in the use of qualitative experimental methods in the analysis of spontaneous sounding speech in a situation of emotional tension. Obtaining objective results becomes possible on the basis of auditory linguistic analysis, which allows with a certain degree of reliability to assess the emotional state of the speaker (deviation from the norm, anxiety, tension, the presence and type of emotions, for example, excitement, surprise, fear / anger, etc.), psychophysiological state (abnormality, the presence of pathology), finally, establish boundary criteria for determining a stress disorder for a particular mental type. Of particular interest is the appeal to the psychological typologization of the personality of K. Leonhard, which makes a reliable analysis not only of the characteristics of the character (the so-called psychological phenotype of the personality), but also of his temperament (the psychological basis of the personality).

Voice analysis for intoxication detection in laboratory versus law enforcement contexts

There is substantial interest in deploying voice biomarkers for the detection of alcohol intoxication, yet it remains unknown how other mental and physical states (e.g. emotion, stress) influence voice biomarkers in intoxicated speech. We compared measurements of voice quality (jitter, shimmer, noise-harmonics ratio) across two datasets of alcohol-intoxicated speech: (1) The alcohol language corpus, which contains laboratory elicited speech from 167 individuals in both sober and intoxicated conditions and (2) a custom dataset of police (control, n = 14) and suspect (intoxicated, n = 32) interactions during traffic stops where intoxication was verified via breath analysis. Measurements were extracted from all stressed vowel tokens and compared across conditions using two-sample t-tests within sex-specific groupings of each dataset. For both males and females, jitter was significantly lower during intoxication as measured from laboratory-elicited speech, but significantly higher for intoxicated individuals when measured from police–suspect interactions. These results suggest that voice biomarkers for alcohol-intoxication are easily confounded by other emotional and physical states (e.g., stress during police interaction), and thus, present a particular challenge for speaker-independent detection systems where baseline voice quality measurements across different emotional states are unknown. [Research funded by Tenvos Incorporated for the development of commercial speaker state-detection algorithms.]

Speech analysis of teaching assistant interventions in small group collaborative problem solving with undergraduate engineering students

AbstractThis descriptive study focuses on using voice activity detection (VAD) algorithms to extract student speech data in order to better understand the collaboration of small group work and the impact of teaching assistant (TA) interventions in undergraduate engineering discussion sections. Audio data were recorded from individual students wearing head‐mounted noise‐cancelling microphones. Video data of each student group were manually coded for collaborative behaviours (eg, group task relatedness, group verbal interaction and group talk content) of students and TA–student interactions. The analysis includes information about the turn taking, overall speech duration patterns and amounts of overlapping speech observed both when TAs were intervening with groups and when they were not. We found that TAs very rarely provided explicit support regarding collaboration. Key speech metrics, such as amount of turn overlap and maximum turn duration, revealed important information about the nature of student small group discussions and TA interventions. TA interactions during small group collaboration are complex and require nuanced treatments when considering the design of supportive tools. Practitioner notesWhat is already known about this topic Student turn taking can provide information about the nature of student discussions and collaboration. Real classroom audio data of small groups typically have lots of background noise and present challenges for audio analysis. TAs have little training in how to productively intervene with students about collaborative skills. What this paper adds TA interaction with groups primarily focused on task progress and understanding of concepts with negligible explicit support on building collaborative skills. TAs intervened with the groups often which gave the students little time for uptake of their suggestions or deeper discussion. Student turn overlap was higher without the presence of TAs. Maximum turn duration can be an important real‐time turn metric to identify the least verbally active student participant in a group. Implications for practice and/or policy TA training should include information about how to monitor groups for collaborative behaviours and when and how they should intervene to provide feedback and support. TA feedback systems should keep track of previous interventions by TAs (especially in contexts where there are multiple TAs facilitating) and the duration since previous intervention to ensure that TAs do not intervene with a group too frequently with little time for student uptake. Maximum turn duration could be used as a real‐time metric to identify the least verbally active student in a group so that support could be provided to them by the TAs.