Speaker Emotion Research Articles

Disfemisme Ungkapan Umpatan pada Terjemahan Arab-Indonesia Cerpen “طبلية من السماء” Karya Yusuf Idris

This article examines the use of dysphemism in swear words in the translation of Yusuf Idris's short story “Thobaliyyah Min As-Sama'” into Herpin Nopiandi Khurosan's short story “Tray from Heaven”. Dysfeminism refers to the use of language that is designed to emphasize negative connotations and make a statement sound worse or harsher. This study aims to determine the form of swear word dysphemism in the Arabic-Indonesian translation of the short story “Thobaliyyah Min As-Sama'”, knowing the function of swear word dysphemism in the Arabic-Indonesian translation of the short story “Thobaliyyah Min As-Sama”. The theory used in this research is Allan and Burridge's theory of dysphemism. The method of this research is descriptive qualitative, with 2 data collection techniques, namely reading the entire contents of the short story and recording the data in accordance with the theory. Then the data were analyzed using the Miles and Huberman method, namely by collecting data, reducing data, presenting data, and conclusions. The results of this study show that there are some data that show the existence of dysphemism in swear words. Among them have functions, namely, to vent the speaker's emotions, to joke, to demean the listener, and to swear. This research makes an important contribution to the understanding of how dysphemisms are translated and how shifts in language can affect the meaning and emotional impact of literary texts. For future research, it is hoped that other researchers can explore the comparison of dysphemisms in various literary genres or in the context of translation between different languages.

한국어 종결어미 ‘-네’와 중국어 어기조사 ‘啊(a), 呢(ne)’의 대조연구

This paper aims to first confirm that in declarative sentences, the Chinese modal particles ‘啊(a)’ and ‘呢(ne)’ correspond to the Korean ‘-ney(-네)’ with a distinction: ‘啊(a)’ expresses the speaker's emotions, while ‘呢(ne)’ indicates the speaker's content, meaning that the speaker wants the listener to pay attention to what is being said. Additionally, the current Korean ‘-ney(-네)’ is discussed within the categories of evidentiality and mirativity based on its semantic functions. This paper intends to discuss the evidentiality and mirativity of Korean sentence-final ending ‘-ney(-네)’ separately, and to reveal that the corresponding Chinese Modal Particles ‘啊(a)’ and ‘呢(ne)’ also fall within the categories of evidentiality and mirativity in declarative sentences. Evidentiality refers to the observation of information, which varies depending on the observer and is related to the deictic category of person. The Korean sentence-final ending ‘-ney(-네)’ generally has a first-person subject, whereas this study seeks to confirm that the Chinese ‘啊(a)’ and ‘呢(ne)’ are restricted by a second-person subject.

Maksim Relevansi pada Film “Calon Lurah” Karya Hajar Pamuji (Tinjauan Pragmatis)

This study aims to analyze the application of the maxim of relevance in the film "Calon Lurah" by Hajar Pamuji using a pragmatic approach and descriptive qualitative methods. The focus of this research is on the dialogues between the characters in the film, which demonstrate adherence to the cooperative principle and the maxim of relevance, where each speaker's contribution is relevant to the topic or context of the conversation. Three main types of utterances were found in this study: assertive, directive, and expressive utterances. Assertive utterances express something that the speaker believes to be a fact, while directive utterances aim to get the listener to do something, and expressive utterances convey the speaker's feelings or emotions. Primary data for this study was taken from transcribed dialogues in the film, while secondary data came from various sources such as books, journals, theses, and related research. Data collection techniques included observing and noting. The results showed that the film "Calon Lurah" effectively applies the maxim of relevance in the dialogues between its characters, making the conversations relevant and supporting the cooperative principle in communication. This research highlights the importance of relevance in communication to ensure effective and efficient conversations and underscores the practical application of this concept in various communication contexts.

Speech emotion recognition based on multi-feature speed rate and LSTM

Correctly recognizing speech emotions is of significant importance in various fields, such as healthcare and human–computer interaction (HCI). However, the complexity of speech signal features poses challenges for speech emotion recognition. This study introduces a novel multi-feature method for speech emotion recognition that combines short-and rhythmic features. Utilizing short-time energy, zero-crossing rate, and average amplitude difference, the proposed approach effectively addressed overfitting concerns by reducing feature dimensionality. Employing an (LSTM) network, the experiment achieved notable accuracy across diverse datasets. Specifically, the proposed method achieved an impressive accuracy of up to 98.47% on the CASIA dataset, 100% on the Emo-DB dataset, and 98.87% on the EMOVO dataset, demonstrating its capability to accurately discern speaker emotions across different languages and emotion classes. These findings underscore the significance of incorporating speech rate for emotional content recognition, which holds promise for application in HCI and auxiliary medical diagnostics.

Development of a method for recognizing emotions from a speech signal

The purpose of research is automatic recognition of the speaker's emotions, based on the processing of sound recordings intended for use in alarm systems when working with operators of locomotive crews and dispatch services.Methods. Human emotion recognition has been a rapidly developing area of research in recent years. Features of the vocal tract, such as sound power, formant frequencies, are used to detect certain emotions with good accuracy. A method was used to determine the signal energy by highlighting the dominant frequency. The work has developed a program code, on the basis of which an analysis of four emotions is given - anger, joy, fear and calm. The most important and difficult step is to determine the features most suitable for distinguishing emotions and the availability of databases. Collecting databases is a complex task requiring the manifestation of sincerity of emotions. Often, the collection of a database takes place in an artificial environment and the speech may sound staged; to eliminate such problems, it is necessary to use call center recordings.Results. Recordings of basic emotional states, such as anger, joy, sadness, fear and surprise, which are the most common case of the study, were obtained and processed. The developed software code allows us to get closer to automatically determining emotions from a speech signal. To analyze speech recordings in samples, indicators of signal energy and identification of the dominant frequency were used.Conclusion. The implemented method of monitoring the emotional state of a human operator using a speech signal is widely used in the prevention and improvement of indicators of the psychophysiological professional suitability of locomotive crew workers and the preservation of their professional health. Distinct differences are observed in the characteristics of all types of emotions.

2F3-4 Possibility of visualizing speaker emotions and state transitions from meeting utterance data

2F3-4 Possibility of visualizing speaker emotions and state transitions from meeting utterance data

Multimodal Sentiment Analysis for Movie Scenes Based on a Few-Shot Learning Approach

Multimodal sentiment analysis involves discerning the emotions of speakers through diverse features such as sound, text, and images. However, current research in this field heavily relies on extensive supervised datasets, demanding substantial labor and computational resources. This study introduces a meta-learning-based approach for multimodal sentiment analysis, aiming to delve into emotional information within movie scenes. Leveraging meta-learning techniques, this approach seeks to accurately capture emotions in movie scenes using a limited number of annotated samples, achieving model generalization under constrained labeled data. Specifically, we introduce an optimization-based meta-learning approach for the multimodal sentiment analysis tasks in text and vision, enhancing the model’s ability to generalize to new tasks with limited annotations. Additionally, an innovative strategy based on Meta-Prompting is proposed to handle multimodal data. The merits of the proposed method are validated across different datasets.

Alternating mechanism of person deixis in Persian interactions: A cognitive-dialogic account

By examining the employment of personal deictic terms in Persian interactions, this study focuses on the dynamic and emerging uses of them in order to shed some light on how to interpret the construction of interpersonal relations of Persian speakers. More importantly, it aims to account for the alternation mechanism between paradigmatic pronouns of lowering and elevating one's status within a single interaction. A cognitive-dialogic analysis of approximately 20 h of everyday conversations and videotaped broadcasts reveals that interlocutors' use of personal deictic terms is a fluid, emergent, and contingent practice, governed by at least six possible interactional factors: i) the speaker's emotions, ii) the speaker's situational intentions, iii) the prior speaker's turn, iv) the shift of discourse topic, v) bystander(s), and vi) the message content. Interlocutors' emergent shift from lowering into elevating terms and vice versa brings about a fluctuation of interpersonal relationships in the course of interaction. By taking these factors into account, this study shows that each usage of lowering and elevating terms conveys multiple, but distinct referential and indexical meanings about the interactants and the context in which it occurs. It also reveals that Persian speakers are aware of the potential indexical meanings of these terms, and that they use them (un)consciously and strategically to define, negotiate, or reshape their social relationships. Above all, this study argues that interlocutors' (inter)subjectivity essentially regulates the alternation of using lowering- and elevating-deictic forms.

Sounding Sentiments: A Cognitive AI Approach to Speaker-Linked Emotion Analysis

Abstract: Over the past few decades, there has been significant progress in sentiment analysis, primarily focusing on analyzing text. However, the field of sentiment analysis linked to audio remains relatively undeveloped in the scientific community. This study aims to address this gap by introducing sentiment analysis applied to voice transcripts, specifically focusing on distinguishing emotions of individual speakers in conversations. The proposed research article seeks to develop a sentiment analysis system capable of rapidly interacting with multiple users and analyzing the sentiment of each user's audio input. Key components of this approach include speech recognition, Mel-frequency cepstral coefficients (MFCC), dynamic time warping (DTW), sentiment analysis, and speaker recognition.

Հուզական խոսքային ակտերի ուսուցումը հուզագնահատողական արտահայտությունների միջոցով / Teaching Expressive Speech Acts Through Emotional-Attitudinal Expressions

The present article explores the integration of expressive speech acts in the communicative learning process of a foreign language, primarily English. Speech acts play a significant pragmatic role in communication, and understanding how to perform them appropriately according to the context is essential. However, many general education textbooks for foreign languages lack activities that help learners perceive and use speech acts effectively, which hinders the development of their communicative skills. The paper specifically focuses on expressive speech acts, which bridge the speaker's emotions and the communicative intent. We view emotional-attitudinal expressions as distinct linguistic units for teaching expressive speech acts, for they serve as a tool for infusing an emotional tone into speech and elucidating the speaker's thoughts and feelings. To address this, we provide a brief analysis of communicative language competence and its components, discuss various types of speech acts, and highlight the importance of emotional-attitudinal expressions in both communication and foreign language learning. The article concludes with a set of exercises designed to enhance learners' understanding of speech acts and emotional-attitudinal expressions, guiding them towards effective use in foreign language communication.

Multitask and Transfer Learning Approach for Joint Classification and Severity Estimation of Dysphonia.

Despite speech being the primary communication medium, it carries valuable information about a speaker's health, emotions, and identity. Various conditions can affect the vocal organs, leading to speech difficulties. Extensive research has been conducted by voice clinicians and academia in speech analysis. Previous approaches primarily focused on one particular task, such as differentiating between normal and dysphonic speech, classifying different voice disorders, or estimating the severity of voice disorders. This study proposes an approach that combines transfer learning and multitask learning (MTL) to simultaneously perform dysphonia classification and severity estimation. Both tasks use a shared representation; network is learned from these shared features. We employed five computer vision models and changed their architecture to support multitask learning. Additionally, we conducted binary 'healthy vs. dysphonia' and multiclass 'healthy vs. organic and functional dysphonia' classification using multitask learning, with the speaker's sex as an auxiliary task. The proposed method achieved improved performance across all classification metrics compared to single-task learning (STL), which only performs classification or severity estimation. Specifically, the model achieved F1 scores of 93% and 90% in MTL and STL, respectively. Moreover, we observed considerable improvements in both classification tasks by evaluating beta values associated with the weight assigned to the sex-predicting auxiliary task. MTL achieved an accuracy of 77% compared to the STL score of 73.2%. However, the performance of severity estimation in MTL was comparable to STL. Our goal is to improve how voice pathologists and clinicians understand patients' conditions, make it easier to track their progress, and enhance the monitoring of vocal quality and treatment procedures. Clinical and Translational Impact Statement: By integrating both classification and severity estimation of dysphonia using multitask learning, we aim to enable clinicians to gain a better understanding of the patient's situation, effectively monitor their progress and voice quality.

SEGMENTAL UNITS AS AN INDICATOR OF EMOTIONAL PERCEPTION OF THE SOUNDING SPEECH

Problem Statement. Our research refers to the theory of emotionality and focuses on expressive means of sounding speech. When considering expressive means of sounding speech, we refer to the segmental level, namely to the distinctive features of vowel phonemes. We suggest that the complexes of semantic features of vowel phonemes and the frequency of their realization is one of the indicators of emotional perception and its correlation with a particular emotion. Besides it affects the perception of sounding speech in a particular emotional way, regardless of its semantics. The author has described and developed the mechanism of psycholinguistic experiment on revealing the perception of emotionally colored speech in connection with the functioning of the units of the language system in their differential attributes in the perception of sounding speech. Results.The results of the research showed that the distinctive features of vowel phonemes may be one of the indicators of correlation of emotional perception with a particular emotion. Moreover, when emotionality increases, one should expect regular and systematic changes of means for its expression. Furthermore, the polar basic emotions of both German and Russian speakers may coincide in the perception of the same material, which can be explained by the presence of universal and culture-bound specifics of emotional perception of the distinctive characteristics of vowels. Conclusion. Due to the results of the research, we may conclude that a systematic approach to the study of the phenomena of speech is possible because of the fact that the relationship between the functioning of the distinctive characteristics of vowel phonemes and the emotional coloring of the sounding text has a systemic nature.

ANALYSIS OF LANGUAGE FUNCTIONS IN THE SHORT STORY THE SUMMER SOLSTICE BY NICK JOAQUIN

Language and literature are essentially connected, and one cannot function without the other. Thus, this study analyzes the language functions used by the main characters in the short story The Summer Solstice by Nick Joaquin. This study aims to determine the types of Language functions in the short story and identify the most dominant language function used by the main characters. Moreover, the study also aims to analyze the context of the language functions identified. The researchers used Janet Holmes’ Theory of Language Function (2013) to analyze the utterances of the main characters and Dell Hymes' SPEAKING Model to analyze the context of the language functions in each utterance. The researchers then applied the descriptive qualitative design to systematically analyze the data. The result of this study shows that out of the eight (8) language functions identified by Janet Holmes (2013), only seven (7) types of language functions were found in the short story, namely expressive, directive, heuristic, referential, metalinguistic, phatic functions. On the other hand, poetic function was not found in this research. The findings revealed that this story dominantly uses expressive functions, in which the speaker's emotion is what captures the interest of the readers.

SER: Performance Evaluation of CNN Model Along with an Overview of Available Indic Speech Datasets, and Transition of Classifiers From Traditional to Modern Era

Speech emotion recognition (SER) is a rapidly evolving field in affective computing and human-computer interaction. In general, a SER system extracts and classifies prominent elements called features from a pre-processed speech signal to target the presence of speaker's certain emotion. This paper explores the utilization of deep learning classifiers in SER and surveys available datasets in both Indic and international languages. The paper highlights the significance of SER in enhancing human-computer interaction and presents deep learning as an effective approach to handle the complexity of speech signals. Various deep learning architectures, including Convolution Neural Networks (CNNs), Recurrent Neural Network (RNNs), and hybrid models, are analysed in terms of training methodology, and performance on benchmark datasets. Additionally, the paper conducts a comprehensive survey of publicly available datasets for speech emotion recognition, considering emotional categories, language diversity, recording conditions, and sample sizes. Challenges in adapting deep learning models to these datasets, such as data augmentation and cross-lingual transfer learning, are discussed. Moreover, the CNN based model is analysed on accuracy, precision, recall and F-1 score on Ryerson Audio-Visual Database of Emotional Speech and Song (RAVDESS) dataset with the value 84%, 85%, 84% and 84% resp. The review concludes with key findings, emphasizing the strengths and limitations of deep learning classifiers for SER. It identifies the need for standardized evaluation protocols, exploration of transfer learning across languages, and development of robust and culturally diverse datasets as future research directions.

Speech Emotion Detection Through Live Calls

Abstract: Speech emotion recognition is a popular study area right now, with the goal of enhancing human-machine connection. Most of the research being done in this field now classifies emotions into different groups by extracting discriminatory features. Most of the work done nowadays concerns verbal expressions used for lexical analysis and emotion recognition. In our project, emotions are categorized into the following categories: angry, calm, fearful, happy, and sad. Speech Emotion Recognition, often known as, SER, is a technology that takes advantage of the fact that tone and pitch in a speech frequently convey underlying emotions. The approach to assessing or anticipating a speaker's gender and emotions from their speech has been given in the proposed work. By graphing the waveform and spectrogram, convolutional neural networks are used to evaluate or predict gender and emotions. A CNN model is created using the input of 12162 samples to ordering to identify the emotions present in the speech. In our study, the suggested model's overall accuracy is calculated using only one feature, the MFCC from the speech, and the 4 datasets (RAVDESS, SAVEE, CREAMA-D, and TESS). The accuracy is first calculated for each emotion and gender before the overall accuracy is discovered.

Identity Retention and Emotion Converted StarGAN for low-resource emotional speaker recognition

In speaker recognition (SR) task, the inconsistency of emotion between enrolled speech and test speech will lead to emotion domain mismatch, which decreases the recognition accuracy and is more serious under low-resource training data condition. To help SR model learn the various emotion distribution when only neutral speech is enrolled, Identity Retention and Emotion Converted StarGAN (IREC-StarGAN) is proposed to generate speech feature in other emotions. Emotion and speaker relevant multi-task loss is designed to retain origin speaker information during adversarial training progress for the generation of emotion feature. To avoid essential speaker or emotion information being contaminated by irrelevant regions, generator uses SC-bottleneck with multi kernel size channels to learn the deep feature of data in various scales. Discriminator adopts densenet with two branches, one of which classifies whether input feature belongs to target emotion domain and the another one focuses on the speaker information. The origin neutral feature will be enrolled with generated feature together to be recognized by Resnet18 for speaker recognition task. The proposed framework obtains the EER of 18.33% on Mandarin Affective Speech Corpus and 8.14% on IEMOCAP, which outperform the existing embedding and GAN methods and reveals the robustness of IREC-StarGAN under low-resource emotion condition.

MMATERIC: Multi-Task Learning and Multi-Fusion for AudioText Emotion Recognition in Conversation

The accurate recognition of emotions in conversations helps understand the speaker’s intentions and facilitates various analyses in artificial intelligence, especially in human–computer interaction systems. However, most previous methods need more ability to track the different emotional states of each speaker in a dialogue. To alleviate this dilemma, we propose a new approach, Multi-Task Learning and Multi-Fusion AudioText Emotion Recognition in Conversation (MMATERIC) for emotion recognition in conversation. MMATERIC can refer to and combine the benefits of two distinct tasks: emotion recognition in text and emotion recognition in speech, and production of fused multimodal features to recognize the emotions of different speakers in dialogue. At the core of MATTERIC are three modules: an encoder with multimodal attention, a speaker emotion detection unit (SED-Unit), and a decoder with speaker emotion detection Bi-LSTM (SED-Bi-LSTM). Together, these three modules model the changing emotions of a speaker at a given moment in a conversation. Meanwhile, we adopt multiple fusion strategies in different stages, mainly using model fusion and decision stage fusion to improve the model’s accuracy. Simultaneously, our multimodal framework allows features to interact across modalities and allows potential adaptation flows from one modality to another. Our experimental results on two benchmark datasets show that our proposed method is effective and outperforms the state-of-the-art baseline methods. The performance improvement of our method is mainly attributed to the combination of three core modules of MATTERIC and the different fusion methods we adopt in each stage.

Translation Report on Emma Watson's "HeForShe" Persuasive Speech at the United Nations

Persuasive speech is a common speech-type in English to Chinese consecutive interpreting. Unlike informative speeches, the persuasive speech serves to try to convert the audience to the speaker's point of view, instead of informing the audience of a fact that they don't know or don't know much about. Persuasive speech, therefore, has a higher tone, and the speaker's emotions are more impassioned, making it more interactive with the audience. The thesis takes the interpretation of Emma Watson's "HeForShe" speech on gender equality delivered at the United Nations as an example, and analyzes the techniques and strategies used in the interpreting, taking into account the experience in conference interpreting simulation.

Emotional Speaker Verification Using Novel Modified Capsule Neural Network

Capsule Neural Network (CapsNet) models are regarded as efficient substitutes for convolutional neural networks (CNN) due to their powerful hierarchical representation capability. Nevertheless, CNN endure their inability of recording spatial information in spectrograms. The main constraint of CapsNet is related to the compression method which can be implemented in CNN models but cannot be directly employed in CapsNet. As a result, we propose a novel architecture based on dual-channel long short-term memory compressed CapsNet (DC-LSTM–COMP CapsNet) for speaker verification in emotional as well as stressful talking environments. The proposed approach is perceived as a modified Capsule network that attempts to overcome the limitations that exist within the original CapsNet, as well as in CNN while enhancing the verification performance. The proposed architecture is assessed on four distinct databases. The experimental analysis reveals that the average speaker verification performance is improved in comparison with CNN, the original CapsNet, as well as the conventional classifiers. The proposed algorithm notably achieves the best verification accuracy across the four speech databases. For example, using the Emirati dataset, the average percentage equal error rates (EERs) obtained is 10.50%, based on the proposed architecture which outperforms other deep and classical models.

SPEECH EMOTION RECOGNITION

Speech Emotion Recognition, abbreviated as SER, is the act of attempting to recognize human emotion and affective states from speech. This is capitalizing on the fact that voice often reflects underlying emotion through tone and pitch. This is also the phenomenon that animals like dogs and horses employ to be able to understand human emotion. SER is tough because emotions are subjective and annotating audio is challenging.The main objective of this SER, recognizing the emotional state of the speaker from his/her speaker. Various emotions like happiness, sadness, anger, etc. can be recognized. SER uses speech processing to recognize the emotional state. Speech Processing is one of the important branches of digital signal processing and finds applications in Humancomputer interfaces, Telecommunication, Assistive technologies, Audio mining, Security, and so on. Speech Emotion Recognition is important to have a natural interaction between human beings and machines. In SER, the emotional state of a speaker is extracted from his/her speech. The acoustic characteristic of the speech signal is Feature. Feature extraction is the process that extracts a small amount of data from the speech signal that can later be used to represent each speaker. Many feature extraction methods are available and Mel Frequency Cepstral Coefficient (MFCC) is the commonly used method. In this project, speaker emotions are recognized using the data extracted from the speaker's voice signal. Mel Frequency Cepstral Coefficient (MFCC) technique is used to recognize the emotion of a speaker from their voice.