Utterance Classification Research Articles

Automatic classification of emotions in speech: methods and data

The subject of this study is the data and methods used in the task of automatic recognition of emotions in spoken speech. This task has gained great popularity recently, primarily due to the emergence of large datasets of labeled data and the development of machine learning models. The classification of speech utterances is usually based on 6 archetypal emotions: anger, fear, surprise, joy, disgust and sadness. Most modern classification methods are based on machine learning and transformer models using a self-learning approach, in particular, models such as Wav2vec 2.0, HuBERT and WavLM, which are considered in this paper. English and Russian datasets of emotional speech, in particular, the datasets Dusha and RESD, are analyzed as data. As a method, an experiment was conducted in the form of comparing the results of Wav2vec 2.0, HuBERT and WavLM models applied to the relatively recently collected Russian datasets of emotional speech Dusha and RESD. The main purpose of the work is to analyze the availability and applicability of available data and approaches to recognizing emotions in speech for the Russian language, for which relatively little research has been conducted up to this point. The best result was demonstrated by the WavLM model on the Dusha dataset - 0.8782 dataset according to the Accuracy metric. The WavLM model also received the best result on the RESD dataset, while preliminary training was conducted for it on the Dusha - 0.81 dataset using the Accuracy metric. High classification results, primarily due to the quality and size of the collected Dusha dataset, indicate the prospects for further development of this area for the Russian language.

Illocutionary Act Analysis of Melati and Isabel Wijsen Speech at United Nations

This study was conducted to analyze the illocutionary act used in Melati and Isabel Wijsen’s United Nations Speech. This speech was performed on World Oceans Day 2017. From the illocutionary act that was analyzed, the readers are able to learn that utterances also play role as actions that can influence people to do something. The qualitative descriptive method was used in this study to collect and analyze the data, followed the model from (Miles et al., 2014). This qualitative model consists of data condensation, data display, and drawing conclusion. The researcher used the illocutionary act types that were proposed by (Searle, 1976) as the grounded theory in making the classification of illocutionary utterances found in Melati and Isabel Wijsen’ speech. As a result, researcher found 18 assertives (56, 25%), 7 declaratives (21, 88%), 4 commissives (12, 5%), 3 expressives (9, 38%), 0 declaratives (0%). From the results found, Melati and Isabel Wijsen tend to use their speech to convince the audience by informing, believing, reporting, showing, and stating. This study suggests that applying illocutionary acts could make a speech more powerful.

A deep learning approach to dysarthric utterance classification with BiLSTM-GRU, speech cue filtering, and log mel spectrograms

A deep learning approach to dysarthric utterance classification with BiLSTM-GRU, speech cue filtering, and log mel spectrograms

ТАКСОНОМИЯ ПАРЕМИЙ В КИТАЙСКОМ ЯЗЫКОВЕДЕНИИ: СТРУКТУРНО-СЕМАНТИЧЕСКИЕ И ЛИНГВОПРАГМАТИЧЕСКИЕ АСПЕКТЫ

The article systematizes linguistic approaches to the classification of Chinese language utterances and ready-made expressions. Stable expressions, known in Chinese linguistics as shuyu, are an integral part of language and culture. Based on the analysis of theoretical works on Chinese paremiology, as well as the author's observation of linguistic facts, the formal-structural and speech features of several varieties of shuyu (yanyu, suyu, sehoyu, chengyu, guanyungyu) are considered, the linguapragmatic properties of each of these types of speech expressions are described, and specific examples are considered. The results of the study contribute to the modern theory of Chinese paremiology, and will also be useful as a theoretical source for comparative studies of Chinese and Russian paremias.

Pragmatic Semantics in Football Fans’ «Banner» Communication
 in the German Language

The article deals with utterances of German football fans written on banners used by them during matches as a means of communication. Football fans’ discourse is a distinctive linguistic semiotic form that reflexes the world view of representatives
 of this subculture. Despite the diversity of discourses it intersects with and a variety of concepts it exchanges with, it is the fan communication that provides the sports discourse with uniqueness and allows us to regard it as an integral part of the sports linguistic view of the world. Football fans’ discourse is formed by a number of subdiscourses, the so-called banner communication is one of them, possessing a prominent
 perlocutionary effect. The classification of utterances is based upon the theory of speech acts by J. Austin and G. Searle, but it also takes into consideration the latest achievements in this field as well as contemporary principles of pragmatic semantics.
 The article reveals the most effective techniques and mechanisms of building utterances to achieve the required pragmatic effect (metaphor, play on words, precedence, etc.),

Multi-Modal Sarcasm Detection and Humor Classification in Code-Mixed Conversations

Sarcasm detection and humor classification are inherently subtle problems, primarily due to their dependence on the contextual and non-verbal information. Furthermore, existing studies in these two topics are usually constrained in non-English languages such as Hindi, due to the unavailability of qualitative annotated datasets. In this work, we make two major contributions considering the above limitations: (1) we develop a Hindi-English code-mixed dataset, MaSaC, for the multi-modal sarcasm detection and humor classification in conversational dialog, which to our knowledge is the first dataset of its kind; (2) we propose MSH-COMICS, a novel attention-rich neural architecture for the utterance classification. We learn efficient utterance representation utilizing a hierarchical attention mechanism that attends to a small portion of the input sentence at a time. Further, we incorporate dialog-level contextual attention mechanism to leverage the dialog history for the multi-modal classification. We perform extensive experiments for both the tasks by varying multi-modal inputs and various submodules of MSH-COMICS. We also conduct comparative analysis against existing approaches. We observe that MSH-COMICS attains superior performance over the existing models by >1 F1-score point for the sarcasm detection and 10 F1-score points in humor classification. We diagnose our model and perform thorough analysis of the results to understand the superiority and pitfalls.

Identifying Discourse Elements in Writing by Longformer for NER Token Classification

Current automatic writing feedback systems cannot distinguish between different discourse elements in students' writing. This is a problem because, without this ability, the guidance provided by these systems is too general for what students want to achieve on arrival. This is cause for concern because automated writing feedback systems are a great tool for combating student writing declines. According to the National Assessment of Educational Progress, less than 30 percent of high school graduates are gifted writers. If we can improve the automatic writing feedback system, we can improve the quality of student writing and stop the decline of skilled writers among students. Solutions to this problem have been proposed, the most popular being the fine-tuning of bidirectional encoder representations from Transformers models that recognize various utterance elements in student written assignments. However, these methods have their drawbacks. For example, these methods do not compare the strengths and weaknesses of different models, and these solutions encourage training models over sequences (sentences) rather than entire articles. In this article, I'm redesigning the Persuasive Essays for Rating, Selecting, and Understanding Argumentative and Discourse Elements corpus so that models can be trained for the entire article, and I've included Transformers, the Long Document Transformer's bidirectional encoder representation, and the Generative Improving a pre trained Transformer 2 model for utterance classification in the context of a named entity recognition token classification problem. Overall, the bi-directional encoder representation of the Transformers model railway using my sequence-merging preprocessing method outperforms the standard model by 17% and 41% in overall accuracy. I also found that the Long Document Transformer model performed the best in utterance classification with an overall f-1 score of 54%. However, the increase in validation loss from 0.54 to 0.79 indicates that the model is overfitting. Some improvements can still be made due to model overfittings, such as B. Implementation of early stopping techniques and further examples of rare utterance elements during training.

Hate Speech in Internet Communication

The article is devoted to the consideration of the qualifying signs of "hate speech" in the aspect of providing linguistic and forensic expertise in the field of Internet communication in order to ensure the legal and linguistic protection of national security, the constitutional order, public order, health and morality of the population. This concept – «hate speech» - is the subject of scientific discussions in various fields of science and practice: sociology, psychology, linguistics, theory of law, judicial practice. The article analyzes the positions of scientists in relation to the content of the concept of "hate speech". The classification of speech utterances containing signs of "hate speech" is given, which allows focusing the attention of experts - criminologists, conducting linguistic examination of statements in Internet communications, on soft forms of hate speech, the prevention of which is designed to ensure the prevention of crimes and offenses of an extremist orientation. The article also provides examples from the practice of conducting linguistic expertise, speech utterances in text files from an Internet resource.

Spoken Utterance Classification Task of Arabic Numerals and Selected Isolated Words

Spoken Utterance Classification Task of Arabic Numerals and Selected Isolated Words

Classification of Utterances that Lead to Dialogue Breakdowns in Chat-oriented Dialogue Systems

Classification of Utterances that Lead to Dialogue Breakdowns in Chat-oriented Dialogue Systems

Вiды сiнтаксiчна нячленных выказванняў нязгоды ў славянскiх мовах: семантычны аспект

The aim of the study was to systematize syntactically indivisible utterances of disagreement in the Belarusian, Ukrainian, Czech and Bulgarian languages. The goal was achieved by using the linguistic comparison method, contextual analysis, elements of component analyss of words semantics, techniques of generalization, interpretation and classification. As a result, a classification of disagreement utterances was created, and the classification was presented in the form of a table.

Prosody-Based Measures for Automatic Severity Assessment of Dysarthric Speech

One of the first cues for many neurological disorders are impairments in speech. The traditional method of diagnosing speech disorders such as dysarthria involves a perceptual evaluation from a trained speech therapist. However, this approach is known to be difficult to use for assessing speech impairments due to the subjective nature of the task. As prosodic impairments are one of the earliest cues of dysarthria, the current study presents an automatic method of assessing dysarthria in a range of severity levels using prosody-based measures. We extract prosodic measures related to pitch, speech rate, and rhythm from speakers with dysarthria and healthy controls in English and Korean datasets, despite the fact that these two languages differ in terms of prosodic characteristics. These prosody-based measures are then used as inputs to random forest, support vector machine and neural network classifiers to automatically assess different severity levels of dysarthria. Compared to baseline MFCC features, 18.13% and 11.22% relative accuracy improvement are achieved for English and Korean datasets, respectively, when including prosody-based features. Furthermore, most improvements are obtained with a better classification of mild dysarthric utterances: a recall improvement from 42.42% to 83.34% for English speakers with mild dysarthria and a recall improvement from 36.73% to 80.00% for Korean speakers with mild dysarthria.

Recognition of speech emotion using custom 2D-convolution neural network deep learning algorithm

Speech emotion recognition has become the heart of most human computer interaction applications in the modern world. The growing need to develop emotionally intelligent devices has opened up a lot of research opportunities. Most researchers in this field have applied the use of handcrafted features and machine learning techniques in recognising speech emotion. However, these techniques require extra processing steps and handcrafted features are usually not robust. They are computationally intensive because the curse of dimensionality results in low discriminating power. Research has shown that deep learning algorithms are effective for extracting robust and salient features in dataset. In this study, we have developed a custom 2D-convolution neural network that performs both feature extraction and classification of vocal utterances. The neural network has been evaluated against deep multilayer perceptron neural network and deep radial basis function neural network using the Berlin database of emotional speech, Ryerson audio-visual emotional speech database and Surrey audio-visual expressed emotion corpus. The described deep learning algorithm achieves the highest precision, recall and F1-scores when compared to other existing algorithms. It is observed that there may be need to develop customized solutions for different language settings depending on the area of applications.

Sequential neural networks for noetic end-to-end response selection

The noetic end-to-end response selection challenge as one track in the 7th Dialog System Technology Challenges (DSTC7) aims to push the state of the art of utterance classification for real world goal-oriented dialog systems, for which participants need to select the correct next utterances from a set of candidates for the multi-turn context. This paper presents our systems that are ranked top 1 on both datasets under this challenge, one focused and small (Advising) and the other more diverse and large (Ubuntu). Previous state-of-the-art models use hierarchy-based (utterance-level and token-level) neural networks to explicitly model the interactions among different turns’ utterances for context modeling. In this paper, we investigate a sequential matching model based only on chain sequence for multi-turn response selection. Our results demonstrate that the potentials of sequential matching approaches have not yet been fully exploited in the past for multi-turn response selection. In addition to ranking top 1 in the challenge, the proposed model outperforms all previous models, including state-of-the-art hierarchy-based models, on two large-scale public multi-turn response selection benchmark datasets.

Classification of humorous interactions with intelligent personal assistants

The study examined humorous interactions with intelligent personal assistants (IPAs, including Google Assistant, Amazon Alexa, Microsoft Cortana, Apple Siri) with the aim of classifying user utterances, IPA responses and user reactions of system responses. Data from online diaries and paper questionnaires were collected and analyzed using content analysis method. The findings suggest that the most frequent types of utterances include questions that test system “personality” and opinions. Joke requests are also frequent and produce pre-programmed humor that users generally find funny. The initial classification of humorous utterances has been validated and expanded using published datasets of humorous utterances for the four investigated IPAs. The findings can be used for immediate improvements to IPA performance as well as long-term development of IPA personas.

Observability of Inter-Organizational Crisis Management Capability

Effective responses to severe strains and large-scale disasters demand efficient crisis management, i.e., capabilities that depend on inter-organizational collaborations. Inter-organizational exercises are performed to maintain and develop the inter-organizational crisis management (ICM) capability. In a previous review of scientific literature, we identified nine themes (i.e., aspects) of ICM capability (ICMC): interaction, coordination/C2, decision making, relationships, situation awareness (SA), resilience, preparedness, system performance, and information infrastructure. This paper presents empirical testing of the nine ICMC aspects, by observational studies in two ICM exercises. The ICMC aspects were implemented in a structured observation protocol, which allowed observations based on classification of single utterances and actions. All ICMC aspects except system performance were observed. Actions related to SA, interaction, and coordination/C2 were frequently observed, while relations, resilience, and preparedness were observed to a lesser extent. In addition, exercise evaluators rated the relevance of all nine ICMC aspects as high.

Native Language Identification in Very Short Utterances Using Bidirectional Long Short-Term Memory Network

Native language identification (NLI) is the task of identifying the first language of a user based on their speech or written text in a second language. In this paper, we propose the use of spectrogram- and cochleagram-based features extracted from very short speech utterances (0.8 s on average) to infer the native language of an Urdu speaker. The bidirectional long short-term memory (BLSTM) neural networks are adopted for the classification of utterances among the native languages. A set of experiments is carried out for the network architecture search and the system’s accuracy is evaluated on the validation data set. Overall accuracy of 74.81% and 71.61% is achieved using the Mel-frequency cepstral coefficients (MFCC) and Gammatone frequency cepstral coefficients (GFCC), respectively. Moreover, the optimized MFCC feature-based BLSTM network and GFCC feature-based BLSTM network are merged together to take advantage of both the feature sets. The experiments show that the performance of the merged network surpasses the individual BLSTM networks and accuracy of 75.69% is achieved on the evaluation data. The effect of test data duration is also analyzed (from 0.27 s to 1.5 s); in addition, it is observed that with very short duration as 0.4 s, an accuracy of over 50% can be achieved.

A construção de uma escala sobre as concepções de deficiência: procedimentos metodológicos

Resumo Este artigo parte dos preceitos da psicologia histórico-cultural para analisar como a deficiência é conceituada e suas implicações. Procura descrever a elaboração de um procedimento metodológico para averiguar concepções de deficiência. Feita a revisão da literatura nacional e internacional, foram circunscritas quatro concepções: orgânica, psicossocial, histórico-cultural e metafísica. Para cada uma delas foram definidas cinco asserções, dispostas em cinco pontos ordinais. A escala foi avaliada por juízes com familiaridade teórico-metodológica na temática deficiência. Foi calculado o índice de concordância entre as classificações originais dos enunciados com as marcações e retorno e revisão das classificações discordantes. Aplicações iniciais e os tratamentos decorrentes permitiram a confecção da Escala de Concepções de Deficiência (ECD).

In-Vehicle Voice Interface with Improved Utterance Classification Accuracy Using Off-the-Shelf Cloud Speech Recognizer

In-Vehicle Voice Interface with Improved Utterance Classification Accuracy Using Off-the-Shelf Cloud Speech Recognizer

Robust Visual Lips Feature Extraction Method for Improved Visual Speech Recognition System

Recently, automatic lips reading ALR acquired a significant interest among many researchers due to its adoption in many applications. One such application is in speech recognition system in noisy environment, where visual cue that contain some integral information added to the audio signal, as well as the way that person merges audio-visual stimulus to identify utterance. The unsolved part of this problem is the utterance classification using only the visual cues without the availability of acoustic signal of the talker's speech. By taking into considerations a set of frames from recorded video for a person uttering a word; a robust image processing technique is used to isolate the lips region, then suitable features are extracted that represent the mouth shape variation during speech. These features are used by the classification stage to identify the uttered word. This paper is solve this problem by introducing a new segmentation technique to isolate the lips region together with a set of visual features base on the extracted lips boundary which able to perform lips reading with significant result. A special laboratory is designed to collect the utterance of twenty six English letters from a multiple speakers which are adopted in this paper (UOTEletters corpus). Moreover; two type of classifier (using Numeral Virtual generalization (NVG) RAM and K nearest neighborhood KNN) where adopted to identify the talker’s utterance. The recognition performance for the input visual utterance when using NVG RAM is 94.679%, which is utilized for the first time in this work. While; 92.628% when KNN is utilize.