Speech Information Research Articles

The Urgency to Use the Rabat Plan of Action in the Hate Speech Offense

The broad spectrum of hate speech makes it difficult for law enforcement officials to identify which speech can be punished. This difficulty creates uncertainty in law enforcement and potentially triggers injustice. Therefore, this article emphasizes the importance of assessing the multidimensional aspects accommodated in the Rabat Plan Action instrument to determine hate speech. The Rabat Plan of Action instrument is one of the international human rights instruments drafted to handle acts of hate speech without regard to the right to freedom of expression. The Rabat Plan of Action has six elements: context, position and status of the speaker, intention, content and form of speech, range, and possibility and potential for harm. In this article, Rabat Plan of Action was applied to the case of Bambang Tri Mulyono, who believed that President Joko Widodo’s diploma was fake and his speech had been uploaded on a YouTube video. The simulation results confirm that the Rabat Plan of Action can effectively provide directions for extracting legal facts and related events so that law enforcement officials can find clues in determining the occurrence of hate speech. Based on these results, the Rabat Plan of Action should be adopted at the national level to complement the legal instruments for handling acts of hate speech.

TINDAK TUTUR EKSPRESIF GURU DAN PESERTA DIDIK DALAM PROSES PEMBELAJARAN DI TK IT AL-BIRUNI MANDIRI JIPANG DI KOTA MAKASSAR

This study aims to describe the forms of expressive speech acts of teachers and students as well as describe the strategies of teacher speech acts towards students in the learning process at Al-Biruni Mandiri Jipang IT Kindergarten in Makassar City. Type of descriptive qualitative research. Data in the form of written data from voice recordings. The data source is teacher and student speech. Data collection techniques use listening, documentation and note-taking The results of the research revealed that there were 11 expressive speech acts by teachers and 1 expressive speech act by students and 6 strategies were found by teachers in the learning process.

British Conversation is Changing: Resonance and Engagement in the BNC1994 and the BNC2014

Abstract This applied study assesses the degree to which speakers re-use and acknowledge parts of one another’s utterances. This form of alignment is called resonance (DuBois 2014; Tantucci and Wang 2021), and is a decisive indicator of creativity and verbal engagement. Consistent absence of resonance indicates interactional detachment, which is distinctive of autistic speech (Tantucci and Wang 2023). We analysed resonance in naturalistic interaction among British speakers in the demographically sampled sections of the British National Corpora of English: the BNC1994 and the BNC2014. We controlled for creativity, age, class, gender, context, dialect, and intra-generational speech for 1,600 turns of informal speech. We discovered that upper-class people from the corporate world and neighbouring sectors mutually resonated much more in 2014 than they used to in 1994. This may be due to the dramatic change in corporate and institutional communication in the 2000s, involving a new turn towards corporate social responsibility, participatory frameworks in higher education, and the enactment of ideologies such as inclusivity, engagement, and equality in higher social grades of British society. This plausibly affected not only the system of values of those communities but also their interactional behaviour, now increasingly geared towards overt acknowledgement of other people’s talk.

Англоязычный авторский трэвел-блог: жанровые, структурные и прагмастилистические особенност

The aim of the article is to identify genre, structural, pragmatic and stylistic features of English travel blogs. Travel blog as a genre is a form of Internet communication that combines elements of different registers; it is marked by its dialogic, interactive and polycode features, that is a combination of verbal and non-verbal components serving informative, appealing and persuasive purposes. The article uses discourse, pragmastylistic and contextual analyses. It is obvious that the genre features of analyzed messages of the travel blog are speech forms combining neutral and colloquial registers. The blog structure is horizontal with themes arranged in line with posts links behind them. The posts themselves have no rigid structure, the mandatory parts are headlines and subheadlines – rubrics and the final part containing either impressions of the trip, advertising a certain destination and sponsor, or appeal to the readers to discuss the contents and persuade them to go to the described places. The main part of the blog can have rubrics (tips or description of sights and their evaluation); it can be the interchange of trip experiences with their evaluation and photos for each of those. The pragmastylistic features of the narrative are the syntactic means serving expressiveness and stimulating interest in the blog content, as well as in future trips: parallel constructions, inversion, parcellation. Lexical means create an atmosphere of real communication and trustworthiness: the use of abbreviated morphological and grammatical forms, words formed by conversion, idioms, interjections; phonetic-graphic means include graphon, emoticons, multiplication, capitalisation to accentuate certain parts of the utterance and convey emotions. The paper concludes that both linguistic and extralinguistic means reflect the key genre features and serve the author’s purpose – to get and keep the readers interested, to persuade them to act. The research material were 113 pragmastylistic language units from 15 posts of an English travel blog, where the author shares with the subscribers her impressions of travelling in Europe, Asia and Latin America. The American blogger and freelance journalist Ciara Jonson was chosen due to her blogs being ranked as one of the best travel blogs by travel editions and other bloggers.

HA-GCEN: Hyperedge-abundant graph convolutional enhanced network for hate speech detection

The proliferation of online social networks (OSNs) has led to the rampant spread of hate speech. However, traditional detection methods often struggle to effectively detect various forms of hate speech with satisfactory performance, primarily because these methods typically rely on graph-based models that tend to focus on pairwise relationships, thus failing to fully exploit the contextual and user-specific information that could unveil more subtle forms of hate speech. However, establishing a complete graph inevitably introduces considerable computational overhead and redundant information. To overcome these limitations, this study introduces a hyperedge-abundant graph convolutional enhanced network (HA-GCEN) learning framework for hate speech detection (HSD) in OSNs. The proposed hypergraph construction method with a hypergraph convolutional enhanced network primarily consists of three content-, relation-, and semanteme-hyperedge components. These components were designed to enhance context sensitivity, to comprehensively improve the understanding of group relationships and detect latent hate speech. Furthermore, the HA-GCEN was carefully designed to extract high-level correlations from the constructed hypergraph through hypergraph convolutional layers. The efficiency of the proposed method was validated on two benchmark datasets, SemEval2019 task 5 and FUNC, achieving significant improvements over state-of-the-art methods with increases of 5.74% and 2.56% in the F1 score, 5.43% and 1.47% in precision, and 5.98% and 3.47% in recall, respectively. These results attest to HA-GCEN’s advanced feature mining and learning capabilities, demonstrating its potential for more effective HSD within OSNs.

Speech Emotion Recognition under Noisy Environments with SNR Down to −6 dB Using Multi-Decoder Wave-U-Net

A speech emotion recognition (SER) model for noisy environments is proposed, by using four band-pass filtered speech waveforms as the model input instead of the simplified input features such as MFCC (Mel Frequency Cepstral Coefficients). The four waveforms retain the entire information of the original noisy speech while the simplified features keep only partial information of the noisy speech. The information reduction at the model input may cause the accuracy degradation under noisy environments. A normalized loss function is used for training to maintain the high-frequency details of the original noisy speech waveform. A multi-decoder Wave-U-Net model is used to perform the denoising operation and the Wave-U-Net output waveform is applied to an emotion classifier in this work. By this, the number of parameters is reduced to 2.8 M for inference from 4.2 M used for training. The Wave-U-Net model consists of an encoder, a 2-layer LSTM, six decoders, and skip-nets; out of the six decoders, four decoders are used for denoising four band-pass filtered waveforms, one decoder is used for denoising the pitch-related waveform, and one decoder is used to generate the emotion classifier input waveform. This work gives much less accuracy degradation than other SER works under noisy environments; compared to accuracy for the clean speech waveform, the accuracy degradation is 3.8% at 0 dB SNR in this work while it is larger than 15% in the other SER works. The accuracy degradation of this work at SNRs of 0 dB, −3 dB, and −6 dB is 3.8%, 5.2%, and 7.2%, respectively.

Audiovisual perception of interrupted speech by nonnative listeners.

The purpose of the present study was to examine the influence of visual cues in audiovisual perception of interrupted speech by nonnative English listeners and to identify the role of working memory, long-term memory retrieval, and vocabulary knowledge in audiovisual perception by nonnative listeners. The participants included 31 Mandarin-speaking English learners between 19 and 41 years of age. The perceptual stimuli were noise-filled periodically interrupted AzBio and QuickSIN sentences with or without visual cues that showed a male speaker uttering the sentences. In addition to sentence recognition, the listeners completed a semantic fluency task, verbal (operation span) and visuospatial (symmetry span) working memory tasks, and two vocabulary knowledge tests (Vocabulary Level Test and Lexical Test for Advanced Learners of English). The results revealed significantly better speech recognition in the audio-visual condition than the audio-only condition, but the magnitude of visual benefit was substantially attenuated for sentences that had limited semantic context. The listeners' vocabulary size in English played a key role in the restoration of missing speech information and audiovisual integration in the perception of interrupted speech. Meanwhile, the listeners' verbal working memory capacity played an important role in audiovisual integration especially for the difficult stimuli with limited semantic context.

Exploration of Speech and Music Information for Movie Genre Classification

Movie genre prediction from trailers is mostly attempted in a multi-modal manner. However, the characteristics of movie trailer audio indicate that this modality alone might be highly effective in genre prediction. Movie trailer audio predominantly consists of speech and music signals in isolation or overlapping conditions. This work hypothesizes that the genre labels of movie trailers might relate to the composition of their audio component. In this regard, speech-music confidence sequences for the trailer audio are used as a feature. In addition, two other features previously proposed for discriminating speech-music are also adopted in the current task. This work proposes a time and channel Attention Convolutional Neural Network (ACNN) classifier for the genre classification task. The convolutional layers in ACNN learn the spatial relationships in the input features. The time and channel attention layers learn to focus on crucial timesteps and CNN kernel outputs, respectively. The Moviescope dataset is used to perform the experiments, and two audio-based baseline methods are employed to benchmark this work. The proposed feature set with the ACNN classifier improves the genre classification performance over the baselines. Moreover, decent generalization performance is obtained for genre prediction of movies with different cultural influences (EmoGDB).

Use Of Language Variations On Tiktok Social Media In Generation Z

This article reviews how the use of Indonesian language in TikTok social media through several accounts of young people in generation Z. Where this tiktok media is used as a means to display his identity as a slang young person in social media with the use of language variations that are sociolect language variations. This study aims to determine the use of language used by generation Z young people in every upload of TikTok video posts, captions used and comments made by netizens. This research uses a method with a type of literature study research which is then used for data collection. Based on the results of the study, it can be concluded that the analysis of the use of language variations on TikTok social media of generation Z young people found several language variations such as acrolec variations, vulgarity, slang, colloquial, jargon, and ken in several posts, captions and in the comment column. Caused by two factors, namely internal factors and external factors. In internal factors, language variation is formed through phonological and morphological processes. The external factors behind the use of language variations are informal or informal speech situations, the age of TikTok users who are still dominated by teenagers, the goals to be achieved by speakers, and differences in education and work levels.

Speaker-based language identification for Ethio-Semitic languages using CRNN and hybrid features

ABSTRACT Natural language is frequently employed for information exchange between humans and computers in modern digital environments. Natural Language Processing (NLP) is a basic requirement for technological advancement in the field of speech recognition. For additional NLP activities like speech-to-text translation, speech-to-speech translation, speaker recognition, and speech information retrieval, language identification (LID) is a prerequisite. In this paper, we developed a Language Identification (LID) model for Ethio-Semitic languages. We used a hybrid approach (a convolutional recurrent neural network (CRNN)), in addition to a mixed (Mel frequency cepstral coefficient (MFCC) and mel-spectrogram) approach, to build our LID model. The study focused on four Ethio-Semitic languages: Amharic, Ge’ez, Guragigna, and Tigrinya. By using data augmentation for the selected languages, we were able to expand our original dataset of 8 h of audio data to 24 h and 40 min. The proposed selected features, when evaluated, achieved an average performance accuracy of 98.1%, 98.6%, and 99.9% for testing, validation, and training, respectively. The results show that the CRNN model with (Mel-Spectrogram + MFCC) combination feature achieved the best results when compared to other existing models.

Word frequency and cognitive effort in turns-at-talk: turn structure affects processing load in natural conversation.

Frequency distributions are known to widely affect psycholinguistic processes. The effects of word frequency in turns-at-talk, the nucleus of social action in conversation, have, by contrast, been largely neglected. This study probes into this gap by applying corpus-linguistic methods on the conversational component of the British National Corpus (BNC) and the Freiburg Multimodal Interaction Corpus (FreMIC). The latter includes continuous pupil size measures of participants of the recorded conversations, allowing for a systematic investigation of patterns in the contained speech and language on the one hand and their relation to concurrent processing costs they may incur in speakers and recipients on the other hand. We test a first hypothesis in this vein, analyzing whether word frequency distributions within turns-at-talk are correlated with interlocutors' processing effort during the production and reception of these turns. Turns are found to generally show a regular distribution pattern of word frequency, with highly frequent words in turn-initial positions, mid-range frequency words in turn-medial positions, and low-frequency words in turn-final positions. Speakers' pupil size is found to tend to increase during the course of a turn at talk, reaching a climax toward the turn end. Notably, the observed decrease in word frequency within turns is inversely correlated with the observed increase in pupil size in speakers, but not in recipients, with steeper decreases in word frequency going along with steeper increases in pupil size in speakers. We discuss the implications of these findings for theories of speech processing, turn structure, and information packaging. Crucially, we propose that the intensification of processing effort in speakers during a turn at talk is owed to an informational climax, which entails a progression from high-frequency, low-information words through intermediate levels to low-frequency, high-information words. At least in English conversation, interlocutors seem to make use of this pattern as one way to achieve efficiency in conversational interaction, creating a regularly recurring distribution of processing load across speaking turns, which aids smooth turn transitions, content prediction, and effective information transfer.

Advanced accent/dialect identification and accentedness assessment with multi-embedding models and automatic speech recognition.

The ability to accurately classify accents and assess accentedness in non-native speakers are challenging tasks due primarily to the complexity and diversity of accent and dialect variations. In this study, embeddings from advanced pretrained language identification (LID) and speaker identification (SID) models are leveraged to improve the accuracy of accent classification and non-native accentedness assessment. Findings demonstrate that employing pretrained LID and SID models effectively encodes accent/dialect information in speech. Furthermore, the LID and SID encoded accent information complement an end-to-end (E2E) accent identification (AID) model trained from scratch. By incorporating all three embeddings, the proposed multi-embedding AID system achieves superior accuracy in AID. Next, leveraging automatic speech recognition (ASR) and AID models is investigated to explore accentedness estimation. The ASR model is an E2E connectionist temporal classification model trained exclusively with American English (en-US) utterances. The ASR error rate and en-US output of the AID model are leveraged as objective accentedness scores. Evaluation results demonstrate a strong correlation between scores estimated by the two models. Additionally, a robust correlation between objective accentedness scores and subjective scores based on human perception is demonstrated, providing evidence for the reliability and validity of using AID-based and ASR-based systems for accentedness assessment in non-native speech. Such advanced systems would benefit accent assessment in language learning as well as speech and speaker assessment for intelligibility, quality, and speaker diarization and speech recognition advancements.

Incorporating Part of Speech Information in span representation for Named Entity Recognition

Named Entity Recognition (NER) plays a pivotal role in knowledge extraction and improving the intelligence of edge computing. The effectiveness of span-based NER models predominantly depends on the representation of spans. Existing methods primarily utilize semantic features to represent spans, often neglecting other vital information. This paper proposes a method incorporating Part of Speech (POS) information into span representations to overcome this limitation. Central to this methodology is a span POS encoder designed to extract the POS feature of spans. For migrating the method to edge devices, this paper introduces a fast span POS encoder, which significantly reduces the time complexity of POS feature extraction. Building upon this innovation, a span-based NER model named IPSI (Incorporating Part of Speech Information in span representation) is developed, exhibiting outstanding performance on nested and flat datasets. Comparison the original and fast span POS encoders reveals that while the fast encoder slightly compromises performance, it markedly accelerates the training and inference processes. Finally, through a series of experiments and sample analyses, this article explores the intrinsic mechanism through which the span POS feature influences entity recognition and further illustrates the importance of the POS feature.

Effect of frequency compression on fricative perception between normal-hearing English and Mandarin listeners.

High-frequency speech information is susceptible to inaccurate perception in even mild to moderate forms of hearing loss. Some hearing aids employ frequency-lowering methods such as nonlinear frequency compression (NFC) to help hearing-impaired individuals access high-frequency speech information in more accessible lower-frequency regions. As such techniques cause significant spectral distortion, tests such as the S-Sh Confusion Test help optimize NFC settings to provide high-frequency audibility with the least distortion. Such tests have been traditionally based on speech contrasts pertinent to English. Here, the effects of NFC processing on fricative perception between English and Mandarin listeners are assessed. Small but significant differences in fricative discrimination were observed between the groups. The study demonstrates possible need for language-specific clinical fitting procedures for NFC.

Tindak Tutur Ekspresif Netizen Pada Pemberitaan Konflik Palestina-Israel di Sosial Media Instagram

The purpose of this study was to expland the form of expressive speech and speech strategy by netizens from conflict of Israel-Palestina news on instagram. This type of research is qualitative research using descriptive methods, especially by employing Searle’s theory of speech acts. Data in this reseach were word, phrases, clauses and sentences that represent the form of expressive speech and speech strategy. Based on the results of the study, it was found five types form of expressive speech, namely condolence speech, insinuating speech, angry speech, complaining speech, and humor speech and four types of speech strategy, namely strategy of speaking frankly without preamble, strategy of speaking speaking with positive politeness, strategy of speaking with negative preamble and the strategy speak vaguely. Based on the results of the study, it can be concluded that the the form of expressive speech by netizens from Israel-Palestina news is a stimulus response from the news. The strategy of expressive speech caused by personality, pretension, and speaker purpose. The form of expressive speech determines the expressive speech strategy.

Hate Speech Detection Using Deep Learning

Abstract: Hate speech is a form of verbal abuse that targetsindividuals depending on their race, religion, ethnicity, gender, sexual orientation, or other personal characteristics. It can be spoken, written, or displayed, and can be found in a variety of contexts, including online, in person, and in the media. Hate Speech has the potential to have a devastating impact on victims, causing emotional distress, social isolation, and even physical harm. Recognizing hate speech is a challenging task, especially in multilingual contexts. This is because hate speech can be expressed in many ways, and can be difficult todistinguish from other forms of speech. However, Artificial Intelligence (AI) has advanced recently, and have made it possible to develop effective hate speech detection systems. Hate speech detection is a challenging task, especially in multilingual contexts. This survey paper reviews the recent advances in hate speech detection using BERT and CNN models. We explored the various approaches that have been proposed, the challengesfaced, and the paths that research will take in the future

Visual scanning patterns of a talking face when evaluating phonetic information in a native and non-native language.

When comprehending speech, listeners can use information encoded in visual cues from a face to enhance auditory speech comprehension. For example, prior work has shown that the mouth movements reflect articulatory features of speech segments and durational information, while pitch and speech amplitude are primarily cued by eyebrow and head movements. Little is known about how the visual perception of segmental and prosodic speech information is influenced by linguistic experience. Using eye-tracking, we studied how perceivers' visual scanning of different regions on a talking face predicts accuracy in a task targeting both segmental versus prosodic information, and also asked how this was influenced by language familiarity. Twenty-four native English perceivers heard two audio sentences in either English or Mandarin (an unfamiliar, non-native language), which sometimes differed in segmental or prosodic information (or both). Perceivers then saw a silent video of a talking face, and judged whether that video matched either the first or second audio sentence (or whether both sentences were the same). First, increased looking to the mouth predicted correct responses only for non-native language trials. Second, the start of a successful search for speech information in the mouth area was significantly delayed in non-native versus native trials, but just when there were only prosodic differences in the auditory sentences, and not when there were segmental differences. Third, (in correct trials) the saccade amplitude in native language trials was significantly greater than in non-native trials, indicating more intensely focused fixations in the latter. Taken together, these results suggest that mouth-looking was generally more evident when processing a non-native versus native language in all analyses, but fascinatingly, when measuring perceivers' latency to fixate the mouth, this language effect was largest in trials where only prosodic information was useful for the task.

EXPERIMENTAL DIAGNOSTICS OF COMMUNICATION AND SPEECH SKILLS OF PRESCHOOL CHILDREN WITH DOWN SYNDROME

The article scientifically substantiates and characterizes the experimental method of psychological-pedagogical study of communication and speech skills in preschool children with Down syndrome. The criteria and indicators of the formation of communication and speech skills in the children of the research group were determined. It was determined that normative indicators of communicative and speech development are the standard for comparing information obtained in the process of diagnostic research. It was established that in the modern scientific publications of Ukraine there are no generalized methods of diagnosing communication and speech abilities in children with Down syndrome. The existing methods of diagnosis are singled out, which are used in an adapted and modified form to study the state of formation of individual components of the speech system and communication skills. The diagnostic methodology includes successive stages of research: analysis of the content of medical and psychological-pedagogical documentation; questionnaires and surveys of parents and correctional teachers; indirect (hidden) observation of independent behavior of children with this genetic anomaly; testing in artificially created conditions. The structure of a complex diagnostic study of the state of formation of communication and speech skills in children with this genetic anomaly is characterized. The elements of each component of communicative and speech development are singled out, according to which the psychological and pedagogical study is carried out. Special diagnostic tasks are proposed for studying the state of formation of the cognitive-linguistic component of communicative-speech development. A general level base is proposed, which subordinates the level of formation of communication and speech skills in preschool children with Down syndrome (low, medium, sufficient, high). It was established that each level depends on the presence of different forms of speech, understanding of spoken speech, the volume of passive and active vocabulary, the ability to establish interpersonal contacts and maintain them, the use of existing communication skills to regulate relations with other people, the ability to distinguish the emotional states of the interlocutor.

Intellectual functionality of speech in children of older preschool age with logopathology

Background. The purpose of this study is to identify the current state of intellectual functioning, which affects the speech readiness to study at school, of older preschool children with logopathology. The study included 607 older preschool children (5–6 years old), 250 with normotypical psychophysical development and 357 with speech disorders (dyslalia, stuttering, rhinolalia, dysarthria). Materials and methods. To evaluate the intellectual functioning of older preschool children with logopathology, methods were developed aimed at studying visual-motor coordination, auditory-verbal memory, verbal-spatial representation, verbal-logical thinking, verbal attention. Results. The results of the experimental study give a clear idea that there are significant differences in the formation of intellectual functionality between children with logopathology and those with normotypical psychophysical development. Older preschool children with speech disorders had low ability to copy graphic images, violations of independent control and planning their own activities, problems with the consistent reproduction of words, the volume of memorization, the strength of retention of stimuli, the speed and accuracy of reproduction of speech information, the difficulties understanding spatial relations in the addressed and own speech; verbal and logical skills are formed heterogeneously. Some preschoolers with logopatho­logy can classify objects by features, generalize them, understand the figurative meaning of words and the content of texts. Others experience certain difficulties and make many mistakes; their verbal attention is not sufficiently developed, which prevents children from perceiving educational material. Conclusions. Accordingly, low indicators of visual-motor coordination, auditory-verbal memory, verbal-spatial representation, verbal-logical thinking, verbal attention in older preschool children with speech disorders indicate insufficiently formed intellectual functionality. This will affect their speaking readiness and learning the curriculum at school.

3D facial animation driven by speech-video dual-modal signals

In recent years, the applications of digital humans have become increasingly widespread. One of the most challenging core technologies is the generation of highly realistic and automated 3D facial animation that combines facial movements and speech. The single-modal 3D facial animation driven by speech typically ignores the weak correlation between speech and upper facial movements as well as head posture. In contrast, the video-driven approach can perfectly solve the posture problem while obtaining natural expressions. However, mapping 2D facial information to 3D facial information may lead to information loss, which make lip synchronization generated by video-driven methods is not as good as the speech-driven methods trained on 4D facial data. Therefore, this paper proposes a dual-modal generation method that uses speech and video information to generate more natural and vivid 3D facial animation. Specifically, the lip movements related to speech are generated by speech-video information, while speech-uncorrelated postures and expressions are generated solely by video information. The speech-driven module is used to extract speech features, and its output lip animation is then used as the foundation for facial animation. The expression and pose module is used to extract temporal visual features for regressing expression and head posture parameters. We fuse speech and video features to obtain chin posture parameters related to lip movements, and use these parameters to fine-tune the lip animation generated form the speech-driven module. This paper introduces multiple consistency losses to enhance the network’s capability to generate expressions and postures. Experiments conducted on the LRS3, TCD-TIMIT and MEAD datasets show that the proposed method achieves better performance on evaluation metrics such as CER, WER, VER and VWER than the current state-of-the-art methods. In addition, a perceptual user study show that over 77% and 70% of cases believe that this paper’s method is better than the comparative algorithms EMOCA and SPECTRE in terms of realism. In terms of lip synchronization, it received over 79% and 66% of cases support, respectively. Both evaluation methods demonstrate the effectiveness of the proposed method.