Individual Utterances Research Articles

Playing single-player tasks together: Dyads’ collaborative activities across two games in Math Garden

Much research on collaborative learning focuses on working together on tasks that are designed with collaboration in mind. However, teachers may ask children to collaborate on single player tasks as well, given a scarcity of available computers in the classroom or assuming that children can learn from each other and motivate each other. In the current study, we studied the conversations of primary school children (N = 64; 39 girls, 25 boys) working together in dyads in an adaptive single-player math learning environment (Math Garden). We aimed to investigate (1) how dyads collaborate across two math tasks aimed at practicing different math skills (arithmetic vs. reasoning skills) embedded within Math Garden and (2) whether collaborative activities differed across the two math tasks. We studied both individual utterances as well as episodes of social metacognitive interactions between the dyad members. We found that children engaged in activities important for successful collaboration, such as relational and metacognitive activities. Moreover, we identified social metacognitive episodes, meaning that children regulated their groups’ learning and that the regulatory processes which took part within the collaborative learning session cannot be reduced to individual activities. We conclude that children engage in collaborative activities when working together on single player tasks. And, in line with previous research, these collaborative activities differed across games, highlighting that collaborative learning is shaped by the task at hand.

A contextual-based approach for sarcasm detection

Sarcasm is a perplexing form of human expression that presents distinct challenges in understanding. The problem of sarcasm detection has centered around analyzing individual utterances in isolation which may not provide a comprehensive understanding of the speaker’s sarcastic intent. Our work addresses this problem by exploring and understanding the specific contextual cues that contribute to sarcasm. In this paper, we propose an enhanced approach for sarcasm detection using contextual features. Our methodology involves employing pre-trained transformer models, RoBERTa and DistilBERT, and fine-tuning them on two datasets: the News Headlines and the Mustard datasets. Incorporating contextual information, the proposed approach yielded the best performance, achieving an impressive F1 score of 99% on News Headlines dataset and 90% on Mustard dataset. Moreover, we experimented summarizing the context into a concise short sentence. This enhancement reduced training time by 35.5% of the original time. We further validated the model trained on the News headlines dataset against the Reddit dataset, which resulted in 49% F1 score without context data. However, with the inclusion of context data, the F1 score surged to 75%. Proposed approach enhances the understanding of sarcasm in different contextual settings, enabling more accurate sentiment analysis and better decision-making in various applications.

FCAN : Speech emotion recognition network based on focused contrastive learning

Traditional speech emotion recognition (SER) models predominantly utilize cross-entropy loss for supervised optimization. Yet, given the limited size of SER datasets, relying solely on cross-entropy loss does not adequately capture the intricacies within the data. Moreover, conventional contrastive learning techniques, though improving performance, do not completely harness the inherent data structure across samples. To surmount these limitations, we introduce an innovative emotion recognition network founded on focused contrastive learning. Our model employs bidirectional gated recurrent unit (GRU) networks to seize critical contextual dependencies. By integrating an attention mechanism, we adeptly merge the contextual features of dialogues with individual utterance attributes, forming a comprehensive multi-stage feature fusion framework. This strategy extracts more elaborate emotional features from the constrained dataset. Additionally, by amalgamating the focal loss function into our training protocol, we mitigate imbalanced sample issues to a degree. We further refine our contrastive loss function by enacting a weighted strategy for intra-dialogue category assignments, where higher penalty weights are ascribed to harder-to-identify categories, thus incentivizing the model to concentrate on distinguishing subtler category differences. This refinement significantly bolsters the model’s proficiency in learning various emotional feature distributions, thereby enhancing its emotion recognition capabilities. Experimental validation on IEMOCAP and MELD benchmark datasets underpins our model’s superiority.

Towards a dynamic functional proposition for dynamic discourse meaning

Abstract Theories of utterance meaning in the post-Gricean tradition have typically focused on the main proposition expressed by the speaker that is recovered by the addressee. In this tradition, successful communication rests on the assumption that speakers and addressees come to a shared understanding of these propositions as they are produced in conversation. We now have a wealth of empirical evidence that speakers and addressees need not always converge on the main proposition expressed in order for communication to proceed unhindered: they may share partial understandings of individual utterances, allowing the overarching discourse meaning to unravel as the interaction progresses. In this paper, we propose a novel unit of meaning that accounts for such a dynamic concept that can emerge and develop over several turns at talk. We call it a ‘dynamic functional proposition’. This unit includes not only the linguistic meaning that has been communicated, but also meaning conveyed through non-linguistic sources, as well as aspects of situation captured through what we call ‘filters’, such as interlocutors’ levels of attention, emotions, and other non-representational aspects. These various aspects will have greater or lesser salience for different speakers, hence offering an explanatory tool for how utterance meanings are negotiated, as well as when and why misunderstandings occur. We finish by proposing ways in which such a unit can be formally represented. We do this by motivating different cognitive, social and linguistic parameters that influence it.

Topic modeling for conversations for mental health helplines with utterance embedding

Conversations with topics that are locally contextual often produces incoherent topic modeling results using standard methods. Splitting a conversation into its individual utterances makes it possible to avoid this problem. However, with increased data sparsity, different methods need to be considered. Baseline bag-of-word topic modeling methods for regular and short-text, as well as topic modeling methods using transformer-based sentence embeddings were implemented. These models were evaluated on topic coherence and word embedding similarity. Each method was trained using single utterances, segments of the conversation, and on the full conversation. The results showed that utterance-level and segment-level data combined with sentence embedding methods performs better compared to other non-sentence embedding methods or conversation-level data. Among the sentence embedding methods, clustering using HDBScan showed the best performance. We suspect that ignoring noisy utterances is the reason for better topic coherence and a relatively large improvement in topic word similarity.

Barriers to and Facilitators of Delabelling of Antimicrobial Allergies: A Qualitative Meta-synthesis.

Patients who report penicillin allergies may receive alternative antibiotics. Such substitution contributes to antimicrobial resistance, lower treatment efficacy, increased frequency of adverse events, and increased costs. Approximately 90% of individuals who report a penicillin allergy can tolerate a penicillin. To identify the barriers to and facilitators of removal by health care workers of inaccurate antimicrobial allergies from patient records, known as delabelling. The MEDLINE database was searched from inception to December 29, 2020. Qualitative studies evaluating health care professionals' perceptions of barriers to and/or facilitators of the act of delabelling a patient's antimicrobial allergies were included in the meta-synthesis. The Theoretical Domains Framework was used to code and group individual utterances from the included studies, which were mapped to the Behaviour Change Wheel and corresponding intervention function and policy categories. Four studies met the inclusion criteria. Eight themes were identified as representing barriers to delabelling: delabelling skills, patient education skills, knowledge, electronic health records (EHRs), communication frameworks, time, fear about allergic reactions, and professional roles. Behaviour change interventions that may overcome these barriers include education, training, algorithms and toolkits, changes to EHRs, use of dedicated personnel, policies, incentivization of correct labelling, and an audit system. Eight themes were identified as barriers to delabelling of antimicrobial allergies. Future behaviour change interventions to address these barriers were proposed. Confidence in the findings of this study was judged to be moderate, according to the GRADE CERQual approach.

PIRNet: Personality-Enhanced Iterative Refinement Network for Emotion Recognition in Conversation.

Emotion recognition in conversation (ERC) is important for enhancing user experience in human-computer interaction. Unlike vanilla emotion recognition in individual utterances, ERC aims to classify constituent utterances in a dialog into corresponding emotion labels, which makes contextual information crucial. In addition to contextual information, personality traits also affect emotional perception based on psychological findings. Although researchers have proposed several approaches and achieved promising results on ERC, current works in this domain rarely incorporate contextual information and personality influence. To this end, we propose a novel framework to integrate these factors seamlessly, called "Personality-enhanced Iterative Refinement Network (PIRNet)." Specifically, PIRNet is a multistage iterative method. To capture personality influence, PIRNet leverages personality traits to mimic emotional transitions and generates personality-enhanced results. Then we exploit sequence models to capture contextual information in conversations. To verify the effectiveness of our proposed method, we conduct experiments on three benchmark datasets for ERC, that is, IEMOCAP, CMU-MOSI, and CMU-MOSEI. Experimental results demonstrate that our PIRNet succeeds over currently advanced approaches to emotion recognition.

20 Global and Local Semantic Coherence of Spontaneous Speech in Persons with Alzheimer's Disease and Healthy Controls

Objective:Growing evidence demonstrates that subtle changes in spontaneous speech can be used to distinguish older adults with and without cognitive impairment, including those with Alzheimer's disease (AD). Recent work suggests that quantification of the meaningful connectedness of speech - termed semantic coherence - may be sensitive to cognitive dysfunction. The current study compared global coherence (GC; the degree to which individual utterances relate to the overall topic being discussed) and local coherence (LC; the degree to which adjoining utterances relate meaningfully to each other) in persons with AD and healthy controls.Participants and Methods:Speech transcripts from 81 individuals with probable AD (Mage = 72.7 years, SD = 8.8, 70.3% female) and 61 healthy controls (HC) (Mage = 63.9 years, SD = 8.5, 62.2% female) from Dementia Bank were analyzed. All participants completed the Cookie Theft and MMSE as part of that larger project. Machine learning analyses of GC and LC were conducted and models evaluated classification accuracy (i.e., AD vs HC) as well as ROC-AUC. Relationships between coherence indices and MMSE performance were also quantified.Results:Though no significant group differences emerged in LC (Estimate = 0.012, p = 0.32), persons with AD differed from healthy controls in GC (Estimate = 0.03, p = 0.006) and produced less semantically coherent speech. GC indices predicted AD diagnoses with 65% accuracy. Interestingly, coherence indices showed only modest correlation with MMSE scores (r = .19).Conclusions:GC metrics of spontaneous speech differentiated between persons with AD and controls, but did not strongly correlate with MMSE performance. Such findings support the notion that many aspects of language are impacted in persons with AD. In addition to replication, future work should evaluate whether GC is also disrupted in persons with pre-clinical AD and its potential to assist with early detection.

초등영어 학습중심평가를 위한 AI 챗봇의 스캐폴딩 유형화 연구

The purpose of this study is to explore the types of scaffolding that AI chatbots can
 provide through systematic feedback based on learners' individual utterances during
 elementary English learning-oriented assessment. For the study, the researcher designed
 an English speaking task and implemented an learning-oriented speaking assessment
 using an AI chatbot for 25 learners in the 6th grade. Subsequently, individual interviews
 were conducted to analyze the learners' responses to the scaffolding. By analyzing
 learner responses and transcripts of conversations between learners and chatbots, the
 researcher categorized the types of scaffolding that AI chatbots can provide during
 the learning-oriented assessment. As a result, first, the scaffolding can be categorized
 into three areas: linguistic, cognitive, and affective. Linguistically, the chatbots
 provided scaffolding for conveying meaning, language forms, and pronunciation
 aspects. In the cognitive aspect, the scaffolding assisted learners in their thinking
 processes, while in the affective aspect, it helped regulate frustration. Second, through
 scaffolding using various strategies, learners demonstrated the ability to self-correct
 erroneous utterances and gradually complete sentences on their own. In addition,
 cognitive scaffolding helped learners solve challenging tasks at their current cognitive
 level, and affective scaffolding contributed to improving confidence in speaking
 English. The study offers implications and recommendations based on the findings.

BASIC CONCEPTS OF THE THEORY OF REAL MONOIDIOM DESCRIPTION

The article deals with the description of the linguistics of speech and its theoretical justification can be based on the main conceptual principles of the metalanguage of monosyllabic description, which includes the main concepts of the theory of real description. Metalanguage is used in applied linguistics to describe other natural languages, and, in the author’s opinion, individual utterances as well. In idiomology as a science of speech, linguistic data constitute a certain monosyllabic linguarie, which distinguishes the metalanguage of its description from dialectological or other types of linguistic terminology, thereby positioning the specificity of describing speech as a real language system in the context of highly organized conceptual approaches to the language system. The problems of metalanguage monosyllabic description are outlined, which are in the lens of modern theoretical and methodological linguistic paradigms, because metalanguage is also considered the language of linguistics, which is characterized by a complete set and classification of linguistic knowledge. At the same time, the concept of metalanguage is expanded in dialectology and specified in the context of linguistics, focusing on the monosyllabic description of speech. The novelty of the speech description lies in the fact that it is based on a monosyllabic linguarie, which is a complex process of scientific and cognitive activity in the modern linguistic paradigm. The metalanguage of monosyllabic description needs to update the system of basic principles and special methods of linguistic research, the methodology of speech description according to a clearly planned organization and research apparatus. The emphasis on the importance of the analysis of colloquialism lies in the composition of the metalanguage of the monosyllabic description, which contains central and universal categories that can be used in the study of various language systems: general linguistic terms and concepts (speaker, colloquial speaker), as well as supplemented and clarified methodological concepts of the monosyllabic linguistic description: spoken community, spoken com- munication, spoken language. A complete presentation of the metalanguage of the monolingual description of the linguarium can be seen in the glossary of the metalanguage of the description for the speech language system.

Decontextualized Utterances Contain More Typical and Stuttering-Like Disfluencies in Preschoolers Who Do and Do Not Stutter.

Stuttering-like disfluencies (SLDs) and typical disfluencies (TDs) are both more likely to occur as utterance length increases. However, longer and shorter utterances differ by more than the number of morphemes: They may also serve different communicative functions or describe different ideas. Decontextualized language, or language that describes events and concepts outside of the "here and now," is associated with longer utterances. Prior work has shown that language samples taken in decontextualized contexts contain more disfluencies, but averaging across an entire language sample creates a confound between utterance length and decontextualization as contributors to stuttering. We coded individual utterances from naturalistic play samples to test the hypothesis that decontextualized language leads to increased disfluencies above and beyond the effects of utterance length. We used archival transcripts of language samples from 15 preschool children who stutter (CWS) and 15 age- and sex-matched children who do not stutter (CWNS). Utterances were coded as either contextualized or decontextualized, and we used mixed-effects logistic regression to investigate the impact of utterance length and decontextualization on SLDs and TDs. CWS were more likely to stutter when producing decontextualized utterances, even when controlling for utterance length. An interaction between decontextualization and utterance length indicated that the effect of decontextualization was greatest for shorter utterances. TDs increased in decontextualized utterances when controlling for utterance length for both CWS and CWNS. The effect of decontextualization on TDs did not differ statistically between the two groups. The increased working memory demands associated with decontextualized language contribute to increased language planning effort. This leads to increased TD in CWS and CWNS. Under a multifactorial dynamic model of stuttering, the increased language demands may also contribute to increased stuttering in CWS due to instabilities in their speech motor systems.

Delicious angst and tooth-rotting fluff: Distant reading community discourses of emotion in Harry Potter fanfiction comments

This article employs distant reading to examine discursive norms for expressing emotion in comments on Harry Potter fanfiction on Archive of Our Own (AO3). Using text analysis tools to identify collocational patterns, the article shows how fanfiction engages this community and how community members express their engagement in accordance with community discourses. The analysis transcends the level of individual utterances by examining large-scale patterns in the data. Comparing comments for three fanfiction genres (angst, fluff and hurt/comfort) reveals three patterns. First, commenters frequently describe stories about suffering in positive terms. Second, metaphors likening fanfiction consumption to food consumption are prevalent in all genres. Commenters also use language blending physical and emotional sensations. Finally, commenters describe stories about positive emotions using negative metaphors of rot, decay, gluttony or overload. This use of positive or negative language need not indicate positive or negative judgements but instead reveals discursive fields to which fanfiction reading is linked. Additionally, comments indicate that fanfiction is used to influence emotional states. These findings contribute to an understanding of the impact of fanfiction within its community and raise questions regarding the role of emotion in fanfiction reading.

Simulating vocal learning of spoken language: Beyond imitation

Computational approaches have an important role to play in understanding the complex process of speech acquisition, in general, and have recently been popular in studies of vocal learning in particular. In this article we suggest that two significant problems associated with imitative vocal learning of spoken language, the speaker normalisation and phonological correspondence problems, can be addressed by linguistically grounded auditory perception. In particular, we show how the articulation of consonant–vowel syllables may be learnt from auditory percepts that can represent either individual utterances by speakers with different vocal tract characteristics or ideal phonetic realisations. The result is an optimisation-based implementation of vocal exploration – incorporating semantic, auditory, and articulatory signals – that can serve as a basis for simulating vocal learning beyond imitation.

GCNet: Graph Completion Network for Incomplete Multimodal Learning in Conversation.

Conversations have become a critical data format on social media platforms. Understanding conversation from emotion, content and other aspects also attracts increasing attention from researchers due to its widespread application in human-computer interaction. In real-world environments, we often encounter the problem of incomplete modalities, which has become a core issue of conversation understanding. To address this problem, researchers propose various methods. However, existing approaches are mainly designed for individual utterances rather than conversational data, which cannot fully exploit temporal and speaker information in conversations. To this end, we propose a novel framework for incomplete multimodal learning in conversations, called "Graph Complete Network (GCNet)," filling the gap of existing works. Our GCNet contains two well-designed graph neural network-based modules, "Speaker GNN" and "Temporal GNN," to capture temporal and speaker dependencies. To make full use of complete and incomplete data, we jointly optimize classification and reconstruction tasks in an end-to-end manner. To verify the effectiveness of our method, we conduct experiments on three benchmark conversational datasets. Experimental results demonstrate that our GCNet is superior to existing state-of-the-art approaches in incomplete multimodal learning.

How redundant is language really? Agent-recipient disambiguation across time and space

Redundant marking of grammatical relations seems to be commonplace across languages, and has been shown to benefit learning as well as robust information transmission. At the same time, languages also exhibit trade-offs between strategies such as case marking or word order, suggesting that redundancy may also be dis-preferred in line with a tendency towards communicative efficiency. In the present paper, we assess redundancy  in terms of number of strategies used simultaneously to mark specific relations within individual utterances (syntagmatic redundancy) in light of these competing motivations. Our test case is participant role disambiguation in English and Dutch, specifically the interaction of constituent order, case, prepositional marking, and agreement to distinguish agents and recipients in ditransitive clauses. Using evidence from corpora of Present Day Dutch and English as well as data from Middle English, we find that redundancy is prevalent, albeit within certain limits.   

The effect of turn-taking on the kinematic properties of prosodic boundaries

Gestures at prosodic boundaries are larger, longer, and further apart than gestures phrase-medially, and this effect is stronger for hierarchically higher boundaries. However, most studies examining kinematic properties of boundaries limit their investigation to read speech of individual utterances, so the kinematic properties of turn-taking are not known. In this study, six pairs of participants competed in a game eliciting semi-controlled interactive speech with target words at phrase-medial, phrase-final but utterance-medial, and turn-final prosodic boundaries. One member of each participant dyad was recorded in an articulatory magnetometer, and the other acted as a conversation partner. We test two competing hypotheses about the behavior of gestures at turn-final boundaries: Hypothesis 1 states that turn-final gestures show more lengthening than turn-medial gestures. This hypothesis comes from Conversation Analysis research suggesting final lengthening is used to indicate the end of a turn, and the fact that hierarchically higher boundaries generally show more lengthening. Conversely, hypothesis 2 states that turn-final gestures show less lengthening than turn-medial gestures. This alternative hypothesis is based on the idea that, since there is no imminent speech to be planned following the end of a turn, no additional lengthening is required in order to accommodate planning time.

BERT-based response selection in dialogue systems using utterance attention mechanisms

Dialogue systems, one of the core research fields of natural language processing, attempt to understand the utterances of a user and generate an appropriate response. Response selection in a retrieval-based dialogue system involves searching for the most context-appropriate subsequent utterance. Conversations are usually composed of multiple turns; therefore, the intention of the speaker must be properly understood prior to the response selection. To accurately capture such intended meaning, we propose a retrieval-based response selection model that effectively comprehends the relationships among words and utterances in a conversation and a response candidate with word and utterance attention. Word representation is generated by using the self-attention mechanism to reflect the contextual information between intentional words in an overall conversation or individual utterance, while utterance representation is by the cross-attention mechanism to reflect the contextual information among utterances. Furthermore, since our model does not need much computation and memory size, it can be easily combined with existing other response selection models or pre-trained language models. Experiments on various utterance embedding methods were also conducted to find a proper representation of the utterance information. Our proposed model exhibits an improvement in performance of approximately 2.1%p in hit@1 in the DSTC8 ubuntu dataset compared to baseline models, as well as significant performance improvements for other datasets.

Misinformation in Italian Online Mental Health Communities During the COVID-19 Pandemic: Protocol for a Content Analysis Study.

BackgroundSocial media platforms are widely used by people suffering from mental illnesses to cope with their conditions. One modality of coping with these conditions is navigating online communities where people can receive emotional support and informational advice. Benefits have been documented in terms of impact on health outcomes. However, the pitfalls are still unknown, as not all content is necessarily helpful or correct. Furthermore, the advent of the COVID-19 pandemic and related problems, such as worsening mental health symptoms, the dissemination of conspiracy narratives, and medical distrust, may have impacted these online communities. The situation in Italy is of particular interest, being the first Western country to experience a nationwide lockdown. Particularly during this challenging time, the beneficial role of community moderators with professional mental health expertise needs to be investigated in terms of uncovering misleading information and regulating communities.ObjectiveThe aim of the proposed study is to investigate the potentially harmful content found in online communities for mental health symptoms in the Italian language. Besides descriptive information about the content that posts and comments address, this study aims to analyze the content from two viewpoints. The first one compares expert-led and peer-led communities, focusing on differences in misinformation. The second one unravels the impact of the COVID-19 pandemic, not by merely investigating differences in topics but also by investigating the needs expressed by community members.MethodsA codebook for the content analysis of Facebook communities has been developed, and a content analysis will be conducted on bundles of posts. Among 14 Facebook groups that were interested in participating in this study, two groups were selected for analysis: one was being moderated by a health professional (n=12,058 members) and one was led by peers (n=5598 members). Utterances from 3 consecutive calendar years will be studied by comparing the months from before the pandemic, the months during the height of the pandemic, and the months during the postpandemic phase (2019-2021). This method permits the identification of different types of misinformation and the context in which they emerge. Ethical approval was obtained by the Università della Svizzera italiana ethics committee.ResultsThe usability of the codebook was demonstrated with a pretest. Subsequently, 144 threads (1534 utterances) were coded by the two coders. Intercoder reliability was calculated on 293 units (19.10% of the total sample; Krippendorff α=.94, range .72-1). Aside from a few analyses comparing bundles, individual utterances will constitute the unit of analysis in most cases.ConclusionsThis content analysis will identify deleterious content found in online mental health support groups, the potential role of moderators in uncovering misleading information, and the impact of COVID-19 on the content.International Registered Report Identifier (IRRID)PRR1-10.2196/35347

Discovering emotion and reasoning its flip in multi-party conversations using masked memory network and transformer

Efficient discovery of a speaker’s emotional states in a multi-party conversation is significant to design human-like conversational agents. During a conversation, the cognitive state of a speaker often alters due to certain past utterances, which may lead to a flip in their emotional state. Therefore, discovering the reasons (triggers) behind the speaker’s emotion-flip during a conversation is essential to explain the emotion labels of individual utterances. In this paper, along with addressing the task of emotion recognition in conversations (ERC), we introduce a novel task – Emotion-Flip Reasoning (EFR), that aims to identify past utterances which have triggered one’s emotional state to flip at a certain time. We propose a masked memory network to address the former and a Transformer-based network for the latter task. To this end, we consider MELD, a benchmark emotion recognition dataset in multi-party conversations for the task of ERC, and augment it with new ground-truth labels for EFR. An extensive comparison with five state-of-the-art models suggests improved performances of our models for both the tasks. We further present anecdotal evidence and both qualitative and quantitative error analyses to support the superiority of our models compared to the baselines.

Multimedia

Multimedia