Utterance Generation Research Articles

Analysing Utterances in LLM-Based User Simulation for Conversational Search

Clarifying underlying user information needs by asking clarifying questions is an important feature of modern conversational search systems. However, evaluation of such systems through answering prompted clarifying questions requires significant human effort, which can be time-consuming and expensive. In our recent work, we proposed an approach to tackle these issues with a user simulator, USi . Given a description of an information need, USi is capable of automatically answering clarifying questions about the topic throughout the search session. However, while the answers generated by USi are both in line with the underlying information need and in natural language, a deeper understanding of such utterances is lacking. Thus, in this work, we explore utterance formulation of large language model (LLM)–based user simulators. To this end, we first analyze the differences between USi , based on GPT-2, and the next generation of generative LLMs, such as GPT-3. Then, to gain a deeper understanding of LLM-based utterance generation, we compare the generated answers to the recently proposed set of patterns of human-based query reformulations. Finally, we discuss potential applications as well as limitations of LLM-based user simulators and outline promising directions for future work on the topic.

Structure-Aware Multimodal Sequential Learning for Visual Dialog

With the ability to collect vast amounts of image and natural language data from the web, there has been a remarkable advancement in Large-scale Language Models (LLMs). This progress has led to the emergence of chatbots and dialogue systems capable of fluent conversations with humans. As the variety of devices enabling interactions between humans and agents expands, and the performance of text-based dialogue systems improves, there has been recently proposed research on visual dialog. However, visual dialog requires understanding sequences of pairs consisting of images and sentences, making it challenging to gather sufficient data for training large-scale models from the web. In this paper, we propose a new multimodal learning method leveraging existing large-scale models designed for each modality, to enable model training for visual dialog with small visual dialog datasets. The key ideas of our approach are: 1) storing the history or context during the progression of visual dialog in the form of spatiotemporal graphs, and 2) introducing small modulation blocks between modality-specific models and the graphs to align the semantic spaces. For implementation, we introduce a novel structure-aware cross-attention method, which retrieves relevant image and text knowledge for utterance generation from the pretrained models. For experiments, we achieved a new state-of-the-art performance on three visual dialog datasets, including the most challenging one COMET.

Responsive Utterance Generation to Show Disagreement to Narrative in Attentive Listening

Responsive Utterance Generation to Show Disagreement to Narrative in Attentive Listening

Development of Interdisciplinary Views on the Energetics of Human Communicative Behavior

This paper presents the results of a retrospective analysis of the scientific views on the energetics of human communicative and speech behavior. The etymology of the lexeme ‘energy’ is looked into; the definitions of the notion of ‘energy’ available in 17 lexicographic sources are analyzed, and semantic units incorporated into its lexical meaning are extracted. The analysis of their content makes it possible to define four essential features of the general scientific notion of ‘energy’ and synthesize its definition. As the energetics of human communicative behavior is traditionally seen as divided into the psychic and the physical components, its adequate description is possible only with the instruments of the integrative approach. The issues of mutual conversion of several types of energy to cater for various individual’s behavioral programs, as well as physical and physiological aspects of speech energetics have already been researched to a certain extent. Less attention has been paid to the psychic energy of a person, which nonetheless is intuitively considered to be a significant motive of any form of human activity. A concise account of historical views on the human psychic energy, nominated “prana”, “chi”, “soul”, “spirit”, “entelechy”, “energeia”, “psychic pneuma”, “animal spirits”, etc. is provided in the paper. Milestones in the research of the neurophysiological aspect of the energy supply of human communicative and speech behavior are reviewed. Special attention is paid to Freud’s description of the human psychic energy “economy” as the first full-fledged theory, capable of forming the basis for modeling the energetics of human communicative and speech behavior. Mention is made of the modern concept of “allostasis” oriented towards an anticipatory forecasting of energy needs by the organism rather than its posterior restoration upon some significant psychic events. Due to a lack of theoretical and experimental linguistic research, taking into consideration energy dynamics of the communicants’ psychic processes, a universal stochastic model of utterance generation, developed by A. Kalyta and O. Klymeniuk appears specifically promising. It could be effectively used in the development of the methodology of an experimental-phonetic study of conflict talks, typically marked with a significant level of emotional tension.

Out-of-scope intent detection with intent-invariant data augmentation

In practical dialogue systems, it is crucial to avoid undesired responses and poor user experiences by detecting Out-Of-Scope (OOS) intents from user utterances. Currently, to detect OOS intents, limited-supervised methods are more potential due to using limited true OOS data to better model the OOS data distribution. However, existing limited-supervised methods often yield low-quality OOS augmentations, i.e., noisy OOS and long-distant OOS, due to the use of the retrieval-based data augmentation mechanism, which will damage the OOS intent detection performance. To tackle this problem, in this paper, we propose a novel OOS intent detection method by leveraging intent-invariant data augmentation, called InInOOS, which can generate high-quality pseudo-OOS utterances with invariant OOS intents but various slot values and different expressions to further enhance the OOS intent detection performance. Specifically, we first generate pseudo-OOS candidates using our pre-trained intent-invariant utterance generation model, and then the most beneficial candidates are selected to train an OOS intent detection model. Extensive experiments on two public datasets show that our method performs better than state-of-the-art baselines.

Development of Grammatical Competence Based on the Binary Oppositions of Verbs

The presented article is devoted to the development of grammatical competence of Linguistic University students, which ensures, in accordance with the modern communicative-cognitive paradigm of anthropocentric education, the adequacy of understanding and generation of utterances during intercultural communication. In the article, from the point of view of an intercultural approach, the content of intercultural communicative competence is concretized as the goal of teaching a modern linguist; from the standpoint of lingua-didactics and cognitive linguistics, the features and conceptual content of grammatical competence, which is a means of intercultural communication, have been identified; the skills included in the grammatical competence are defined and classified; a methodological model in the development of grammatical competence has been developed and scientifically substantiated, involving the analysis of grammatical phenomena that, in combination with other linguistic means, form binary verbal oppositions of three contacting linguistic cultures.
 
 Received: 17 May 2023 / Accepted: 28 August 2023 / Published: 5 September 2023

Использование аппарата ситуационной семантики для моделирования процесса порождения речи

The paper analyzes and systematizes certain elements of the conceptual and terminological apparatus of situational semantics, which is required for modeling the process of speech generation. Involving the works of Russian and foreign scholars, the authors present a part of the conceptual and terminological apparatus of situational semantics relevant to generative theory: the paper systematically defines and interprets the elements of the conceptual and terminological apparatus, demonstrates its use to describe the situation in the process of utterance generation and speech generation as a whole. The aim of the study is to show the expediency of applying the apparatus of situational semantics for modeling the process of utterance generation. The novelty of the study lies in the fact that on the basis of using the apparatus of situational semantics, a number of sociocultural (discursive) factors lacking in the models of Chomskyan generative grammar were included into the generative model of utterances. As a result, the terminological apparatus needed for achieving the above-mentioned aim was formed from disparate elements and systematized and its application in the course of analyzing utterance generation was demonstrated.

A Study of Discourse Markers Employed in College English Writing by Non-English Majors

Discourse marker is a common language phenomenon in verbal communication including parts of conjunctions, adverbs, interjections, some phrases and small words. From the perspective of grammar, these modifiers do not have many a semantic meanings and they are less related to the sentences. However, from the perspective of pragmatics and discourse, discourse marker plays the role of organizing the discourse, attracting the hearer’s attention, and expressing the emotion and attitude of the speaker. In the process of utterance generation and understanding, discourse marker works as a signal or makes the information clear so that the hearer can obtain the communicative intention of the speaker easily, thus producing the contextual effect. Based on the framework of Relevance Theory by Sperber and Wilson, this essay adopts Fraser’s classification of discourse markers, analyzing the use of discourse markers by non-English majors in Guangxi region. The result of this research shows that discourse markers indeed play a crucial role in the process of utterance generation and understanding, and the correct use of discourse markers contributes to the cohesion of discourse or text and the improvement of the student’s writing ability.

SAPBERT: Speaker-Aware Pretrained BERT for Emotion Recognition in Conversation

Emotion recognition in conversation (ERC) is receiving more and more attention, as interactions between humans and machines increase in a variety of services such as chat-bot and virtual assistants. As emotional expressions within a conversation can heavily depend on the contextual information of the participating speakers, it is important to capture self-dependency and inter-speaker dynamics. In this study, we propose a new pre-trained model, SAPBERT, that learns to identify speakers in a conversation to capture the speaker-dependent contexts and address the ERC task. SAPBERT is pre-trained with three training objectives including Speaker Classification (SC), Masked Utterance Regression (MUR), and Last Utterance Generation (LUG). We investigate whether our pre-trained speaker-aware model can be leveraged for capturing speaker-dependent contexts for ERC tasks. Experiments show that our proposed approach outperforms baseline models through demonstrating the effectiveness and validity of our method.

Дуализм лексической системы специального английского языка радиообмена в авиации

The research aims to analyze the features peculiar to the lexical system of special English for radiotelephony communication in aviation. The paper is novel in that it is the first to analyze the impact of the phenomena of language redundancy and language means economy on the formation and functioning of the lexical system of the radiotelephony communication language in aviation. The research findings prove that the balanced interaction of these two mechanisms determines and regulates the use of the specific lexical means that facilitate generation and perception of utterances even under difficult communication conditions in speech.

Особенности развития семантической стороны речи младших школьников с умственной отсталостью

The article discusses the problem of education of children with disabilities, which is relevant today due to the objective difficulties of social functioning and the child's entry into society. It is known that the basis of a person's speech organization is vocabulary, the shortcomings of which significantly complicate the process of social adaptation. The features of the intellectual and speech development of students with mental retardation limit the ability of these children to understand the speech of others, to adequately express their own thoughts and lead to an inferiority of social and everyday orientation. Consequently, one of the urgent areas of work with children with mental retardation should be the formation and development of vocabulary, characterized by sufficiency and usefulness, in terms of the volume of vocabulary, semantics and syntagmatic characteristics. The solution of this practical problem will allow primary schoolchildren with intellectual disabilities to expand their vocabulary, to form their basic cognitive processes, in the structure of which speech occupies one of the basic positions. To study the level of formation of the semantic side of speech of children with intellectual underdevelopment at the present stage, we conducted a psychological and pedagogical study, in which 1376 schoolchildren from Kaluga and Obninsk, Kaluga region, took part. The diagnostic results made it possible to reveal the fact that with mental retardation, operations and levels of speech utterance generation (semantic, linguistic, sensorimotor levels) are impaired to varying degrees. The most underdeveloped are highly organized, complex levels (semantic, linguistic), requiring a high degree of formation of the operations of analysis, synthesis, generalization, the sensorimotor level of speech in these children suffers in different ways. The study showed that the symptomatology and mechanism of speech disorders in these children are determined not only by the presence of a general underdevelopment of the cerebral systems, which causes systemic speech impairment, but also local pathology on the part of systems directly related to speech, which further complicates the picture of speech impairments in mentally retarded children.

Concurrent listening affects speech planning and fluency: the roles of representational similarity and capacity limitation

ABSTRACT In a novel continuous speaking-listening paradigm, we explored how speech planning was affected by concurrent listening. In Experiment 1, Dutch speakers named pictures with high versus low name agreement while ignoring Dutch speech, Chinese speech, or eight-talker babble. Both name agreement and type of auditory input influenced response timing and chunking, suggesting that representational similarity impacts lexical selection and the scope of advance planning in utterance generation. In Experiment 2, Dutch speakers named pictures with high or low name agreement while either ignoring Dutch words, or attending to them for a later memory test. Both name agreement and attention demand influenced response timing and chunking, suggesting that attention demand impacts lexical selection and the planned utterance units in each response. The study indicates that representational similarity and attention demand play important roles in linguistic dual-task interference, and the interference can be managed by adapting when and how to plan speech.

Characteristics of Language Behavior and Speech Activity in Children with Autistic Disorders: Theoretical Aspect

Speech development of a child with autism does not always take place at the appropriate age and does not always obey the laws of development of speech functions. According to the analysis of literature sources, the direct study of speech activity of children with autistic disorders requires a more detailed study. According to our predictions, a child with autistic disorders of older preschool age due to the peculiarities of communicative and behavioral spheres will show a low level of speech activity, which can be explained directly by the specificity of speech development along with limited language experience and insufficient knowledge of language and its use in communication. Language behavior consists of two complementary and interrelated processes: psychological formation (generation) of speech utterance and perception of the expanded speech of the interlocutor. The model of speech utterance generation includes five consecutive, interconnected stages (phases) identified by O.O. Leontiev (1967): the motive of utterance; the idea of expression; internal programming; lexical and grammatical development of the utterance; implementation of speech expression in external speech. Speech activity is one of the many forms of general activity, a reflection of the needs that arise in accordance with specific communication situations, a prerequisite and an important component of language behavior. The term "speech activity" is considered by us in the sense of the presence of a motive for speech utterance and direct speech utterance, which may occur as a reaction-response to the interlocutor's remark or as a desire to inform the interlocutor of their own thoughts, experiences, emotions, needs.

A survey on the animation of signing avatars: From sign representation to utterance synthesis

Signing avatars make it possible for deaf people to access information in their preferred language. However, sign language synthesis represents a challenge for the computer animation community as the motions generated must be realistic and have a precise semantic meaning. In this article, we distinguish the synthesis of isolated signs deprived of any contextual inflections from the generation of full sign language utterances. In both cases, the animation engine takes as input a representation of the synthesis objective to create the final animation. Because of their spatiotemporal characteristics, signs and utterances cannot be described by a sequential representation like phonetics in spoken languages. For this reason, linguistic and gestural studies have aimed to capture the typical and special features of signs and sign language syntax to promote different sign language representations. Those sign representations can then be used to produce an avatar animation thanks to sign synthesis techniques based on keyframes, procedural means or data-driven approaches. Novel utterances can also be generated using concatenative or articulatory techniques.This article constitutes a survey of (i) the challenges specific to sign languages avatars, (ii) the sign representations developed in order to synthesize isolated signs, (iii) the possible sign synthesis approaches, (iv) the different utterance specifications, and (v) the challenges and animation techniques for generating sign language utterances.

The core vocabulary of South African Afrikaans-speaking Grade R learners without disabilities.

BackgroundAugmentative and alternative communication (AAC) can enable individuals with little or no functional speech to communicate functionally in a variety of communication contexts. AAC systems for individuals who are not (yet) fully literate often require that the vocabulary for the system be preselected. By including the most commonly- and most frequently-used words (core vocabulary) in an AAC system, access to novel utterance generation can arguably be facilitated. At present, no Afrikaans core vocabulary list based on children’s speech samples exists.ObjectivesThis study aimed to identify the most frequently- and commonly-used words of South African Afrikaans-speaking Grade R learners without disabilities.MethodSpontaneous speech samples were collected from 12 Afrikaans-speaking Grade R learners during regular preschool activities. Samples were transcribed and analysed to determine the number of different words used, the frequency with which each word was used, as well as the commonality of word use across the 12 participants.ResultsA total of 239 words met the criteria for inclusion in the core vocabulary (words used with a frequency of more than 0.05% in the sample, and used by at least half of the participants). These words accounted for 79.4% of words used in the entire speech sample.ConclusionThe established core vocabulary consists of a relatively small set of words that was found to represent a large proportion of speech. AAC team members may consider including these words on Afrikaans AAC systems that are intended to give access to a measure of novel utterance generation.

ACCOUNTING OF THE INTRA-LEXIC SYSTEM IN THE PRACTICAL COURSE OF THE RUSSIAN LANGUAGE

The article deals with issues related to the system structure of the language and the system relations between language units. The authors proceed from the modern understanding of the definition of the lexical-semantic system, which is generally accepted. Three main dimensions of lexical consistency are described: paradigmatic, syntagmatic, and derivational. the basic elements of the linguistic term system are the terms “syntagmatics” and “paradigmatics”. The essence of paradigmatic relations is the similarity of units under one family and opposites - for others. Syntagmatic relations in the vocabulary are manifested in the vocabulary in the rules of word combination, in their connection with other lexical units within specific utterances. The article offers a model of lexical presentation taking into account intra-lexical consistency. The functional level of the lexical unit implementation is conditionally accepted as the activity form of the word. Indicators of word assimilation in the process of its gradual transition from the subject form to the activity form were lexical operations that are part of the actions for the perception and generation of speech utterance. The authors describe the experience of developing and using training tasks for the C1 level in accordance with the requirements of the new standard curriculum.

Automated Utterance Generation

Conversational AI assistants are becoming popular and question-answering is an important part of any conversational assistant. Using relevant utterances as features in question-answering has shown to improve both the precision and recall for retrieving the right answer by a conversational assistant. Hence, utterance generation has become an important problem with the goal of generating relevant utterances (sentences or phrases) from a knowledge base article that consists of a title and a description. However, generating good utterances usually requires a lot of manual effort, creating the need for an automated utterance generation. In this paper, we propose an utterance generation system which 1) uses extractive summarization to extract important sentences from the description, 2) uses multiple paraphrasing techniques to generate a diverse set of paraphrases of the title and summary sentences, and 3) selects good candidate paraphrases with the help of a novel candidate selection algorithm.

Design and evaluation of a context-adaptive AAC application for people with aphasia

Communication with an Alternative and Augmentative Communication (AAC) device tends to be extremely slow, mainly due to difficulties in navigating the extensive vocabulary needed for the generation of spontaneous utterances in a manner that allows the user to easily find and select appropriate words when communicating. This work aims to design, implement, and evaluate a context-adaptive AAC application that suggests the most appropriate words according to the user location and past vocabulary usage.

Parody and Garden Path

AbstractThis is a study of Chinese rhetorical constructions, parody and garden path (GP), from the perspective of lexico-constructional pragmatics (LCP). LCP adopts a holographic view of lexicon and construction so that they can be analyzed alike. We take parody and GP examples from Chinese advertisements for analysis. The LCP analysis highlights the pragmaticity and rhetoricality of each case: for a particular effect. When difficulty arises, pragmatic means may be used to “pragma-coerce” the right, clever, or erroneous use of a rhetorical construction for delivery of a retrievable intended effect, an Aha-effect. We conduct a mini-questionnaire with two cases, the former dealing with parody and the latter with GP. The study indicates the humor competence of ordinary Chinese participants (around the level of BA) as far as parody and GP are concerned and the participants’ potential for cognition of the pragma-rhetorical values of parody and GP or their potential for generation of such utterances. It is suggested that rhetorical constructions outwit the less rhetorical or grammatical constructionsiffthey are available and accessible, and that LCP can offer us a feasible interpretation of such tropes as parody and GP.

Utterance Generation With Variational Auto-Encoder for Slot Filling in Spoken Language Understanding

Slot filling must be trained using human-labeled data that are expensive and only a limited amount of labeled utterances are readily available for learning. Data generation methods can help increase the size of the dataset and make variations to the training dataset by means of emerging new instances. We propose a novel labeled utterance generation algorithm to augment training data. Our hypothesis is that words in an utterance can be separated into the two parts, namely, slot values that are instances of slot types and the accompanying contexts. Our model aims to generate utterances that are diverse combinations of slot values and contexts that can appear together. To create various utterances containing a given condition, our deep generative model uses a conditional variational auto-encoder architecture. We conduct experiments on various slot filling datasets, specifically airline travel information systems (ATIS), Snips, and MIT Corpus. A quantitative analysis shows that the application of data augmentation using the proposed model improves the F1 score for slot filling. We also demonstrate that our labeled utterance generation model yields more desirable utterances.