Dialogue Model Research Articles

On-device query intent prediction with lightweight LLMs to support ubiquitous conversations

Conversational Agents (CAs) have made their way to providing interactive assistance to users. However, the current dialogue modelling techniques for CAs are predominantly based on hard-coded rules and rigid interaction flows, which negatively affects their flexibility and scalability. Large Language Models (LLMs) can be used as an alternative, but unfortunately they do not always provide good levels of privacy protection for end-users since most of them are running on cloud services. To address these problems, we leverage the potential of transfer learning and study how to best fine-tune lightweight pre-trained LLMs to predict the intent of user queries. Importantly, our LLMs allow for on-device deployment, making them suitable for personalised, ubiquitous, and privacy-preserving scenarios. Our experiments suggest that RoBERTa and XLNet offer the best trade-off considering these constraints. We also show that, after fine-tuning, these models perform on par with ChatGPT. We also discuss the implications of this research for relevant stakeholders, including researchers and practitioners. Taken together, this paper provides insights into LLM suitability for on-device CAs and highlights the middle ground between LLM performance and memory footprint while also considering privacy implications.

Dialogue act-based partner persona extraction for consistent personalized response generation

The ability of a dialogue model to keep not being out of context during a conversation, so-called keeping the consistency, has long been a critical issue in generating more human-like personalized responses. However, most of the previous works have focused on a self persona to sustain the self consistency during a conversation. Since the consistency is not limited to only the self side, there still lies the problem where generated responses often contradict the utterance of a partner. This kind of behavior discourages the user from responding and eventually causes the user to leave the conversation. To prevent this from happening, our work focuses on recognizing the partner persona. We propose a new model, PEDA (Persona Extractor based on Dialogue Act), and construct an appropriate dataset for training the model. Specifically, dialogue acts are utilized to identify the utterances that capture a user’s persona. The proposed model extracts the partner persona from the combination of utterances and their dialogue acts. We propose a dialogue act conductor to properly consider the dialogue act as an input and make use of it with the pre-trained language model. A proposed gating mechanism controls the probability distributions from dialogue act conductor and pre-trained language model. Finally, we train the model further by setting up a reinforcement learning framework with our evaluation network.

Concept and Implementation Model Multicultural Islamic Education FKUB Balikpapan City in Developing a Tolerant Attitude

In Balikpapan City as a city whose level of diversity is so complex, it cannot be separated from social problems that can lead to social disintegration. The preventive action taken by the Balikpapan City Government is to build a partnership between the Balikpapan City Religious Ummah Communication Forum (FKUB). The Balikpapan City Government's step in starting with FKUB is a strategic step in strengthening multicultural values to anticipate the disintegration of socialization. The review of this paper is based on the results of research with a qualitative descriptive approach to the phenomena and facts that occur at the research locus. Using this type of phenomenology, the author reveals the concepts built and models implemented about the values of multicultural Islamic education in Balikpapan City. The existence of the term multicultural as a social term that views cultural equality, then multicultural value in the context of Islamic teachings is the concept of social ethics in covering cultural diversity according to the views of Islamic teachings. The values developed and most crucial are: Silaturrahim, please help, equality and equal rights, justice and care. In instilling Islamic education values, the models and multicultural approaches carried out by FKUB Balikpapan City are: a. Interreligious Dialogue Model, b. Social Action Model, c. Empowerment and Engagement of Social Resources Model 1. In developing the tolerant attitude of the people of Balikpapan City, FKUB takes a role by instilling and strengthening the values of multicultural Islamic education. 2. The values of multicultural Islamic education become crucial values in a society with such a complex level of plurality as in Balikpapan City. 3. In instilling and strengthening the values of Islamic education, the multicultural model carried out by FKUB Balikpapan City is the Interfaith Dialogue Model, Social Action Model, Empowerment Model and Social Resource Involvement.

MOSS: An Open Conversational Large Language Model

Conversational large language models (LLMs) such as ChatGPT and GPT-4 have recently exhibited remarkable capabilities across various domains, capturing widespread attention from the public. To facilitate this line of research, in this paper, we report the development of MOSS, an open-sourced conversational LLM that contains 16 B parameters and can perform a variety of instructions in multi-turn interactions with humans. The base model of MOSS is pre-trained on large-scale unlabeled English, Chinese, and code data. To optimize the model for dialogue, we generate 1.1 M synthetic conversations based on user prompts collected through our earlier versions of the model API. We then perform preference-aware training on preference data annotated from AI feedback. Evaluation results on real-world use cases and academic benchmarks demonstrate the effectiveness of the proposed approaches. In addition, we present an effective practice to augment MOSS with several external tools. Through the development of MOSS, we have established a complete technical roadmap for large language models from pre-training, supervised fine-tuning to alignment, verifying the feasibility of chatGPT under resource-limited conditions and providing a reference for both the academic and industrial communities. Model weights and code are publicly available at https://github.com/OpenMOSS/MOSS.

Moving to Advanced Research on Human Machine Interface Design

This poster paper will discuss a variety of items related to Human-Machine Teaming and research in support in-creasing control of autonomous machines present in phys-ical problem domains of interest. Many military tasks can be decomposed into their primary elements – intelligence preparation, reconnaissance, movement, maneuver, fires, and and support across the combat domains of interest – air, land, sea, etc. There are an increasing number of au-tonomous and semi-autonomous ground-based systems available, such as the Multi-Utility Tactical Transport (MUTT) Unmanned Ground Vehicle, for movement of materials, or Quadrupedal Unmanned Ground Vehicle (QUGV) for the disposal of explosive ordinance, comple-mented by aerial platforms considering of a wide variety of Unmanned Aerial Systems (UAS) for the gathering and transmitting of information, held together by a common backbone and network. The preponderance of new systems and capabilities brings new issues, one of which is critical to research - how to control and manage a large number of systems. Commercial systems of significantly less capability, such as light-up drone shows, involve approximately 2 people per 100 drones; meaning that a single controller for a sin-gle drone is simply a non-starter. How can research be applied to scale the complexity of operations upwards without additional demands of personnel? This poster discusses several portions of early-stage research into a variety of applications, including: Reasoning about mixed-team processes, includ-ing real/synthetic teammates, and how infor-mation gathered about the use of synthetic teammates within simulation can be transitioned into utilization for a robotic teammate Embedding affective information into dialogue channels in order to save cognitive bandwidth Repurposing of foundational dialogue models for specific tasks and purposes Utilizing psychological research to design sys-tems in order to infer user intent Theory of mind research for autonomous systems regarding their human operators Simulated environments and agents in order to test the simulations in representative areas. Common-Sense reasoning augmented by Large Language Model technology for instructing ro-botic platforms The following things will be discussed at the poster ses-sion, representing a portfolio of ongoing work addressing the problems of large-scale multi-agent command and control. While the near-term application of these tech-nologies is into military problem domains; the poster presentation is cleared for public release.

Combining statistical dialog management and intent recognition for enhanced response selection

Abstract Conversational interfaces are becoming ubiquitous in an increasing number of application domains as Artificial Intelligence, Natural Language Processing and Machine Learning methods associated with the recognition, understanding and generation of natural language advance by leaps and bounds. However, designing the dialog model of these systems is still a very demanding task requiring a great deal of effort given the number of information sources to be considered related to the analysis of user utterances, interaction context, information repositories, etc. In this paper, we present a general framework for increasing the quality of the system responses by combining a statistical dialog management technique and a deep learning-based intention recognizer that allow replacing the system responses initially selected by the statistical dialog model with other presumably better candidates. This approach is portable to different task-oriented domains, a diversity of methodologies for dialog management and intention estimation techniques. We have evaluated our two-step proposal using two conversational systems, assessed several intention recognition methodologies and used the developed modules to dynamically select the system responses. The results of the evaluation show that the proposed framework achieves satisfactory results by making it possible to reduce the number of non-coherent dialog responses by replacing them by more coherent alternatives.

Conversational Speech Behaviors Are Context Dependent.

According to the interpersonal synergy model of spoken dialogue, interlocutors modify their communicative behaviors to meet the contextual demands of a given conversation. Although a growing body of research supports this postulation for linguistic behaviors (e.g., semantics, syntax), little is understood about how this model applies to speech behaviors (e.g., speech rate, pitch). The purpose of this study is to test the hypothesis that interlocutors adjust their speech behaviors across different conversational tasks with different conversational goals. In this study, 28 participants each engaged in two different types of conversations (i.e., relational and informational) with two partners (i.e., Partner 1 and Partner 2), yielding a total of 112 conversations. We compared six acoustic measures of participant speech behavior across conversational task and partner. Linear mixed-effects models demonstrated significant differences between speech feature measures in informational and relational conversations. Furthermore, these findings were generally robust across conversations with different partners. Results suggest that contextual demands influence speech behaviors. These findings provide empirical support for the interpersonal synergy model and highlight important considerations for assessing speech behaviors in individuals with communication disorders.

War, terror and mourning. Cultural memory in the inner dialogue

ABSTRACT This text compares four essays dealing with war, state terror in dictatorships, social violence such as racism, mourning and the avoidance of mourning. It shows that dictatorships share similarities in their techniques of manipulation, linguistic style and reference to history. They seek to exploit national myths through manipulative alienation. Myths are a central element of cultural memory, and their effect can be understood through a model of internal dialogue. This dialogue determines whether the regime's attempts at manipulation are successful.

ConvoSense: Overcoming Monotonous Commonsense Inferences for Conversational AI

Abstract Mastering commonsense understanding and reasoning is a pivotal skill essential for conducting engaging conversations. While there have been several attempts to create datasets that facilitate commonsense inferences in dialogue contexts, existing datasets tend to lack in-depth details, restate information already present in the conversation, and often fail to capture the multifaceted nature of commonsense reasoning. In response to these limitations, we compile a new synthetic dataset for commonsense reasoning in dialogue contexts using GPT, ℂonvoSense, that boasts greater contextual novelty, offers a higher volume of inferences per example, and substantially enriches the detail conveyed by the inferences. Our dataset contains over 500,000 inferences across 12,000 dialogues with 10 popular inference types, which empowers the training of generative commonsense models for dialogue that are superior in producing plausible inferences with high novelty when compared to models trained on the previous datasets. To the best of our knowledge, ℂonvoSense is the first of its kind to provide such a multitude of novel inferences at such a large scale.

Translating motivational interviewing for the HPV vaccine into a computable ontology model for automated AI conversational interaction.

Human papillomavirus (HPV) vaccinations are lower than expected. To protect the onset of head and neck cancers, innovative strategies to improve the rates are needed. Artificial intelligence may offer some solutions, specifically conversational agents to perform counseling methods. We present our efforts in developing a dialogue model for automating motivational interviewing (MI) to encourage HPV vaccination. We developed a formalized dialogue model for MI using an existing ontology-based framework to manifest a computable representation using OWL2. New utterance classifications were identified along with the ontology that encodes the dialogue model. Our work is available on GitHub under the GPL v.3. We discuss how an ontology-based model of MI can help standardize/formalize MI counseling for HPV vaccine uptake. Our future steps will involve assessing MI fidelity of the ontology model, operationalization, and testing the dialogue model in a simulation with live participants.

Using explainable AI to unravel classroom dialogue analysis: Effects of explanations on teachers' trust, technology acceptance and cognitive load

AbstractDeep neural networks are increasingly employed to model classroom dialogue and provide teachers with prompt and valuable feedback on their teaching practices. However, these deep learning models often have intricate structures with numerous unknown parameters, functioning as black boxes. The lack of clear explanations regarding their classroom dialogue analysis likely leads teachers to distrust and underutilize these AI‐powered models. To tackle this issue, we leveraged explainable AI to unravel classroom dialogue analysis and conducted an experiment to evaluate the effects of explanations. Fifty‐nine pre‐service teachers were recruited and randomly assigned to either a treatment (n = 30) or control (n = 29) group. Initially, both groups learned to analyse classroom dialogue using AI‐powered models without explanations. Subsequently, the treatment group received both AI analysis and explanations, while the control group continued to receive only AI predictions. The results demonstrated that teachers in the treatment group exhibited significantly higher levels of trust in and technology acceptance of AI‐powered models for classroom dialogue analysis compared to those in the control group. Notably, there were no significant differences in cognitive load between the two groups. Furthermore, teachers in the treatment group expressed high satisfaction with the explanations. During interviews, they also elucidated how the explanations changed their perceptions of model features and attitudes towards the models. This study is among the pioneering works to propose and validate the use of explainable AI to address interpretability challenges within deep learning‐based models in the context of classroom dialogue analysis.Practitioner notesWhat is already known about this topic Classroom dialogue is recognized as a crucial element in the teaching and learning process. Researchers have increasingly utilized AI techniques, particularly deep learning methods, to analyse classroom dialogue. Deep learning‐based models, characterized by their intricate structures, often function as black boxes, lacking the ability to provide transparent explanations regarding their analysis. This limitation can result in teachers harbouring distrust and underutilizing these models. What this paper adds This paper highlights the importance of incorporating explainable AI approaches to tackle the interpretability issues associated with deep learning‐based models utilized for classroom dialogue analysis. Through an experimental study, this paper demonstrates that providing model explanations enhances teachers' trust in and technology acceptance of AI‐powered classroom dialogue models, without increasing their cognitive load. Teachers express satisfaction with the model explanations provided by explainable AI. Implications for practice and/or policy The integration of explainable AI can effectively address the challenge of interpretability in complex AI‐powered models used for analysing classroom dialogue. Intelligent teaching systems designed for classroom dialogue can benefit from advanced AI models and explainable AI approaches, which offer users both automated analysis and clear explanations. By enabling users to understand the underlying rationale behind the analysis, the explanations can contribute to fostering trust and acceptance of the AI models among users.

Reference Image Aided Color Matching Design Based on Interactive Genetic Algorithm

This paper proposes an interactive genetic algorithm to assist designers in color selection and match different color combinations extracted. Then, based on the CorelDraw development environment, this paper establishes a graphic-based man-machine dialogue model. Through simulation experiments, genetic algorithms such as random selection, hybridization, variation, color region adjustment, etc. Finally, the best color combination is obtained and combined with the image. Within the scope of color extraction value, designers can use interactive genetic algorithm rules to view color planning from a holistic perspective. Under the influence of human-computer interaction, each scheme is optimized step by step, so that the color ideas matching the designer can be found quickly.

BVA-Transformer: Image-text multimodal classification and dialogue model architecture based on Blip and visual attention mechanism

Multimodal tasks have become a hot research direction in recent years. The emergence of large-scale models has gradually elevated the extensive multimodal tasks, achieving remarkable achievements. However, when it comes to fusing multiple modalities for multimodal tasks, how to better integrate multimodal features is still a problem worth exploring. In tasks such as sentiment analysis targeting a wide range of social media content, the use of features derived solely from the [CLS] token may lead to insufficient information. This paper proposes BVA-Transformer model architecture for image-text multimodal classification and dialogue, which incorporates the EF-CaTrBERT method for feature fusion and introduces BLIP for the transformation of images to the textual space. This enables the fusion of images and text in the same information space, avoiding issues of information redundancy and conflict compared to traditional feature fusion methods. In addition, we proposed a Global Features Encoder (GFE) module based on visual attention in the BVA-Transformer, which can provide more global and targeted auxiliary features for the [CLS] token. This enables the model to utilize more feature information in classification tasks under this feature fusion method and dynamically select information to focus on. We also introduced the Trv structure from EVA-02 in the Decoder part of BVA-Transformer, investigating its impact on the model performance. Furthermore, we designed a three-stage training to further enhance the model’s performance. Experimental results demonstrate that BVA-Transformer achieves high-quality classification while generating dialogue sentences. Compared to existing multimodal classification models on our validation dataset, it exhibits excellent performance.

The Gaze Dialogue Model: Nonverbal Communication in HHI and HRI.

When humans interact with each other, eye gaze movements have to support motor control as well as communication. On the one hand, we need to fixate the task goal to retrieve visual information required for safe and precise action-execution. On the other hand, gaze movements fulfil the purpose of communication, both for reading the intention of our interaction partners, as well as to signal our action intentions to others. We study this Gaze Dialogue between two participants working on a collaborative task involving two types of actions: 1) individual action and 2) action-in-interaction. We recorded the eye-gaze data of both participants during the interaction sessions in order to build a computational model, the Gaze Dialogue, encoding the interplay of the eye movements during the dyadic interaction. The model also captures the correlation between the different gaze fixation points and the nature of the action. This knowledge is used to infer the type of action performed by an individual. We validated the model against the recorded eye-gaze behavior of one subject, taking the eye-gaze behavior of the other subject as the input. Finally, we used the model to design a humanoid robot controller that provides interpersonal gaze coordination in human-robot interaction scenarios. During the interaction, the robot is able to: 1) adequately infer the human action from gaze cues; 2) adjust its gaze fixation according to the human eye-gaze behavior; and 3) signal nonverbal cues that correlate with the robot's own action intentions.

Opening the Black Box: Unraveling the Classroom Dialogue Analysis (Student Abstract)

This paper explores proposing interpreting methods from explainable artificial intelligence to address the interpretability issues in deep learning-based models for classroom dialogue. Specifically, we developed a Bert-based model to automatically detect student talk moves within classroom dialogues, utilizing the TalkMoves dataset. Subsequently, we proposed three generic interpreting methods, namely saliency, input*gradient, and integrated gradient, to explain the predictions of classroom dialogue models by computing input relevance (i.e., contribution). The experimental results show that the three interpreting methods can effectively unravel the classroom dialogue analysis, thereby potentially fostering teachers' trust.

Fact-Driven Logical Reasoning for Machine Reading Comprehension

Recent years have witnessed an increasing interest in training machines with reasoning ability, which deeply relies on accurately and clearly presented clue forms. The clues are usually modeled as entity-aware knowledge in existing studies. However, those entity-aware clues are primarily focused on commonsense, making them insufficient for tasks that require knowledge of temporary facts or events, particularly in logical reasoning for reading comprehension. To address this challenge, we are motivated to cover both commonsense and temporary knowledge clues hierarchically. Specifically, we propose a general formalism of knowledge units by extracting backbone constituents of the sentence, such as the subject-verb-object formed ``facts''. We then construct a supergraph on top of the fact units, allowing for the benefit of sentence-level (relations among fact groups) and entity-level interactions (concepts or actions inside a fact). Experimental results on logical reasoning benchmarks and dialogue modeling datasets show that our approach improves the baselines substantially, and it is general across backbone models. Code is available at https://github.com/ozyyshr/FocalReasoner.

OSTOD: One-Step Task-Oriented Dialogue with activated state and retelling response

As of present, the progress of conversational AI research has been greatly propelled by large-scale pre-trained language models. In particular, task-oriented dialogue systems have gained widespread attention owing to their immense potential in helping individuals accomplish diverse objectives, such as booking hotels, making restaurant reservations, and purchasing train tickets. In the past, task-oriented dialogue systems were typically viewed as a multi-step process that included spoken language understanding, dialogue state tracking, dialogue policy learning, and natural language generation. More recently, large-scale pre-trained language models enables the development of end-to-end neural pipeline task-oriented dialogue systems, which combine multiple steps into a single model, allowing for joint optimization and preventing error propagation. However, in order to explicitly retrieve information from databases to ensure the interpretability of the system, almost all end-to-end neural pipeline methods inevitably require predicting dialogue state as an intermediate result specialized for the domain or task, which results in significant challenges for generalization. To solve the problem above, we propose One-Step Task-Oriented Dialogue (OSTOD) in this paper, which models task-oriented dialogue by synchronously generating activated states and retelling responses, where activated states refer to slot values that contribute to database access, and retelling responses are system responses that contain activated state information. Specifically, first, automatic methods are designed to build data containing activated states and retelling responses. Then, a joint generation model that synchronously predicts activated states and retelling responses in a single step is proposed for task-oriented dialogue modelling. Based on empirical results obtained from the MultiWOZ 2.0 and MultiWOZ 2.1 datasets, our OSTOD model demonstrates comparable performance to state-of-the-art baselines. Moreover, our model exhibits exceptional generalization capabilities in few-shot learning and domain transfer scenarios.

Information-seeking dialogue for explainable artificial intelligence: Modelling and analytics

Explainable artificial intelligence has become a vitally important research field aiming, among other tasks, to justify predictions made by intelligent classifiers automatically learned from data. Importantly, efficiency of automated explanations may be undermined if the end user does not have sufficient domain knowledge or lacks information about the data used for training. To address the issue of effective explanation communication, we propose a novel information-seeking explanatory dialogue game following the most recent requirements to automatically generated explanations. Further, we generalise our dialogue model in form of an explanatory dialogue grammar which makes it applicable to interpretable rule-based classifiers that are enhanced with the capability to provide textual explanations. Finally, we carry out an exploratory user study to validate the corresponding dialogue protocol and analyse the experimental results using insights from process mining and argument analytics. A high number of requests for alternative explanations testifies the need for ensuring diversity in the context of automated explanations.

Exploring the role of research in teacher education and teachers’ professional development

This paper explores and discusses the role of research in teacher education and teachers’ professional development. It does so by presenting some perspectives on research-based teacher education and conceptual frameworks for teacher professionalism. The relationship between theoretical and practical knowledge is discussed together with central dimensions of teacher professionalism. Three metaphors and figures of thought for the theory–practice relationship are introduced and discussed: research as a mirror, as a compass, and as a stone in the shoe. Against this backdrop, a couple of examples of models for teachers’ professional development through critical engagement and enquiry are presented. Special attention is given to a critical dialogical model for teacher education and professional development that promotes reflective learning through sharing experiences and engaging with theoretical frameworks. This model integrates reflective-analogical, critical-analytical, and interactive self-building processes, enabling teachers and students to transform their knowledge and practices based on new insights.

MedChatZH: A tuning LLM for traditional Chinese medicine consultations

Generative Large Language Models (LLMs) have achieved significant success in various natural language processing tasks, including Question-Answering (QA) and dialogue systems. However, most models are trained on English data and lack strong generalization in providing answers in Chinese. This limitation is especially evident in specialized domains like traditional Chinese medical QA, where performance suffers due to the absence of fine-tuning and high-quality datasets. To address this, we introduce MedChatZH, a dialogue model optimized for Chinese medical QA based on transformer decoder with LLaMA architecture. Continued pre-training on a curated corpus of Chinese medical books is followed by fine-tuning with a carefully selected medical instruction dataset, resulting in MedChatZH outperforming several Chinese dialogue baselines on a real-world medical dialogue dataset. Our model, code, and dataset are publicly available on GitHub (https://github.com/tyang816/MedChatZH) to encourage further research in traditional Chinese medicine and LLMs.