Open-domain Dialogue Research Articles

Generative commonsense knowledge subgraph retrieval for open-domain dialogue response generation

Grounding on a commonsense knowledge subgraph can help the model generate more informative and diverse dialogue responses. Prior Traverse-based works explicitly retrieve a subgraph from the external knowledge base (eKB). Notably, the available knowledge is strictly restricted by the eKB. To break this restriction, Generative Retrieval methods externalize knowledge from the language model. However, they always generate boring knowledge due to their one-pass externalization procedure. This work proposes a novel TiLM Traverse in Language Model (TiLM), which uses three ‘Chain-of-Thought’ sub-tasks, i.e., Query Entity Production, Topic Entity Prediction, and Knowledge Subgraph Completion, to build a high-quality knowledge subgraph to ground the next Response Generation without explicitly accessing the eKB in inference. Experimental results on both Chinese and English datasets demonstrate TiLM’s outstanding performance even only with a small scale of parameters.

Saliency infused dialogue response generation: Improving task oriented text generation using feature attribution

Challenges persist in dialogue scenarios, particularly in multi-turn dialogues where response generation often disregards contextual information beyond the last user utterance, resulting in fluent yet inadequate responses. This paper addresses these issues by identifying and resolving common shortcomings in base model responses during response generation and proposes methods to enhance response quality in unannotated dialogue settings. Our approach involves augmenting information from multiple sources, including keywords, salient features, and knowledge graph triples. We compare the effectiveness of these methods against both the base model and human annotation, which includes dialogue acts and entities. Our findings demonstrate that appending extracted tokens significantly enhances response quality compared to annotated information. In task-oriented dialogue, models perform best when infused with saliency and knowledge graph triples, as shown in the MultiWOZ dataset. Conversely, focusing solely on saliency yields better results for open-domain dialogue, as demonstrated with the DailyDialog dataset. For contextual relevance, the information infusion could also approach the performance of the LLama2 model with only a tenth of the available parameters.

An Adaptive Contextual Relation Model for Improving Response Generation

An Adaptive Contextual Relation Model for Improving Response Generation

From easy to hard: Improving personalized response generation of task-oriented dialogue systems by leveraging capacity in open-domain dialogues

A task-oriented dialogue system (TOD) is an important application of artificial intelligence. In the past few years, works on personalized TODs have attracted increased research attention and have seen much progress. The main challenge of such dialogue systems is finding ways to exploit user profiles under conditions of fixed and monolithic dialogue style. However, most of the existing works overlook the observation that the personalization capability of the dialogue system is fundamental and generic, and they treat all attributes of the user profile equally throughout the dialogue flows, which makes them inadequate for developing a well-performing personalized TOD. In this paper, we propose a two-stage learning framework equipped with GPT2 as a backbone to alleviate the above two problems. In the first learning phase, we finetune the GPT2 model on the personalized open-domain dialogues to preliminarily acquire personalization power. Then, we transfer this power to personalized task-oriented dialogues for the second stage of learning. After that, we enable the proposed model to be capable of the desired personalization capacity. Moreover, we present a dynamic profile fusion mechanism and an auxiliary task that detects which attributes contribute to the current utterance to facilitate the model’s performance. Eventually, we rewrite the attribute descriptions of user profiles in sentences to mitigate the consistency gap between the open-domain and task-oriented dialogues. The experimental results show that the proposed model achieves superior results compared to the state-of-the-art models on two versions of the Personalized bAbI dataset.

MODE: a multimodal open-domain dialogue dataset with explanation

MODE: a multimodal open-domain dialogue dataset with explanation

Comparative study on personalized open-domain generative dialogue systems

Developing personalized dialogue systems is a monumental challenge in the rapidly evolving domain of Natural Language Processing (NLP). These systems are engineered to facilitate more natural, intuitive, and user-friendly interactions between humans and machines. Their theoretical significance extends to real-world applications, making them an invaluable asset in today's digital age. Over recent years, this area has garnered immense attention from researchers and academics, becoming a hotbed for innovation and study. This paper aims to contribute to this burgeoning field by conducting an in-depth comparative analysis of the current leading personalized language models. Specifically, this paper will scrutinize models grounded in deep learning algorithms and those that employ reinforcement learning techniques. This research objective is to dissect each approach's unique advantages and limitations. By doing so, the paper hopes to identify actionable avenues for improvement and optimization. Furthermore, the paper will offer a forward-looking perspective, outlining potential advancements and innovations that could shape the future landscape of personalized dialogue systems in Natural Language Processing.

CORECODE: A Common Sense Annotated Dialogue Dataset with Benchmark Tasks for Chinese Large Language Models

As an indispensable ingredient of intelligence, commonsense reasoning is crucial for large language models (LLMs) in real-world scenarios. In this paper, we propose CORECODE, a dataset that contains abundant commonsense knowledge manually annotated on dyadic dialogues, to evaluate the commonsense reasoning and commonsense conflict detection capabilities of Chinese LLMs. We categorize commonsense knowledge in everyday conversations into three dimensions: entity, event, and social interaction. For easy and consistent annotation, we standardize the form of commonsense knowledge annotation in open-domain dialogues as "domain: slot = value". A total of 9 domains and 37 slots are defined to capture diverse commonsense knowledge. With these pre-defined domains and slots, we collect 76,787 commonsense knowledge annotations from 19,700 dialogues through crowdsourcing. To evaluate and enhance the commonsense reasoning capability for LLMs on the curated dataset, we establish a series of dialogue-level reasoning and detection tasks, including commonsense knowledge filling, commonsense knowledge generation, commonsense conflict phrase detection, domain identification, slot identification, and event causal inference. A wide variety of existing open-source Chinese LLMs are evaluated with these tasks on our dataset. Experimental results demonstrate that these models are not competent to predict CORECODE's plentiful reasoning content, and even ChatGPT could only achieve 0.275 and 0.084 accuracy on the domain identification and slot identification tasks under the zero-shot setting. We release the data and codes of CORECODE at https://github.com/danshi777/CORECODE to promote commonsense reasoning evaluation and study of LLMs in the context of daily conversations.

Seq2Set2Seq: A Two-stage Disentangled Method for Reply Keyword Generation in Social Media

Social media produces large amounts of content every day. How to predict the potential influences of the contents from a social reply feedback perspective is a key issue that has not been explored. Thus, we propose a novel task named reply keyword prediction in social media, which aims to predict the keywords in the potential replies in as many aspects as possible. One prerequisite challenge is that the accessible social media datasets labeling such keywords remain absent. To solve this issue, we propose a new dataset, 1 to study the reply keyword prediction in social media. This task could be seen as a single-turn dialogue keyword prediction for open-domain dialogue system. However, existing methods for dialogue keyword prediction cannot be adopted directly, which has two main drawbacks. First, they do not provide an explicit mechanism to model topic complementarity between keywords which is crucial in social media to controllably model all aspects of replies. Second, the collocations of keywords are not explicitly modeled, which also makes it less controllable to optimize for fine-grained prediction since the context information is much less than that in dialogue. To address these issues, we propose a two-stage disentangled framework, which can optimize the complementarity and collocation explicitly in a disentangled fashion. In the first stage, we use a sequence-to-set paradigm via multi-label prediction and determinantal point processes, to generate a set of keyword seeds satisfying the complementarity. In the second stage, we adopt a set-to-sequence paradigm via seq2seq model with the keyword seeds guidance from the set, to generate the more-fine-grained keywords with collocation. Experiments show that this method can generate not only a more diverse set of keywords but also more relevant and consistent keywords. Furthermore, the keywords obtained based on this method can achieve better reply generation results in the retrieval-based system than others.

Personality prediction from task-oriented and open-domain human–machine dialogues

If a dialogue system can predict the personality of a user from dialogue, it will enable the system to adapt to the user’s personality, leading to better task success and user satisfaction. In a recent study, personality prediction was performed using the Myers–Briggs Type Indicator (MBTI) personality traits with a task-oriented human–machine dialogue using an end-to-end (neural-based) system. However, it is still not clear whether such prediction is generally possible for other types of systems and user personality traits. To clarify this, we recruited 378 participants, asked them to fill out four personality questionnaires covering 25 personality traits, and had them perform three rounds of human–machine dialogue with a pipeline task-oriented dialogue system or an end-to-end task-oriented dialogue system. We also had another 186 participants do the same with an open-domain dialogue system. We then constructed BERT-based models to predict the personality traits of the participants from the dialogues. The results showed that prediction accuracy was generally better with open-domain dialogue than with task-oriented dialogue, although Extraversion (one of the Big Five personality traits) could be predicted equally well for both open-domain dialogue and pipeline task-oriented dialogue. We also examined the effect of utilizing different types of dialogue on personality prediction by conducting a cross-comparison of the models trained from the task-oriented and open-domain dialogues. As a result, we clarified that the open-domain dialogue cannot be used to predict personality traits from task-oriented dialogue, and vice versa. We further analyzed the effects of system utterances, task performance, and the round of dialogue with regard to the prediction accuracy.

Towards Building an Open-Domain Dialogue System Incorporated With Internet Memes

In recent years, Internet memes have been widely used in online chatting. Compared with text-based communication, conversations become more expressive and attractive when Internet memes are incorporated. This paper presents our solutions for the Meme incorporated Open-domain Dialogue (MOD) Challenge of DSTC10, where three tasks are involved: text response modeling, meme retrieval, and meme emotion classification. Firstly, we leverage a large-scale pre-trained dialogue model for coherent and informative response generation. Secondly, based on interaction-based text-matching, our approach can retrieve appropriate memes with good generalization ability. Thirdly, we propose to model the emotion flow (EF) in conversations and introduce an auxiliary task of emotion description prediction (EDP) to boost the performance of meme emotion classification. Experimental results on the MOD dataset demonstrate that our methods can incorporate Internet memes into dialogue systems effectively.

Emotion-and-knowledge grounded response generation in an open-domain dialogue setting

The neural-based interactive dialogue system focuses on engaging and retaining humans in long-lasting conversations. This has been explored for a variety of goal-oriented dialogue domains, such as education, health care, entertainment, sports, and politics. To develop an understanding and awareness of social and cultural norms, and to address specific social skills, we need to invent strategies for building interactive systems that take into account the user’s emotions and relevant-facts in a multi-turn conversation. In this paper, we propose a new neural generative model that combines step-wise co-attention with a self-attention-based transformer network along with an emotion classifier to jointly control emotion and knowledge transfer during response generation. Quantitative, qualitative, and human evaluation results on the benchmark Topical Chat and the CMU_DoG dataset show that the proposed models can generate natural and coherent sentences, capturing essential facts with considerable improvement over emotional content.

HumanoidBot: framework for integrating full-body humanoid robot with open-domain chat system

ABSTRACT State-of-the-art open-domain dialogue systems have made significant progress over the past few years using new model architectures that reliably preserve the context of dialogue and large dataset collection. However, there are no examples of such state-of-the-art open-domain dialogue systems implemented on humanoid robots. Implementing a dialogue system that can naturally interact with humans is a fundamental issue influencing the future of humanoid robots. Therefore, this paper proposes HumanoidBot, a framework for implementing an open-domain chat system on a full-body humanoid robot. Furthermore, an evaluation experiment using a humanoid robot implemented with HumanoidBot verified how balance control affects the dialogue and impression of the robot in humanoid robot dialogue.

Heterogeneous-Branch Collaborative Learning for Dialogue Generation

With the development of deep learning, advanced dialogue generation methods usually require a greater amount of computational resources. One promising approach to obtaining a high-performance and lightweight model is knowledge distillation, which relies heavily on the pre-trained powerful teacher. Collaborative learning, also known as online knowledge distillation, is an effective way to conduct one-stage group distillation in the absence of a well-trained large teacher model. However, previous work has a severe branch homogeneity problem due to the same training objective and the independent identical training sets. To alleviate this problem, we consider the dialogue attributes in the training of network branches. Each branch learns the attribute-related features based on the selected subset. Furthermore, we propose a dual group-based knowledge distillation method, consisting of positive distillation and negative distillation, to further diversify the features of different branches in a steadily and interpretable way. The proposed approach significantly improves branch heterogeneity and outperforms state-of-the-art collaborative learning methods on two widely used open-domain dialogue datasets.

Less is More: Mitigate Spurious Correlations for Open-Domain Dialogue Response Generation Models by Causal Discovery

Abstract In this paper, we conduct the first study on spurious correlations for open-domain response generation models based on a corpus CGDialog curated by ourselves. The current models indeed suffer from spurious correlations and have a tendency to generate irrelevant and generic responses. Inspired by causal discovery algorithms, we propose a novel model-agnostic method for training and inference using a conditional independence classifier. The classifier is trained by a constrained self-training method, coined ConSTrain, to overcome data sparsity. The experimental results based on both human and automatic evaluation show that our method significantly outperforms the competitive baselines in terms of relevance, informativeness, and fluency.

A Stack-Propagation Framework for Low-Resource Personalized Dialogue Generation

With the resurgent interest in building open-domain dialogue systems, the dialogue generation task has attracted increasing attention over the past few years. This task is usually formulated as a conditional generation problem, which aims to generate a natural and meaningful response given dialogue contexts and specific constraints, such as persona. And maintaining a consistent persona is essential for the dialogue systems to gain trust from the users. Although tremendous advancements have been brought, traditional persona-based dialogue models are typically trained by leveraging a large number of persona-dense dialogue examples. Yet, such persona-dense training data are expensive to obtain, leading to a limited scale. This work presents a novel approach to learning from limited training examples by regarding consistency understanding as a regularization of response generation. To this end, we propose a novel stack-propagation framework for learning a generation and understanding pipeline. Specifically, the framework stacks a Transformer encoder and two Transformer decoders, where the first decoder models response generation and the second serves as a regularizer and jointly models response generation and consistency understanding. The proposed framework can benefit from the stacked encoder and decoders to learn from much smaller personalized dialogue data while maintaining competitive performance. Under different low-resource settings, subjective and objective evaluations prove that the stack-propagation framework outperforms strong baselines in response quality and persona consistency and largely overcomes the shortcomings of traditional models that rely heavily on the persona-dense dialogue data.

Open-Domain Response Generation in Low-Resource Settings using Self-Supervised Pre-Training of Warm-Started Transformers

Learning response generation models constitute the main component of building open-domain dialogue systems. However, training open-domain response generation models requires large amounts of labeled data and pre-trained language generation models that are often nonexistent for low-resource languages. In this article, we propose a framework for training open-domain response generation models in low-resource settings. We consider Dialectal Arabic (DA) as a working example. The framework starts by warm-starting a transformer-based encoder-decoder with pre-trained language model parameters. Next, the resultant encoder-decoder model is adapted to DA by employing self-supervised pre-training on large-scale unlabeled data in the desired dialect. Finally, the model is fine-tuned on a very small labeled dataset for open-domain response generation. The results show significant performance improvements on three spoken Arabic dialects after adopting the framework’s three stages, highlighted by higher BLEU and lower Perplexity scores compared with multiple baseline models. Specifically, our models are capable of generating fluent responses in multiple dialects with an average human-evaluated fluency score above 4. Our data is made publicly available.

An effective emotion tendency perception model in empathic dialogue.

The effectiveness of open-domain dialogue systems depends heavily on emotion. In dialogue systems, previous models primarily detected emotions by looking for emotional words embedded in sentences. However, they did not precisely quantify the association of all words with emotions, which has led to a certain bias. To overcome this issue, we propose an emotion tendency perception model. The model uses an emotion encoder to accurately quantify the emotional tendencies of all words. Meanwhile, it uses a shared fusion decoder to equip the decoder with the sentiment and semantic capabilities of the encoder. We conducted extensive evaluations on Empathetic Dialogue. Experimental results demonstrate its efficacy. Compared with the state of the art, our approach has distinctive advantages.

Keyword-Aware Transformers Network for Chinese Open-Domain Conversation Generation

The open-domain conversation generation task aims to generate contextually relevant and informative responses based on a given conversation history. A critical challenge in open-domain dialogs is the tendency of models to generate safe responses. Existing work has often incorporated keyword information in the conversation history for response generation to relieve this problem. However, these approaches interact weakly between responses and keywords or ignore the association between keyword extraction and conversation generation. In this paper, we propose a method based on a Keyword-Aware Transformers Network (KAT) that can fuse contextual keywords. Specifically, the model enables keywords and contexts to fully interact with responses for keyword semantic enhancement. We jointly model the keyword extraction task and the dialog generation task in a multi-task learning fashion. Experimental results of two Chinese open-domain dialogue datasets showed that our proposed model outperformed the methods in both semantic and non-semantic evaluation metrics, improving Coherence, Fluency, and Informativeness in manual evaluation.

EVA2.0: Investigating Open-domain Chinese Dialogue Systems with Large-scale Pre-training

Large-scale pre-training has shown remarkable performance in building open-domain dialogue systems. However, previous works mainly focus on showing and evaluating the conversational performance of the released dialogue model, ignoring the discussion of some key factors towards a powerful human-like chatbot, especially in Chinese scenarios. In this paper, we conduct extensive experiments to investigate these under-explored factors, including data quality control, model architecture designs, training approaches, and decoding strategies. We propose EVA2.0, a large-scale pre-trained open-domain Chinese dialogue model with 2.8 billion parameters, and will make our models and codes publicly available. Automatic and human evaluations show that EVA2.0 significantly outperforms other open-source counterparts. We also discuss the limitations of this work by presenting some failure cases and pose some future research directions on large-scale Chinese open-domain dialogue systems.

EmoKbGAN: Emotion controlled response generation using Generative Adversarial Network for knowledge grounded conversation.

Neural open-domain dialogue systems often fail to engage humans in long-term interactions on popular topics such as sports, politics, fashion, and entertainment. However, to have more socially engaging conversations, we need to formulate strategies that consider emotion, relevant-facts, and user behaviour in multi-turn conversations. Establishing such engaging conversations using maximum likelihood estimation (MLE) based approaches often suffer from the problem of exposure bias. Since MLE loss evaluates the sentences at the word level, we focus on sentence-level judgment for our training purposes. In this paper, we present a method named EmoKbGAN for automatic response generation that makes use of the Generative Adversarial Network (GAN) in multiple-discriminator settings involving joint minimization of the losses provided by each attribute specific discriminator model (knowledge and emotion discriminator). Experimental results on two bechmark datasets i.e the Topical Chat and Document Grounded Conversation dataset yield that our proposed method significantly improves the overall performance over the baseline models in terms of both automated and human evaluation metrics, asserting that the model can generate fluent sentences with better control over emotion and content quality.