Task-oriented Dialogue System Research Articles

From easy to hard: Improving personalized response generation of task-oriented dialogue systems by leveraging capacity in open-domain dialogues

A task-oriented dialogue system (TOD) is an important application of artificial intelligence. In the past few years, works on personalized TODs have attracted increased research attention and have seen much progress. The main challenge of such dialogue systems is finding ways to exploit user profiles under conditions of fixed and monolithic dialogue style. However, most of the existing works overlook the observation that the personalization capability of the dialogue system is fundamental and generic, and they treat all attributes of the user profile equally throughout the dialogue flows, which makes them inadequate for developing a well-performing personalized TOD. In this paper, we propose a two-stage learning framework equipped with GPT2 as a backbone to alleviate the above two problems. In the first learning phase, we finetune the GPT2 model on the personalized open-domain dialogues to preliminarily acquire personalization power. Then, we transfer this power to personalized task-oriented dialogues for the second stage of learning. After that, we enable the proposed model to be capable of the desired personalization capacity. Moreover, we present a dynamic profile fusion mechanism and an auxiliary task that detects which attributes contribute to the current utterance to facilitate the model’s performance. Eventually, we rewrite the attribute descriptions of user profiles in sentences to mitigate the consistency gap between the open-domain and task-oriented dialogues. The experimental results show that the proposed model achieves superior results compared to the state-of-the-art models on two versions of the Personalized bAbI dataset.

Revisiting clustering for efficient unsupervised dialogue structure induction

In the development of a task-oriented dialogue system, defining the dialogue structure is a time-consuming task. Hence, several works have looked into automatically inferring it from data, e.g., actual conversations between a customer and a support agent. To recover such dialogue structure, recent methods based on discrete variational models learn to jointly encode and cluster utterances in dialogue states, but (i) represent utterances by only considering preceding dialogue context, and (ii) are slow to train since they are optimized with a compute-expensive decoding objective. We revisit and improve upon an existing efficient pipeline approach, commonly adopted as a baseline, that first encodes utterances and then clusters them with k-means to induce the dialogue structure. However, the existing approach represents utterances as bag-of-words or skip-thought vectors, which have been shown to perform poorly in semantic similarity tasks, and without considering dialogue context. We therefore first investigate the use of more powerful transformer-based encoders for encoding utterances. Next, we propose ellodar, a method for learning representations that capture both preceding and subsequent dialogue context, inspired by word-to-vec training strategies. ellodar is efficient since representations are learned directly in the encoding space by finetuning just a single linear layer on top of a frozen sentence encoder with a vector-to-vector regression training objective. Extensive experiments on representative datasets for dialogue structure induction (SimDial, Schema Guided Dialogues, DSTC2, and CamRest676) demonstrate that in terms of effectiveness to induce the correct dialogue structure, (i) clustering utterances represented by transformed-based encoders improves recent joint models by 13%–32% on standard cluster metrics, and (ii) clustering ellodar’s representations yields additional improvements ranging from +20% to +26%, with speedups of ×\\documentclass[12pt]{minimal} \\usepackage{amsmath} \\usepackage{wasysym} \\usepackage{amsfonts} \\usepackage{amssymb} \\usepackage{amsbsy} \\usepackage{mathrsfs} \\usepackage{upgreek} \\setlength{\\oddsidemargin}{-69pt} \\begin{document}$$\ imes $$\\end{document} 10\\documentclass[12pt]{minimal} \\usepackage{amsmath} \\usepackage{wasysym} \\usepackage{amsfonts} \\usepackage{amssymb} \\usepackage{amsbsy} \\usepackage{mathrsfs} \\usepackage{upgreek} \\setlength{\\oddsidemargin}{-69pt} \\begin{document}$$\ extbf{10}$$\\end{document}–104\\documentclass[12pt]{minimal} \\usepackage{amsmath} \\usepackage{wasysym} \\usepackage{amsfonts} \\usepackage{amssymb} \\usepackage{amsbsy} \\usepackage{mathrsfs} \\usepackage{upgreek} \\setlength{\\oddsidemargin}{-69pt} \\begin{document}$$\ extbf{10}^{\ extbf{4}}$$\\end{document} compared to the recent joint models.

KMc-ToD: Structure knowledge enhanced multi-copy network for task-oriented dialogue system

Task-oriented dialogue (ToD) system aims to assist users in completing various tasks, which has attracted great interest from researchers. However, the current models introduce delexicalization prepossessing to improve the generalization ability of the models, which makes it difficult to integrate dialogue slots into response generation, resulting in unsatisfactory performance for users. In this paper, we propose a structure knowledge-enhanced multi-copy network (KMc-ToD) for the ToD system, which uses the multi-copy mechanism to selectively copy dialogue slots from the dialogue history and schema graph into the response. Furthermore, to select the appropriate slots, we design the schema graph that helps the model understand the relation between different slots. Specifically, this graph adjusts the weights between nodes according to different utterances to prevent introducing noise. Experimental results on the MultiWOZ datasets show that our model achieves promising performance.

Personality prediction from task-oriented and open-domain human–machine dialogues

If a dialogue system can predict the personality of a user from dialogue, it will enable the system to adapt to the user’s personality, leading to better task success and user satisfaction. In a recent study, personality prediction was performed using the Myers–Briggs Type Indicator (MBTI) personality traits with a task-oriented human–machine dialogue using an end-to-end (neural-based) system. However, it is still not clear whether such prediction is generally possible for other types of systems and user personality traits. To clarify this, we recruited 378 participants, asked them to fill out four personality questionnaires covering 25 personality traits, and had them perform three rounds of human–machine dialogue with a pipeline task-oriented dialogue system or an end-to-end task-oriented dialogue system. We also had another 186 participants do the same with an open-domain dialogue system. We then constructed BERT-based models to predict the personality traits of the participants from the dialogues. The results showed that prediction accuracy was generally better with open-domain dialogue than with task-oriented dialogue, although Extraversion (one of the Big Five personality traits) could be predicted equally well for both open-domain dialogue and pipeline task-oriented dialogue. We also examined the effect of utilizing different types of dialogue on personality prediction by conducting a cross-comparison of the models trained from the task-oriented and open-domain dialogues. As a result, we clarified that the open-domain dialogue cannot be used to predict personality traits from task-oriented dialogue, and vice versa. We further analyzed the effects of system utterances, task performance, and the round of dialogue with regard to the prediction accuracy.

A tag-based methodology for the detection of user repair strategies in task-oriented conversational agents

Mutual comprehension is a crucial component that makes a conversation succeed. While it can be easily reached through the cooperation of the parties in human–human dialogues, such cooperation is often lacking in human–computer interaction due to technical problems, leading to broken conversations. Our goal is to work towards an effective detection of breakdowns in a conversation between humans and Conversational Agents (CA), as well as the different repair strategies users adopt when such communication problems occur. In this work, we propose a novel tag system designed to map and classify users’ repair attempts while interacting with a CA. We subsequently present a set of Machine Learning models11https://github.com/rogerferrod/boht. trained to automatize the detection of such repair strategies. The tags are employed in a manual annotation exercise, performed on a publicly available dataset 22The dataset is also available at https://github.com/rogerferrod/boht. of text-based task-oriented conversations. The batch of annotated data was then used to train the neural network-based classifiers. The analysis of the annotations provides interesting insights about users’ behaviour when dealing with breakdowns in a task-oriented dialogue system. The encouraging results obtained from neural models confirm the possibility of automatically recognizing occurrences of misunderstanding between users and CAs on the fly.

A plug-and-play adapter for consistency identification in task-oriented dialogue systems

Task-oriented Dialogue system (ToD) has gained significant attention due to its aim to assist users in accomplishing various tasks. However, the neural network-based dialogue system is like a black box, which may lead to erroneous responses and result in an unfriendly user experience. To address this issue, consistency identification is proposed to prevent generating inconsistent responses. However, the existing consistency identification methods require frequent interaction with the knowledge base, making them susceptible to the introduction of noise during the knowledge base fusion process, ultimately leading to a decline in performance. In this paper, we propose a plug-and-play method for consistency identification, which can introduce external knowledge into the internal reasoning process of the pre-trained language model (PLM) without modifying PLM’s structure. Additionally, we design a new fusion mechanism that effectively fuses the knowledge base information related to the current utterance, which helps the model avoid introducing noise from the irrelevant knowledge base. The experimental results demonstrate that our method achieves state-of-the-art performance on the consistency identification task, improving F1 scores by 2.9% absolute points over the previous methods. Finally, we investigate different knowledge base fusion methods and provide extensive experiments to show the advantages of our proposed method.

Unified Approach for Scalable Task-Oriented Dialogue System

Unified Approach for Scalable Task-Oriented Dialogue System

PluDG: enhancing task-oriented dialogue system with knowledge graph plug-in module

PluDG: enhancing task-oriented dialogue system with knowledge graph plug-in module

A task-oriented neural dialogue system capable of knowledge accessing

As one of the most important AI applications, task-oriented dialogue systems have been a hot topic in this field. These systems are primarily designed to fulfill tasks and provide answers to queries, such as checking the weather or booking flights. However, there are two major challenges in implementing task-oriented dialogue systems: selecting the right knowledge from the relevant knowledge base and combining the knowledge to produce grammatically correct and fluent replies. In this paper, we present a task-oriented dialogue system using an end-to-end trainable neural model. This dialogue system consists of four components: an encoder for encoding the dialogue history, a three-hop memory network for storing the relevant knowledge entries and dialogue history words, a knowledge screener for selecting the knowledge that is really related to the dialogue history, and a decoder for generating the replies. Finally, we validate the proposed dialogue system on a benchmark dataset (KVRET). The experimental results show that the proposed model generates more accurate replies than baselines in automatic and human evaluations. Also, we verify the superiorities of the two improvements by ablation experiments.

Parameter-Efficient Fine-Tuning Method for Task-Oriented Dialogue Systems

The use of Transformer-based pre-trained language models has become prevalent in enhancing the performance of task-oriented dialogue systems. These models, which are pre-trained on large text data to grasp the language syntax and semantics, fine-tune the entire parameter set according to a specific task. However, as the scale of the pre-trained language model increases, several challenges arise during the fine-tuning process. For example, the training time escalates as the model scale grows, since the complete parameter set needs to be trained. Furthermore, additional storage space is required to accommodate the larger model size. To address these challenges, we propose a new new task-oriented dialogue system called PEFTTOD. Our proposal leverages a method called the Parameter-Efficient Fine-Tuning method (PEFT), which incorporates an Adapter Layer and prefix tuning into the pre-trained language model. It significantly reduces the overall parameter count used during training and efficiently transfers the dialogue knowledge. We evaluated the performance of PEFTTOD on the Multi-WOZ 2.0 dataset, a benchmark dataset commonly used in task-oriented dialogue systems. Compared to the traditional method, PEFTTOD utilizes only about 4% of the parameters for training, resulting in a 4% improvement in the combined score compared to the existing T5-based baseline. Moreover, PEFTTOD achieved an efficiency gain by reducing the training time by 20% and saving up to 95% of the required storage space.

Towards Complex Scenarios: Building End-to-End Task-Oriented Dialogue System across Multiple Knowledge Bases

With the success of the sequence-to-sequence model, end-to-end task-oriented dialogue systems (EToDs) have obtained remarkable progress. However, most existing EToDs are limited to single KB settings where dialogues can be supported by a single KB, which is still far from satisfying the requirements of some complex applications (multi-KBs setting). In this work, we first empirically show that the existing single-KB EToDs fail to work on multi-KB settings that require models to reason across various KBs. To solve this issue, we take the first step to consider the multi-KBs scenario in EToDs and introduce a KB-over-KB Heterogeneous Graph Attention Network (KoK-HAN) to facilitate model to reason over multiple KBs. The core module is a triple-connection graph interaction layer that can model different granularity levels of interaction information across different KBs (i.e., intra-KB connection, inter-KB connection and dialogue-KB connection). Experimental results confirm the superiority of our model for multiple KBs reasoning.

Class Incremental Learning for Task-Oriented Dialogue System with Contrastive Distillation on Internal Representations (Student Abstract)

The ability to continually learn over time by grasping new knowledge and remembering previously learned experiences is essential for developing an online task-oriented dialogue system (TDS). In this paper, we work on the class incremental learning scenario where the TDS is evaluated without specifying the dialogue domain. We employ contrastive distillation on the intermediate representations of dialogues to learn transferable representations that suffer less from catastrophic forgetting. Besides, we provide a dynamic update mechanism to explicitly preserve the learned experiences by only updating the parameters related to the new task while keeping other parameters fixed. Extensive experiments demonstrate that our method significantly outperforms the strong baselines.

Balanced Meta Learning and Diverse Sampling for Lifelong Task-Oriented Dialogue Systems

In real-world scenarios, it is crucial to build a lifelong taskoriented dialogue system (TDS) that continually adapts to new knowledge without forgetting previously acquired experiences. Existing approaches mainly focus on mitigating the catastrophic forgetting in lifelong TDS. However, the transfer ability to generalize the accumulated old knowledge to new tasks is underexplored. In this paper, we propose a two-stage lifelong task-oriented dialogue generation method to mitigate catastrophic forgetting and encourage knowledge transfer simultaneously, inspired by the learning process. In the first stage, we learn task-specific masks which adaptively preserve the knowledge of each visited task so as to mitigate catastrophic forgetting. In this stage, we are expected to learn the task-specific knowledge which is tailored for each task. In the second stage, we bring the knowledge from the encountered tasks together and understand thoroughly. To this end, we devise a balanced meta learning strategy for both forward and backward knowledge transfer in the lifelong learning process. In particular, we perform meta-update with a meta-test set sampled from the current training data for forward knowledge transfer. In addition, we employ an uncertainty-based sampling strategy to select and store representative dialogue samples into episodic memory and perform meta-update with a meta-test set sampled from the memory for backward knowledge transfer. With extensive experiments on 29 tasks, we show that MetaLTDS outperforms the strong baselines in terms of both effectiveness and efficiency. For reproducibility, we submit our code at: https: //github.com/travis-xu/MetaLTDS.

Joint Intent Detection Model for Task-oriented Human-Computer Dialogue System using Asynchronous Training

How to accurately understand low-resource languages is the core of the task-oriented human-computer dialogue system. Language understanding consists of two sub-tasks, i.e., intent detection and slot filling. Intent detection still faces challenges due to semantic ambiguity and implicit intentions with users’ input. Moreover, separately modeling intent detection and slot filling significantly decrease the correctness and relevance between questions and answers. To address these issues, we propose a joint intent detection method using asynchronous training strategy. The proposed method firstly encodes local text information extracted by CNN and relationship information among words emphasized by attention structure. Later, a joint intent detection model with asynchronous training strategy is proposed by either fusing hidden states of intent detection and slot filling layers, or adopting the key information to fine-tune the whole network, greatly increasing the relevance of intent detection and slot filling subtasks. The accuracy achieved by the proposed method tested on an open-source airline travel dataset and a self-collected electricity service dataset, i.e., ATIS and ECSF, are 97.49% and 89.68%, respectively, which proves the effectiveness of joint learning and asynchronous training.

Enriching the dialogue state tracking model with a asyntactic discourse graph

Dialogue state tracking is a key component of a task-oriented dialogue system. Recently, pretrained language models are widely used to track a dialogue state by generating or extracting values from a dialogue. These models have a shortcoming that they can explicitly encode contextual information of a dialogue. In this paper, we present a Syntactically ENriched Discourse Dialogue State Tracking model (SEND-DST) that fully leverages the discourse and syntax information of a dialogue. Our model consists of two parts: a dialogue encoding module and a slot-value extraction module. To provide a model with syntactic structures of utterances and the discourse flow of a dialogue, we devise a Syntactic Discourse (SD) graph using the dependency trees of the utterances. Experimental results show that SEND-DST achieves state-of-the-art results on multiple datasets, such as MultiWOZ 2.1, demonstrating its effectiveness.

Efficient structured reporting in radiology using an intelligent dialogue system based on speech recognition and natural language processing

BackgroundStructured reporting (SR) is recommended in radiology, due to its advantages over free-text reporting (FTR). However, SR use is hindered by insufficient integration of speech recognition, which is well accepted among radiologists and commonly used for unstructured FTR. SR templates must be laboriously completed using a mouse and keyboard, which may explain why SR use remains limited in clinical routine, despite its advantages. Artificial intelligence and related fields, like natural language processing (NLP), offer enormous possibilities to facilitate the imaging workflow. Here, we aimed to use the potential of NLP to combine the advantages of SR and speech recognition.ResultsWe developed a reporting tool that uses NLP to automatically convert dictated free text into a structured report. The tool comprises a task-oriented dialogue system, which assists the radiologist by sending visual feedback if relevant findings are missed. The system was developed on top of several NLP components and speech recognition. It extracts structured content from dictated free text and uses it to complete an SR template in RadLex terms, which is displayed in its user interface. The tool was evaluated for reporting of urolithiasis CTs, as a use case. It was tested using fictitious text samples about urolithiasis, and 50 original reports of CTs from patients with urolithiasis. The NLP recognition worked well for both, with an F1 score of 0.98 (precision: 0.99; recall: 0.96) for the test with fictitious samples and an F1 score of 0.90 (precision: 0.96; recall: 0.83) for the test with original reports.ConclusionDue to its unique ability to integrate speech into SR, this novel tool could represent a major contribution to the future of reporting.

A Knowledge-Grounded Task-Oriented Dialogue System with Hierarchical Structure for Enhancing Knowledge Selection

For a task-oriented dialogue system to provide appropriate answers to and services for users' questions, it is necessary for it to be able to utilize knowledge related to the topic of the conversation. Therefore, the system should be able to select the most appropriate knowledge snippet from the knowledge base, where external unstructured knowledge is used to respond to user requests that cannot be solved by the internal knowledge addressed by the database or application programming interface. Therefore, this paper constructs a three-step knowledge-grounded task-oriented dialogue system with knowledge-seeking-turn detection, knowledge selection, and knowledge-grounded generation. In particular, we propose a hierarchical structure of domain-classification, entity-extraction, and snippet-ranking tasks by subdividing the knowledge selection step. Each task is performed through the pre-trained language model with advanced techniques to finally determine the knowledge snippet to be used to generate a response. Furthermore, the domain and entity information obtained because of the previous task is used as knowledge to reduce the search range of candidates, thereby improving the performance and efficiency of knowledge selection and proving it through experiments.

Bridge Pre-Training and Clustering: A Unified Contrastive Learning Framework for OOD Intent Discovery

Discovering Out-of-Domain (OOD) intents is essential for developing new skills in a task-oriented dialogue system. Previous methods suffer from poor knowledge transferability from in-domain (IND) intents to OOD intents, and inefficient iterative clustering. In this paper, we propose an efficient unified contrastive learning framework to discover OOD intents, bridging the gap between IND pre-training stage and OOD clustering stage. Specifically, we employ a supervised contrastive learning (SCL) objective to learn discriminative pre-trained intent features for clustering. And we introduce an efficient end-to-end contrastive clustering method to jointly learn representations and cluster assignments. Besides, we propose an adaptive contrastive learning (ACL) method to automatically adjust the weights of different negative sample pairs for a given anchor according to their semantic similarities. Extensive experiments on two benchmark datasets show that our method is more robust and achieves substantial improvements over the state-of-the-art methods.

Tracking Dialog States in Goal-Oriented Dialogues using a BERT-Based Siamese Network

A dialogue state tracker is a component in a task-oriented dialogue system that monitors the current state of a conversation and gives information about its context and history to other system components. The dynamic and open-ended character of human interactions is one of the primary obstacles in dialogue state monitoring, necessitating robust and adaptable models to keep up with the quick context changes. Recently, numerous deep learning-based algorithms have been developed for this purpose. Still, these models are typically heavily-engineered and conceptually sophisticated, making them challenging to deploy, debug, and maintain in a production environment. To overcome these challenges, we offer the BERT-SIAM-DST model, a unique way to dialogue state monitoring employing a Siamese network with BERT as the base network. This model uses the robust representation capabilities of BERT and the ability of Siamese networks to record correlations between inputs to make accurate predictions regarding the current state of the discussion. In addition, the number of parameters does not increase proportionally with the size of the ontology, and the model is adaptable to alterations in the domain ontology. We test the performance of the BERT-SIAM-DST model on the standard WoZ 2.0 dataset of annotated dialogues and compare it to other approaches. Compared to numerous baseline models, the BERT-SIAM-DST model is effective at tracking the state of discussions, demonstrating the promise of BERT-based Siamese networks for this purpose.

Incorporating BERT With Probability-Aware Gate for Spoken Language Understanding

Spoken language understanding (SLU) is an essential part of a task-oriented dialogue system, which mainly includes intent detection and slot filling. Some existing approaches obtain enhanced semantic representation by establishing the correlation between two tasks. However, those methods show little improvement when applied to BERT, since BERT has learned rich semantic features. In this paper, we propose a BERT-based model with the probability-aware gate mechanism, called PAGM ( <underline xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">P</u> robability <underline xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">A</u> ware <underline xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">G</u> ated <underline xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">M</u> odel). PAGM aims to learn the correlation between intent and slot from the perspective of probability distribution, which explicitly utilizes intent information to guide slot filling. Besides, in order to efficiently incorporate BERT with the probability-aware gate, we design the stacked fine-tuning strategy. This approach introduces a mid-stage before target model training, which enables BERT to get better initialization for final training. Experiments show that PAGM achieves significant improvement on two benchmark datasets, and outperforms the previous state-of-the-art results.