Improving Open-Domain Dialogue Systems via Multi-Turn Incomplete Utterance Restoration

Zhufeng Pan,Kun Bai,Lianqiang Zhou,Yan Wang,Xiaojiang Liu

doi:10.18653/v1/d19-1191

Abstract

In multi-turn dialogue, utterances do not always take the full form of sentences. These incomplete utterances will greatly reduce the performance of open-domain dialogue systems. Restoring more incomplete utterances from context could potentially help the systems generate more relevant responses. To facilitate the study of incomplete utterance restoration for open-domain dialogue systems, a large-scale multi-turn dataset Restoration-200K is collected and manually labeled with the explicit relation between an utterance and its context. We also propose a “pick-and-combine” model to restore the incomplete utterance from its context. Experimental results demonstrate that the annotated dataset and the proposed approach significantly boost the response quality of both single-turn and multi-turn dialogue systems.

Highlights

Dialogue systems have attracted increasing attention due to the promising potentials on applications like virtual assistants or customer support systems (Hauswald et al, 2015; Poulami Debnath, 2018)
Our study shows on average only 17.7% words in previous utterances overlap with the restored utterance, while 100% words in the original utterance are included in the restored utterance
We propose the restoration score to evaluate the performance of incomplete utterance restoration model

Summary

Introduction

Dialogue systems have attracted increasing attention due to the promising potentials on applications like virtual assistants or customer support systems (Hauswald et al, 2015; Poulami Debnath, 2018). Studies (Carbonell, 1983) show that users of dialogue systems tend to use succinct language which often omits entities or concepts made in previous utterances. Dialogue systems must be equipped with the ability to understand these incomplete utterances. Take Example 1 in Table 1 for instance, contents in parentheses are information omitted in the utterance. A3 means what kind of dessert matches B’s taste, instead of what kind of shop B likes. Failing to understand this utterance would be a disaster for dialogue systems to generate a relevant and coherent response. According to our survey (details in Table 2), in about 60% conversations, fully comprehending current utterance depends on previous context. We will refer conversation history (A1 to B2 in above example) as previous utterances, the utterance to be restored (A3) as original utterance, and the complete form of A3 as restored utterance

Methods

Results

Conclusion