A Bichannel Transformer with Context Encoding for Document-Driven Conversation Generation in Social Media

Yuanyuan Cai,Haitao Xiong,Min Zuo,Ke Li,Qingchuan Zhang

doi:10.1155/2020/3710104

Abstract

Along with the development of social media on the internet, dialogue systems are becoming more and more intelligent to meet users’ needs for communication, emotion, and social intercourse. Previous studies usually use sequence-to-sequence learning with recurrent neural networks for response generation. However, recurrent-based learning models heavily suffer from the problem of long-distance dependencies in sequences. Moreover, some models neglect crucial information in the dialogue contexts, which leads to uninformative and inflexible responses. To address these issues, we present a bichannel transformer with context encoding (BCTCE) for document-driven conversation. This conversational generator consists of a context encoder, an utterance encoder, and a decoder with attention mechanism. The encoders aim to learn the distributed representation of input texts. The multihop attention mechanism is used in BCTCE to capture the interaction between documents and dialogues. We evaluate the proposed BCTCE by both automatic evaluation and human judgment. The experimental results on the dataset CMU_DoG indicate that the proposed model yields significant improvements over the state-of-the-art baselines on most of the evaluation metrics, and the generated responses of BCTCE are more informative and more relevant to dialogues than baselines.

Highlights

Dialogue systems such as Siri, Cortana, and Duer have been widely used to facilitate interactions between humans and intelligence devices as virtual assistants and social Chatbots
We conduct experiments on the conversation generation task regarding many metrics, including BLEU [27], METEOR [28], NW [24], and perplexity [19]. e experimental results indicate that the proposed model significantly outperforms the state-of-the-art methods
E contributions of this work are as follows: We propose a novel bichannel transformer with context encoding (BCTCE) based on the transformer framework to build an encoder-decoder generator for document-driven conversation. e experimental results show that our model achieves new state-of-the-art performance

Summary

Introduction

Dialogue systems such as Siri, Cortana, and Duer have been widely used to facilitate interactions between humans and intelligence devices as virtual assistants and social Chatbots. Conversational response generation, as a challenging task in natural language processing, plays a critical role in conversational systems. Conversational generation aims to produce grammatical, coherent, and plausible responses in accordance with the input from users. Previous studies on dialogue generation mainly focus on either one-round conversation [1] or multiturn conversation [2]. One-round conversation tasks commonly determine responses on the basis of a single current query, while a multiturn conversation that consists of context-message-response triples commonly builds context-sensitive generators according to the dialogue history [2, 3]. Multiturn conversation tasks tend to generate a variety of correlative responses in either goal-driven customer services [4,5,6] or chitchat without predefined goals [7]

Methods

Results

Conclusion