Abstract

Natural language generation (NLG) is an important component in spoken dialog systems (SDSs). A model for NLG involves sequence to sequence learning. State-of-the-art NLG models are built using recurrent neural network (RNN) based sequence to sequence models (Ondřej Dusek and Filip Jurcicek, 2016a). Convolutional sequence to sequence based models have been used in the domain of machine translation but their application as Natural Language Generators in dialogue systems is still unexplored. In this work, we propose a novel approach to NLG using convolutional neural network (CNN) based sequence to sequence learning. CNN-based approach allows to build a hierarchical model which encapsulates dependencies between words via shorter path unlike RNNs. In contrast to recurrent models, convolutional approach allows for efficient utilization of computational resources by parallelizing computations over all elements, and eases the learning process by applying constant number of nonlinearities. We also propose to use CNN-based reranker for obtaining responses having semantic correspondence with input dialogue acts. The proposed model is capable of entrainment. Studies using a standard dataset shows the effectiveness of the proposed CNN-based approach to NLG.

Highlights

  • In task-specific spoken dialogue systems (SDS), the function of natural language generation (NLG) components is to generate natural language response from a dialogue act (DA) (Young et al, 2009)

  • We present a novel approach of using convolutional sequence to sequence model (ConvSeq2Seq) for the task of NLG

  • We evaluate our model on the Alex Context natural language generation (NLG) dataset of Dusek and Jurcicek (2016a) and demonstrate that our model outperforms the RNNbased model of Dusek and Jurcicek (2016a) (TGen model) in automatic metrics

Read more

Summary

Introduction

In task-specific spoken dialogue systems (SDS), the function of natural language generation (NLG) components is to generate natural language response from a dialogue act (DA) (Young et al, 2009). Recent advances have been in the direction of developing a fully trainable context aware NLG model (Dusek and Jurcicek, 2016a) All these approaches are based on recurrent sequence to sequence architecture. ConvSeq2Seq generator is an encoder decoder model where convolutional neural networks (CNNs) are used to build both encoder and decoder states CNN reranker implements one dimensional convolution on beam search responses and generates binary vectors These binary vectors are used to penalize the responses having missing and/or irrelevant information. The main contributions of this work are (i) ConvSeq2Seq generator for NLG and (ii) CNN-based reranker for ranking n-best beam search responses for obtaining semantically appropriate responses with respect to input DA.

Related Work
Proposed Approach
ConvSeq2Seq Generator
CNN Reranker
Experimental Studies
Studies of the models using 13a version of the metrics
Studies of the models using 11b version of the metrics
Studies on the models based on training time
Findings
Conclusion and Future Work
Full Text
Published version (Free)

Talk to us

Join us for a 30 min session where you can share your feedback and ask us any queries you have

Schedule a call