Abstract
Conversation modeling is one of most important applications of natural language processing. Building response generation model for open domain conversation in a Chatbot is one of the hardest challenges in this area. The deep neural network architectures such as sequence to sequence models and its hierarchical variants provide a significant improvement in the field of conversation modeling. Although these models require large size corpus, they may cause huge data loss in training phase. Also, these models are unable to concentrate on important data in given context. It affects on generation of responses. To tackle these issues, this research work proposes a Variational Hierarchical Conversation RNN with Attention mechanism (VHCRA) model for response generation. The VHCRA uses the concept of latent variable representation to avoid data degeneracy and the attention mechanism to identify important data within context. The model is trained on large size benchmark dataset, i.e., Cornell Movie Dialog corpus which contains conversations from different movies. The model is evaluated using automatic evaluation metrics such as Negative Log-likelihood and Embedding-Based Metrics. The experimental result shows that the proposed model gains significant improvement in comparison with recently proposed approaches and generate meaningful responses according to the context.
Highlights
RECENT years have seen rapid growth of sequential conversational data on Internet which have become an essential part of natural language processing
In this work we propose a Variational Hierarchical Con-versational RNN with Attention mechanism (VHCRA) which is the combination of variational hierarchical model and attention mechanism to solve the data degeneration problem as well as produce relevant response generation
Experimental result shows that the proposed Variational Hierarchical Conversation RNN with Attention mechanism (VHCRA) model outperforms the various state-of-the art response generation model when evaluating automatically
Summary
RECENT years have seen rapid growth of sequential conversational data on Internet which have become an essential part of natural language processing. The objective of this work is to propose a response generation model for open domain conversation in Chatbot which overcomes the data degeneration problem and produces high quality responses for given context. In this work we propose a Variational Hierarchical Con-versational RNN with Attention mechanism (VHCRA) which is the combination of variational hierarchical model and attention mechanism to solve the data degeneration problem as well as produce relevant response generation. Experimental result shows that the proposed VHCRA model outperforms the various state-of-the art response generation model when evaluating automatically. It prevents model from data degeneration problem by giving stable KL-divergence. The paper ends by giving conclusion in the Section V
Talk to us
Join us for a 30 min session where you can share your feedback and ask us any queries you have