Knowledge dialogue generation with multi-head attention mechanism

Abstract

An increasing number of research efforts are focusing on knowledge dialogue generation. Less attention is focused on increasing knowledge diversity in generated responses. A model of knowledge selection guided by a multi-head attention mechanism is proposed. First, the current input discourse and knowledge content are input into the Bi-GRU module to obtain the coding vector, and then obtain multiple aspects of semantics from the user input discourse coding vector based on the multi-head attention mechanism, so as to select different knowledge. A punishment item method is proposed to force different attention to focus on different aspects, and finally, use the user input and selected knowledge for the decoding stage. Experiments with manual and automated evaluations have proven that the model is superior to the baseline model compared to previous work.