A plug-and-play adapter for consistency identification in task-oriented dialogue systems

Abstract

Task-oriented Dialogue system (ToD) has gained significant attention due to its aim to assist users in accomplishing various tasks. However, the neural network-based dialogue system is like a black box, which may lead to erroneous responses and result in an unfriendly user experience. To address this issue, consistency identification is proposed to prevent generating inconsistent responses. However, the existing consistency identification methods require frequent interaction with the knowledge base, making them susceptible to the introduction of noise during the knowledge base fusion process, ultimately leading to a decline in performance. In this paper, we propose a plug-and-play method for consistency identification, which can introduce external knowledge into the internal reasoning process of the pre-trained language model (PLM) without modifying PLM’s structure. Additionally, we design a new fusion mechanism that effectively fuses the knowledge base information related to the current utterance, which helps the model avoid introducing noise from the irrelevant knowledge base. The experimental results demonstrate that our method achieves state-of-the-art performance on the consistency identification task, improving F1 scores by 2.9% absolute points over the previous methods. Finally, we investigate different knowledge base fusion methods and provide extensive experiments to show the advantages of our proposed method.