Tracking Dialog States in Goal-Oriented Dialogues using a BERT-Based Siamese Network

Abstract

A dialogue state tracker is a component in a task-oriented dialogue system that monitors the current state of a conversation and gives information about its context and history to other system components. The dynamic and open-ended character of human interactions is one of the primary obstacles in dialogue state monitoring, necessitating robust and adaptable models to keep up with the quick context changes. Recently, numerous deep learning-based algorithms have been developed for this purpose. Still, these models are typically heavily-engineered and conceptually sophisticated, making them challenging to deploy, debug, and maintain in a production environment. To overcome these challenges, we offer the BERT-SIAM-DST model, a unique way to dialogue state monitoring employing a Siamese network with BERT as the base network. This model uses the robust representation capabilities of BERT and the ability of Siamese networks to record correlations between inputs to make accurate predictions regarding the current state of the discussion. In addition, the number of parameters does not increase proportionally with the size of the ontology, and the model is adaptable to alterations in the domain ontology. We test the performance of the BERT-SIAM-DST model on the standard WoZ 2.0 dataset of annotated dialogues and compare it to other approaches. Compared to numerous baseline models, the BERT-SIAM-DST model is effective at tracking the state of discussions, demonstrating the promise of BERT-based Siamese networks for this purpose.