Training a Dialogue Act Tagger for human-human and human-computer travel dialogues

Abstract

While dialogue acts provide a useful schema for characterizing dialogue behaviors in human-computer and human-human dialogues, their utility is limited by the huge effort involved in hand-labelling dialogues with a dialogue act labelling scheme. In this work, we examine whether it is possible to fully automate the tagging task with the goal of enabling rapid creation of corpora for evaluating spoken dialogue systems and comparing them to human-human dialogues. We report results for training and testing an automatic classifier to label the information provider's utterances in spoken human-computer and human-human dialogues with DATE (Dialogue Act Tagging for Evaluation) dialogue act tags. We train and test the DATE tagger on various combinations of the DARPA Communicator June-2000 and October-2001 human-computer corpora, and the CMU human-human corpus in the travel planning domain. Our results show that we can achieve high accuracies on the human-computer data, and surprisingly, that the human-computer data improves accuracy on the human-human data, when only small amounts of human-human training data are available.