Spoken dialogue understanding using an incremental speech understanding method

Abstract

In spoken dialogue, word sequences that span multiple user utterances may convey one meaning when they are interpreted together as a group and a different meaning when each utterance is interpreted in turn. Natural language processing methods that can deal with units that span multiple utterances are needed for such input. However, in order to enable a spoken dialogue system to respond quickly to the user, it is desirable that interpretations are determined on a per-utterance basis as each utterance is given. In order to satisfy both demands simultaneously in this paper we present an incremental speech understanding method and the ISSS algorithm that is one implementation of this method. The incremental speech understanding method that we propose involves concatenating speech recognition results with previously obtained recognition results, enumerating a list of speech understanding hypotheses that can be derived from these results, ordering these using a priority measure that combines both separate priority measures assigned by both the speech and language processing modules to determine the most probable understanding result. In practice, this is implemented using the N-best hypotheses from the speech recognition system; we avoid an exhaustive enumeration of understanding hypotheses, and adopt a beam-search in the ISSS algorithm that we formulate here. In addition, in order to evaluate the proposed method, we also present a new evaluation metric for speech understanding methods that takes context into account, and have evaluated our proposed method using data gathered from human–computer dialogues. We have compared the method to a speech understanding method that does not consider word sequences that span multiple utterances and show that our proposed method results in statistically significant improvements in speech understanding accuracy. © 2005 Wiley Periodicals, Inc. Syst Comp Jpn, 36(12): 75–84, 2005; Published online in Wiley InterScience (www.interscience.wiley.com). DOI 10.1002/scj.20211