Using lexical, syntactic and semantic features for non-terminal grammar rule induction in Spoken Dialogue Systems

Abstract

In this work, we propose an algorithm for the automatic induction of non-terminal grammar rules for Spoken Dialogue Systems (SDS). Initially, a grammar developer provides the system with a minimal set of rules that serve as seeding examples. Using these seed rules and (optionally) a seed corpus, in-domain data are harvested and filtered from the web. A challenging task is identifying relevant chunks (phrases) in the web-harvested corpus that are good candidates for enhancing the seed grammar. We propose and evaluate rule-based and statistical classification algorithms for this purpose that use lexical, syntactic and semantic features. Induced grammars are evaluated in terms of accuracy of the proposed rules for two spoken dialogue domains. Results show up to four times absolute precision improvement compared to the naive grammar induction approach using semantic phrase similarity.