Evaluating the intelligence in dialogue systems

Abstract

The notions of dialogue, dialogue systems and intelligence are explored. Eight general aspects of dialogue and dialogue systems are identified to help describe current efforts at making communication with computers more intelligent. Four evaluative dimensions are discussed on which computer dialogue systems can be assessed, and then this descriptive/evaluative scheme is applied to three example dialogue systems: ELIZA , an early conversational system, GUS , a frame-based dialogue system and GUIDON , a medical diagnostic tutor. The paper ends with a few comments on the current state of evaluative work and future directions for our methodology.