User representations of computer systems in human-computer speech interaction

Abstract

We report an experiment designed to study whether models of human-human voice dialogues can be applied successfully to human-computer communication using natural spoken language. Two groups of six subjects were asked to obtain information about air travel via dialogue with a remote "travel agent". Subjects in the computer group were led to believe they were talking to a computer whereas subjects in the operator group were told they were talking to a human. Both groups of subjects actually talked to the same human experimenter. The study focuses on subjects' representations of interlocutor skill and knowledge, and differs from previous analogous studies in several respects: the task is more complex, giving rise to structured exchanges in natural language rather than to question/answer pairs in simplified language; specific attention has been paid to the design, which attempts to avoid biases that have flawed other studies (in particular, conditions are identical for both groups); the time factor has been taken into account (subjects take part in three sessions, at 1-week intervals). Some results confirm those of the literature, namely that subjects of the computer group tend to control and simplify their use of language more than those in the operator group. However, most observations are either new or in contradiction with previous results: subjects in the computer group produce more utterances but no significant differences were observed with respect to most structural and pragmatic features of language; the time factor plays a dual role. Subjects in both groups tend to become more concise. Operator group strategies differ significantly across sessions as regards scenario processing (problem solving) whereas computer group strategies remain stable. These differences in behavior between groups are ascribed to differences in representations of interlocutor ability.