Reformulation of symptom descriptions in dialogue systems for fault diagnosis: How to ask for clarification?

Abstract

• A dialogue system's reformulations of ambiguous user utterances influence performance. • Common terms reduce time and verbal effort but tend to be misunderstood when incorrect. • Technical terms are not accepted uncritically but lead to more clarification questions. • Visual information mitigates the problem of increased error rates for common terms. • Verbal alignment with the dialogue system is low but increases with technical terms. Psycholinguistic research can inform the design of dialogue systems for fault diagnosis. When users provide ambiguous symptom descriptions, dialogue systems can reformulate these descriptions to check the correctness of their interpretation. The present study investigated whether such reformulations should be performed by users or dialogue systems, and how they should be phrased. In a Wizard-of-Oz study, subjects described faults symptoms to a chatbot, which subsequently asked for clarification. Experiment 1 compared the effects of requests for subjects to self-correct their descriptions and reformulations provided by the dialogue system in either common or technical terms. Experiment 2 combined reformulations in common and technical terms with each other and with pictures of fault symptoms. The results revealed that requests for self-correction increased solution times and verbal effort, that common terms decreased solution times but led to errors when seemingly easy reformulations were incorrect, and that technical terms did not mislead subjects into accepting them uncritically. Enrichments with pictures reduced the risk of accepting incorrect reformulations and were considered particularly helpful when combined with common terms. Lexical alignment with dialogue system reformulations was low, but subjects adopted its technical terms most readily when no common terms were provided. Taken together, the results suggest that combining reformulations in everyday language with visual information is most suitable to support grounding.