Student Evaluations of a (Rude) Spoken Dialogue System Insights from an Experimental Study

Regina Jucks,Benjamin Brummernhenrich,Gesa A Linnemann

doi:10.1155/2018/8406187

Regina Jucks, Benjamin Brummernhenrich + Show 1 more

Open Access

https://doi.org/10.1155/2018/8406187

Copy DOI

Abstract

Communicating with spoken dialogue systems (SDS) such as Apple’s Siri® and Google’s Now is becoming more and more common. We report a study that manipulates an SDS’s word use with regard to politeness. In an experiment, 58 young adults evaluated the spoken messages of our self-developed SDS as it replied to typical questions posed by university freshmen. The answers were either formulated politely or rudely. Dependent measures were both holistic measures of how students perceived the SDS as well as detailed evaluations of each single answer. Results show that participants not only evaluated the content of rude answers as being less appropriate and less pleasant than the polite answers, but also evaluated the rude system as less accurate. Lack of politeness also impacted aspects of the perceived trustworthiness of the SDS. We conclude that users of SDS expect such systems to be polite, and we then discuss some practical implications for designing SDS.

Highlights

Advances in speech recognition move our interactions with spoken dialogue systems (SDS), such as Apple’s Siri or Google ever closer to human dialogue
With hypothesis 1, we assumed that a polite SDS would be judged as more likable and polite responses as more appropriate and pleasant than a rude SDS
Polite responses were perceived as more appropriate (M = 6.53, SE = 0.28) than rude responses (M = 5.59, SE = 0.28), F(1,55)=13.59, p

Summary

Introduction

Advances in speech recognition move our interactions with spoken dialogue systems (SDS), such as Apple’s Siri or Google ever closer to human dialogue. Over and above the capability of conversing in natural language, one strain of development concerns genuinely social aspects of human communication, such as alignment, addressing by name, and politeness. This paper addresses how students evaluate the communication behavior of an SDS that employs the social strategies of politeness and rudeness. Their evaluations address variables such as acceptance, appropriateness, and competence, all of which are relevant in evaluating human behavior. We briefly address the technological aspects of SDS. We provide results of our empirical research on the perception of SDS, which offer insights about how to design them effectively

Objectives

Results

Conclusion