I need a CAVAA. How Conversational Agent – Voting Advice Applications (CAVAA) affect users’ political knowledge and tool experience

Abstract

Voting Advice Applications (VAAs) have become an important campaign element in election times. In these survey-tools, users answer attitude questions about political issues and receive a voting advice based on their answers. While the effects of VAAs on political knowledge and political interest are promising, research shows at the same time that VAA users frequently experience comprehension problems when answering the VAA statements and also that they spend only very little effort to resolve these problems. We describe two studies in which we test a new type of VAA, called Conversational Agent VAA (in short: CAVAA) that responds to this processing mode of low elaboration. In a CAVAA, users can access relevant information by asking specific questions about the political statements. The CAVAA, in turn, automatically provides a personalized response that addresses the information need of the user. Study 1 reports about an online experiment (N = 229) with a 2 (Type: traditional VAA or CAVAA) x 2 (Political sophistication: low or high) between-subjects design. Results show that CAVAAs lead to more perceived and factual political knowledge than traditional VAAs. Also, participants’ CAVAA experience was evaluated better. In Study 2 (N = 180) we tested 3 CAVAA designs against one another (a structured design with buttons, an open design were respondents could type in their questions, and a semi-structured design with both buttons and an open text field), again for higher and lower politically sophisticated users. Results of the 3 (CAVAA design) x 2 (Political sophistication) between-subjects experiment show that while the three designs score equally high on factual and perceived knowledge indicators, the experience of the structured CAVAA was evaluated more positively than the open version on usefulness, ease of use and playfulness. In both studies we observed no interaction with the user’s level of political sophistication. The studies’ implications for theory and practice are discussed.