Using memes and emoji-scales in a web survey: experimental assessment of consequences for multimodal cognitive effort and data quality

Abstract

ABSTRACT Decreasing web survey participation rates lead to higher importance of ways to increase respondents’ motivation to participate and gamification is seen as one of possible ways to do so. However, there is no clear understanding on how it affects respondents’ cognitive load. This study investigates whether gamification reduces cognitive load among 18–25-year-old survey respondents. A laboratory experiment was conducted with 128 randomly assigned students to either a non-gamified or gamified web survey design. Gamification was implemented using emojis for ordinal scales and motivating memes between sets of questions. Cognitive load was measured through paradata, subjective evaluation, and neurophysiological indicator (pupil diameter dynamics). Results show higher cognitive effort for the gamification group based on pupil diameter dynamics, but subjective evaluation, mouse movements, and completion times show increased cognitive effort for the non-gamified group. Attention was higher for the gamification group, and data quality did not differ. Therefore, gamification may increase cognitive load, but it also increases motivation and engagement, leading to similar data quality compared to non-gamified web surveys.