Empirical Study on the Effect of Achievement Badges in TRAKLA2 Online Learning Environment

Abstract

Achievement badges are a form of gamification that can be used to motivate users and to encourage desired actions. In this study, we describe and evaluate the use of achievement badges in the TRAKLA2 online learning environment where students complete interactive, automatically assessed exercises about data structures and algorithms. The students' activity in TRAKLA2 was logged in order to find out whether the achievement badges had an effect on their behavior. We used a between-subject experimental design where the students (N=281) were randomly divided into a treatment and a control group, with and without achievement badges. Students in the treatment group were awarded achievement badges, for example, for solving exercises with only one attempt, returning exercises early, or completing an exercise round with full points. Course grading was similar for both groups, i.e. collecting badges did not affect the final grade. Our results show that achievement badges can be used to affect the behavior of students even when the badges have no impact on the grading. Statistically significant differences in students' behavior were observed with some badge types, while some badges did not seem to have such an effect. We also found that students in the two studied courses responded differently to the badges. Based on our findings, achievement badges seem like a promising method to motivate students and to encourage desired study practices.